Papers

We explore the utility of information contained within a dropout based Bayesian neural network (BNN) for the task of detecting out of distribution (OOD) data. We first show how previous attempts to leverage the randomized embeddings induced by the intermediate layers of a dropout BNN can fail due to the distance metric used. We introduce an alternative approach to measuring embedding uncertainty, and demonstrate how incorporating embedding uncertainty improves OOD data identification across three tasks: image classification, language classification, and malware detection.

poster session 1 @ blue 5, poster session 11 @ blue 5, oral session 1 @ blue 5, poster session 1, poster session 11, oral session 1

Most of the existing methods for \textit{search result diversification} (SRD) appeal to the \textit{greedy strategy} for generating diversified results, which is formulated as a sequential process of selecting documents one-by-one, and the locally optimal choice is made at each round. Unfortunately, this strategy suffers from the following shortcomings: (1) Such a one-by-one selection process is rather time-consuming for both training and inference. (2) It works well on the premise that the preceding choices are optimal or close to the optimal solution. (3) The mismatch between the objective function used in training and the final evaluation measure used in testing has not been taken into account. We propose a novel framework through direct metric optimization for SRD (referred to as \textit{MO4SRD}) based on the \textit{score-and-sort strategy}. Specifically, we represent the diversity score of each document that determines its rank position based on a probability distribution. These distributions over scores naturally give rise to expectations over rank positions. Armed with this advantage, we can get the differentiable variants of the widely used diversity metrics. Thanks to this, we are able to directly optimize the evaluation measure used in testing. Moreover, we have devised a novel probabilistic neural scoring function. It jointly scores candidate documents by taking into account both cross-document interaction and permutation equivariance, which makes it possible to generate a diversified ranking via a simple sorting. The experimental results on benchmark collections show that the proposed method achieves significantly improved performance over the state-of-the-art results.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

Online travel platforms (OTPs), e.g., bookings.com and Ctrip.com, deliver travel experiences to online users by providing travel-related products. Although much progress has been made, the state-of-the-arts for cold-start problems are largely sub-optimal for user representation, since they do not take into account the unique characteristics exhibited from the user behaviors. In this work, we propose a State-aware Multi-aspect Interests representation Network (SMINet) for cold-start users recommendation at OTPs, which consists of a multi-aspect interests extractor, a co-attention layer, and a state-aware gating layer. The key component of the model is the multi-aspect interests extractor, which is able to extract representations for the user's multi-aspect interests.Furthermore, to learn the interactions between the user behaviors in the current session and the above multi-aspect interests, we carefully design a co-attention layer which allows the cross attentions between the two modules. Additionally, we propose a travel state-aware gating layer to attentively select the multi-aspect interests. The final user representation is obtained by fusing the three components. Comprehensive experiments conducted both offline and online demonstrate the superior performance of the proposed model at user representation, especially for cold-start users, compared with state-of-the-art methods.

poster session 1 @ blue 5, poster session 11 @ blue 5, oral session 11 @ blue 5, poster session 1, poster session 11, oral session 11

Graph walking based on reinforcement learning (RL) has shown great success in navigating an agent to automatically complete various reasoning tasks over an incomplete knowledge graph (KG) by exploring multi-hop relational paths. However, existing multi-hop reasoning approaches only work well on short reasoning paths and tend to miss the target entity with the increasing path length. This is undesirable for many reasoning tasks in real-world scenarios, where short paths connecting the source and target entities are not available in incomplete KGs, and thus the reasoning performances drop drastically unless the agent is able to seek out more clues from longer paths. To address the above challenge, in this paper, we propose a dual-agent reinforcement learning framework, which trains two agents (\ts{giant} and \ts{dwarf}) to walk over a KG \ti{jointly} and search for the answer \ti{collaboratively}. Our approach tackles the reasoning challenge in long paths by assigning one of the agents (\ts{giant}) searching on cluster-level paths quickly and providing stage-wise hints for another agent (\ts{dwarf}). Finally, experimental results on several KG reasoning benchmarks show that our approach can search answers more accurately and efficiently, and outperforms existing RL-based methods for long path queries by a large margin.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

We present a novel dynamic recommendation model that focuses on users who have interactions in the past but turn relatively inactive recently. Making effective recommendations to these {\em time-sensitive cold-start users} is critical to maintain the user base of a recommender system. Due to the sparse recent interactions, it is challenging to capture these users' current preferences precisely. Solely relying on their historical interactions may also lead to outdated recommendations misaligned with their recent interests. The proposed model leverages historical and current user-item interactions and dynamically factorizes a user's (latent) preference into time-specific and time-evolving representations that jointly affect user behaviors. These latent factors further interact with an optimized item embedding to achieve accurate and timely recommendations. Experiments over real-world data help demonstrate the effectiveness of the proposed time-sensitive cold-start recommendation model.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

We propose a new discrete model for simultaneously spreading two opinions within a social network inspired by the famous Target Set Selection problem. We are given a social network, a seed-set of agents for each opinion, and two thresholds per agent. The first threshold represents the willingness of an agent to adopt an opinion if she has no opinion at all, while the second threshold states the readiness to acquire a second opinion. The goal is to add as few agents as possible to the initial seed-sets such that, once the process started with these seed-set stabilises, each agent has either both opinions or none. We perform an initial study of its computational complexity. It is not surprising that the problem is NP-hard even in quite restricted settings. Therefore, we investigate the complexity of the problem from the parameterized point-of-view with special focus on sparse networks, which appears often in practice. Among other things, we show that the proposed problem is in the FPT complexity class if we parameterize by the vertex cover number of the underlying graph.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

We study the problem of predicting an event sequence given some meta data. In particular, we are interested in learning easily interpretable models that can accurately generate a sequence based on an attribute vector. To this end, we propose to learn a sparse event-flow graph over the training sequences, and statistically robust rules that use meta data to determine which paths to follow. We formalize the problem in terms of the Minimum Description Length (MDL) principle, by which we identify the best model as the one that compresses the data best. As the resulting optimization problem is NP-hard, we propose the efficient ConSequence algorithm to discover good event-flow graphs from data. Through an extensive set of experiments including a case study, we show that it ably discovers compact, interpretable and accurate models for the generation and prediction of event sequences from data, has a low sample complexity, and is particularly robust against noise.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

As a decisive part in the success of Mobility-as-a-Service (MaaS), spatio-temporal predictive modeling for crowd movements is a challenging task particularly considering scenarios where societal events drive mobility behavior deviated from the normality. While tremendous progress has been made to model high-level spatio-temporal regularities with deep learning, most, if not all of the existing methods are neither aware of the dynamic interactions among multiple transport modes nor adaptive to unprecedented volatility brought by potential societal events. In this paper, we are therefore motivated to improve the canonical spatio-temporal network (ST-Net) from two perspectives: (1) design a heterogeneous mobility information network (HMIN) to explicitly represent intermodality in multimodal mobility; (2) propose a memory-augmented dynamic filter generator (MDFG) to generate sequence-specific parameters in an on-the-fly fashion for various scenarios. The enhanced event-aware spatio-temporal network, namely EAST-Net, is evaluated on several real-world datasets with a wide variety and coverage of societal events. Both quantitative and qualitative experimental results verify the superiority of our approach compared with the state-of-the-art baselines. Code and data are published on https://github.com/underdoc-wang/EAST-Net.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

Density estimation is a widely used method to perform unsupervised anomaly detection. By learning the density function, data points with relatively low densities are classified as anomalies. Unfortunately, the presence of anomalies in training data may significantly impact the density estimation process, thereby imposing significant challenges to the use of more sophisticated density estimation methods such as those based on deep neural networks. In this work, we propose RobustRealNVP, a deep density estimation framework that enhances the robustness of flow-based density estimation methods, enabling their application to unsupervised anomaly detection. RobustRealNVP differs from existing flow-based models from two perspectives. First, RobustRealNVP discards data points with low estimated densities during optimization to prevent them from corrupting the density estimation process. Furthermore, it imposes Lipschitz regularization to the flow-based model to enforce smoothness in the estimated density function. We demonstrate the robustness of our algorithm against anomalies in training data from both theoretical and empirical perspectives. The results show that our algorithm achieves competitive results as compared to state-of-the-art unsupervised anomaly detection methods.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

Novelty detection aims to automatically identify out-of-distribution (OOD) data, without any prior knowledge of them. It is a critical step in data monitoring, behavior analysis and other applications, helping enable continual learning in the field. Conventional methods of OOD detection perform multi-variate analysis on an ensemble of data or features, and usually resort to the supervision with OOD data to improve the accuracy. In reality, such supervision is impractical as one cannot anticipate the anomalous data. In this paper, we propose a novel, self-supervised approach that does not rely on any pre-defined OOD data: (1) The new method evaluates the Mahalanobis distance of the gradients between the in-distribution and OOD data. (2) It is assisted by a self-supervised binary classifier to guide the label selection to generate the gradients, and maximize the Mahalanobis distance. In the evaluation with multiple datasets, such as CIFAR-10, CIFAR-100, SVHN and TinyImageNet, the proposed approach consistently outperforms state-of-the-art supervised and unsupervised methods in the area under the receiver operating characteristic (AUROC) and area under the precision-recall curve (AUPR) metrics. We further demonstrate that this detector is able to accurately learn one OOD class in continual learning.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

We present a novel movie recommendation system, GRU4RecBE, which extends the GRU4Rec architecture with rich item features extracted by the pre-trained BERT model. GRU4RecBE outperforms state-of-the-art session-based models over the benchmark MovieLens 1m and MovieLens 20m datasets.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

Bundle recommendation aims to recommend the user a bundle of items as a whole. Previous models capture user’s preferences on both items and the association of items. Nevertheless, they usually neglect the diversity of user’s intents on adopting items and fail to disentangle user’s intents in representations. In the real scenario of bundle recommendation, a user’s intent may be naturally distributed in the different bundles of that user (Global view). And a bundle may contain multiple intents of a user (Local view). Each view has its advantages for intent disentangling: 1) In the global view, more items are involved to present each intent, which can demonstrate the user’s preference under each intent more clearly. 2) The local view can reveal the association between items under each intent since the items within the same bundle are highly correlated to each other. To this end, in this paper we propose a novel model named Multi-view Intent Disentangle Graph Networks (MIDGN), which is capable of precisely and comprehensively capturing the diversity of user intent and items’ associations at the finer granularity. Specifically, MIDGN disentangles user’s intents from two different perspectives, respectively: 1) taking the Global view, MIDGN disentangles the user’s intent coupled with inter-bundle items; 2) taking the Local view, MIDGN disentangles the user’s intent coupled with items within each bundle. Meanwhile, we compare user’s intents disentangled from different views by a contrast method to improve the learned intents. Extensive experiments are conducted on two benchmark datasets and MIDGN outperforms the state-of-the-art methods by over 10.7% and 26.8%, respectively.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

We propose a new approach, the calibrated nonparametric scan statistic (CNSS), for more accurate detection of anomalous patterns in large-scale, real-world graphs. Scan statistics identify connected subgraphs that are interesting or unexpected through maximization of a likelihood ratio statistic; in particular, nonparametric scan statistics (NPSSs) identify subgraphs with a higher than expected proportion of individually significant nodes. However, we show that recently proposed NPSS methods are miscalibrated, failing to account for the maximization of the statistic over the multiplicity of subgraphs. This results in both reduced detection power for subtle signals, and low precision of the detected subgraph even for stronger signals. Thus we develop a new statistical approach to recalibrate NPSSs, correctly adjusting for multiple hypothesis testing and taking the underlying graph structure into account. While the recalibration, based on randomization testing, is computationally expensive, we propose both an efficient (approximate) algorithm and new, closed-form lower bounds (on the expected maximum proportion of significant nodes for subgraphs of a given size, under the null hypothesis of no anomalous patterns). These advances, along with the integration of recent core-tree decomposition methods, enable CNSS to scale to large real-world graphs, with substantial improvement in the accuracy of detected subgraphs. Extensive experiments on both semi-synthetic and real-world datasets are demonstrated to validate the effectiveness of our proposed methods, in comparison with state-of-the-art counterparts.

poster session 1 @ blue 5, poster session 11 @ blue 5, oral session 1 @ blue 5, poster session 1, poster session 11, oral session 1

In sequential recommender system applications, it is important to develop models that can capture users' evolving interest over time to successfully recommend future items that they are likely to interact with. For users with long histories, typical models based on recurrent neural networks tend to forget important items in the distant past. Recent works have shown that storing a small sketch of past items can improve sequential recommendation tasks. However, these works all rely on static sketching policies, i.e., heuristics to select items to keep in the sketch, which are not necessarily optimal and cannot improve over time with more training data. In this paper, we propose a differentiable policy for sketching (DiPS), a framework that learns a data-driven sketching policy in an end-to-end manner together with the recommender system model to explicitly maximize recommendation quality in the future. We also propose an approximate estimator of the gradient for optimizing the sketching algorithm parameters that is computationally efficient. We verify the effectiveness of DiPS on real-world datasets under various practical settings and show that it requires up to $50\%$ fewer sketch items to reach the same predictive quality than existing sketching policies.

poster session 1 @ blue 5, poster session 11 @ blue 5, oral session 11 @ blue 5, poster session 1, poster session 11, oral session 11

We introduce the triangle-densest-$k$-subgraph problem (TD$k$S) for undirected graphs: given a size parameter $k$, compute a subset of $k$ vertices that maximizes the number of induced triangles. The problem corresponds to the simplest generalization of the edge based densest-$k$-subgraph problem (D$k$S) to the case of higher-order network motifs. We prove that TD$k$S is NP-hard and is not amenable to efficient approximation, in the worst-case. By judiciously exploiting the structure of the problem, we propose a relaxation algorithm for the purpose of obtaining high-quality, sub-optimal solutions. Our approach utilizes the fact that the cost function of TD$k$S is submodular to construct a convex relaxation for the problem based on the Lov\'asz extension for submodular functions. We demonstrate that our approaches attain state-of-the-art performance on real-world graphs and can offer substantially improved exploration of the optimal density-size curve compared to sophisticated approximation baselines for D$k$S. We use document summarization to showcase why TD$k$S is a useful generalization of D$k$S.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

Answering complex First-Order Logical (FOL) queries on large-scale incomplete knowledge graphs (KGs) is an important yet challenging task. Recent advances embed logical queries and KG entities in the vector space and conduct query answering via dense similarity search. However, most of the designed logical operators in existing works do not satisfy the axiomatic system of classical logic. Moreover, these logical operators are parameterized so that they require a large number of complex FOL queries as training data, which are often arduous or even inaccessible to collect in most real-world KGs. In this paper, we present FuzzQE, a fuzzy logic based query embedding framework for answering FOL queries over KGs. FuzzQE follows fuzzy logic to define logical operators in a principled and learning free manner. Extensive experiments on two benchmark datasets demonstrate that FuzzQE achieves significantly better performance in answering FOL queries compared to the state-of-the-art methods. In addition, FuzzQE trained with only KG link prediction without any complex queries can achieve comparable performance with the systems trained with all FOL queries.

poster session 1 @ blue 5, poster session 11 @ blue 5, oral session 1 @ blue 5, poster session 1, poster session 11, oral session 1

Spatio-temporal forecasting is challenging attributing to the high nonlinearity in temporal dynamics as well as complex location-characterized patterns in spatial domains, especially in fields like weather forecasting. Graph convolutions are usually used for modeling the spatial dependency in meteorology to handle the irregular distribution of sensors' spatial location. In this work, a novel graph-based convolution for imitating the meteorological flows is proposed to capture the local spatial patterns. Based on the assumption of smoothness of location-characterized patterns, we propose conditional local convolution whose shared kernel on nodes' local space is approximated by neural networks, with local representations of coordinate obtained by horizon maps into cylindrical-tangent space as its input. We further propose the distance and orientation scaling terms to reduce the impacts of irregular spatial distribution. The convolution is embedded in a Recurrent Neural Network architecture to model the temporal dynamics, leading to the Conditional Local Convolution Recurrent Network (CLCRN). Our model is evaluated on real-world weather benchmark datasets, achieving state-of-the-art performance with obvious improvements. We conduct further analysis on local pattern visualization, model's framework choice, advantages of horizon maps and etc.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

In many real-world scenarios, we often deal with streaming data that is sequentially collected over time. Due to the non-stationary nature of the environment, the streaming data distribution may change in unpredictable ways, which is known as the concept drift in the literature. To handle concept drift, previous methods first detect when/where the concept drift happens and then adapt models to fit the distribution of the latest data. However, there are still many cases that some underlying factors of environment evolution are predictable, making it possible to model the future concept drift trend of the streaming data, while such cases are not fully explored in previous work. In this paper, we propose a novel method DDG-DA, that can effectively forecast the evolution of data distribution and improve the performance of models. Specifically, we first train a predictor to estimate the future data distribution, then leverage it to generate training samples, and finally train models on the generated data. We conduct experiments on three real-world tasks (forecasting on stock price trend, electricity load and solar irradiance) and obtained significant improvement on multiple widely-used models.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

In recent years, plentiful evidence illustrates that Graph Convolutional Networks (GCNs) achieve extraordinary accomplishments on the node classification task. However, GCNs may be vulnerable to adversarial attacks on label-scarce dynamic graphs. Many existing works aim to strengthen the robustness of GCNs; for instance, adversarial training is used to shield GCNs against malicious perturbations. However, these works fail on dynamic graphs for which label scarcity is a pressing issue. To overcome label scarcity, self-training attempts to iteratively assign pseudo-labels to highly confident unlabeled nodes but such attempts may suffer serious degradation under dynamic graph perturbations. In this paper, we generalize noisy supervision as a kind of self-supervised learning method and then propose a novel Bayesian self-supervision model, namely GraphSS, to address the issue. Extensive experiments demonstrate that GraphSS can not only affirmatively alert the perturbations on dynamic graphs but also effectively recover the prediction of a node classifier when the graph is under such perturbations. These two advantages prove to be generalized over three classic GCNs across five public graph datasets.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

Geographical information provided in social media data is useful for many valuable applications. However, only a small proportion of social media posts are explicitly geotagged with their posting locations, which makes the pursuit of these applications challenging. Motivated by this, we propose a 2-level hierarchical classification method that builds upon a BERT model, coupled with textual information and temporal context, which we denote HierBERT. As far as we are aware, this work is the first to utilize a 2-level hierarchical classification approach alongside BERT and temporal information for geolocation prediction. Experimental results based on two social media datasets show that HierBERT outperforms various state-of-art baselines in terms of accuracy and distance error metrics.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

This paper exploits self-supervised learning (SSL) to learn more accurate and robust representations from the user-item interaction graph. Particularly, we propose a novel SSL model that effectively leverages contrastive multi-view learning and pseudo-siamese network to construct a pre-training and post-training framework. Moreover, we present three graph augmentation techniques during the pre-training stage and explore the effects of combining different augmentations, which allow us to learn general and robust representations for the GNN-based recommendation. Simple experimental evaluations on real-world datasets show that the proposed solution significantly improves the recommendation accuracy, especially for sparse data, and is also noise resistant.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

An effective and efficient encoding of the source code of a computer program is critical to the success of sequence-to-sequence deep neural network models for code representation learning. In this study, we propose to use the Prufer sequence of the Abstract Syntax Tree (AST) of a computer program to design a sequential representation scheme that preserves the structural information in an AST. Our representation makes it possible to develop deep-learning models in which signals carried by lexical tokens in the training examples can be exploited automatically and selectively based on their syntactic role and importance. Unlike other recently-proposed approaches, our representation is concise and lossless in terms of the structural information of the AST. Results from our experiment show that prufer-sequence-based representation is indeed highly effective and efficient.

poster session 1 @ blue 5, poster session 11 @ blue 5, poster session 1, poster session 11

Spectral graph convolution is extremely time-consuming for large graphs, thus existing Graph Convolutional Networks (GCNs) reconstruct the kernel by a polynomial, which is (almost) fixed. To extract features from the graph data by learning kernels, Low-pass Collaborative Filter Network (LCFN) was proposed as a new paradigm with trainable kernels. However, there are two demerits of LCFN: (1) The hypergraphs in LCFN are constructed by mining 2-hop connections of the user-item bipartite graph, thus 1-hop connections are not used, resulting in serious information loss. (2) LCFN follows the general network structure of GCNs, thus the structure is suboptimal. To address these issues, we utilize the bipartite graph to define the graph space directly and explore the best network structure based on experiments. Comprehensive experiments on two real-world datasets demonstrate the effectiveness of the proposed model.

poster session 1 @ blue 5, poster session 11 @ blue 5, oral session 11 @ blue 5, poster session 1, poster session 11, oral session 11