Papers

This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.

poster session 4 @ red 4, poster session 11 @ red 4, oral session 4 @ red 4, poster session 4, poster session 11, oral session 4

This paper studies a basic function fitting problem, which we coin \textit{first-order convex fitting} (FCF): given any two vector sequences $\{\bvec{x}_i\}_{i\in[T]}$ and $\{\bvec{p}_i\}_{i\in[T]}$ in $\mathbb{R}^d$, when is it possible to \emph{efficiently} construct a convex function $f(\bvec{x})$ that ``fits'' the two sequences in the first-order sense, i.e, its gradient $\nabla f(\bvec{x}_i)$ equals $\bvec{p}_i$ for all $i \in [T] = \{ 1,\cdots, T \}$? Despite the close connection between function fitting and machine learning, FCF has surprisingly been overlooked in the past literature. With an efficient constructive proof, we discover a clean answer to this question: \name{} is possible \textit{if and only if} the two sequences are \textit{permutation stable}: $\sum_{i=1}^{T} \bvec{x}_i \cdot \bvec{p}_i \geq \sum_{i=1}^T \bvec{x}_i \cdot \bvec{p}_{\sigma(i)}$ for any permutation $\sigma$ of $[T]$. We demonstrate the usefulness of FCF in two applications. First, we empirically show how it can be used as a surrogate to significantly accelerate the minimization of the original convex function. Second, we study how it can be used as an empirical risk minimization procedure to learn the original convex function. We provide efficient PAC-learnability bounds for special classes of convex functions learned via FCF, and demonstrate its application to multiple economic problems where only function gradients (as opposed to function values) can be observed.

poster session 4 @ red 4, poster session 11 @ red 4, oral session 4 @ red 4, poster session 4, poster session 11, oral session 4

Missing value imputation is crucial for real-world data science workflows. Imputation is harder in the online setting, as it requires the imputation method itself to be able to evolve over time. For practical applications, imputation algorithms should produce imputations that match the true data distribution, handle data of mixed types, including ordinal, boolean, and continuous variables, and scale to large datasets. In this work we develop a new online imputation algorithm for mixed data using the Gaussian copula. The online Gaussian copula model produces meets all the desiderata: its imputations match the data distribution even for mixed data, improve over its offline counterpart on the accuracy when the streaming data has a changing distribution, and on the speed (up to an order of magnitude) especially on large scale datasets. By fitting the copula model to online data, we also provide a new method to detect change points in the multivariate dependence structure for mixed data with missing values. Experimental results on synthetic and real world data validate the performance of the proposed methods.

poster session 4 @ red 4, poster session 11 @ red 4, oral session 4 @ red 4, poster session 4, poster session 11, oral session 4