This paper resolves one of the longest standing basic problems in the streaming computational model. Namely, optimal construction of quantile sketches. An ε approximate quantile sketch receives a stream of items x 1 , . . . , x n and allows one to approximate the rank of any query up to additive error εn with probability at least 1 − δ. The rank of a query x is the number of stream items such that x i ≤ x. The minimal sketch size required for this task is trivially at least 1/ε. Felber and Ostrovsky obtain a O((1/ε) log(1/ε)) space sketch for a fixed δ. To date, no better upper or lower bounds were known even for randomly permuted streams or for approximating a specific quantile, e.g., the median. This paper obtains an O((1/ε) log log(1/δ)) space sketch and a matching lower bound. This resolves the open problem and proves a qualitative gap between randomized and deterministic quantile sketching. One of our contributions is a novel representation and modification of the widely used merge-and-reduce construction. This subtle modification allows for an analysis which is both tight and extremely simple. Similar techniques should be useful for improving other sketching objectives and geometric coreset constructions.

Finding a maximal independent set (MIS) in a graph is a cornerstone task in distributed computing. The local nature of an MIS allows for fast solutions in a static distributed setting, which are logarithmic in the number of nodes or in their degrees [Luby 1986, Ghaffari 2015. By running a (static) distributed MIS algorithm after a topology change occurs, one can easily obtain a solution with the same complexity also for the dynamic distributed model, in which edges or nodes may be inserted or deleted.In this paper, we take a different approach which exploits locality to the extreme, and show how to update an MIS in a dynamic distributed setting, either synchronous or asynchronous, with only a single adjustment, meaning that a single node changes its output, and in a single round, in expectation. These strong guarantees hold for the complete fully dynamic setting: we handle all cases of insertions and deletions, of edges as well as nodes, gracefully and abruptly. This strongly separates the static and dynamic distributed models, as super-constant lower bounds exist for computing an MIS in the former.We prove that for any deterministic algorithm, there is a topology change that requires n adjustments, thus we also strongly separate deterministic and randomized solutions.Our results are obtained by a novel analysis of the surprisingly simple solution of carefully simulating the greedy sequential MIS algorithm with a random ordering of the nodes. As such, our algorithm has a direct application as a 3-approximation algorithm for correlation clustering. This adds to the important toolbox of distributed graph decompositions, which are widely used as crucial building blocks in distributed computing.Finally, our algorithm enjoys a useful history-independence property, which means that the distribution of the output structure depends only on the current graph, and does not depend on the history of topology changes that constructed that graph. This means that the output cannot be chosen, or even biased, by the adversary, in case its goal is to prevent us from optimizing some objective function. Moreover, history independent algorithms compose nicely, which allows us to obtain history independent coloring and matching algorithms, using standard reductions.

We propose TabTransformer, a novel deep tabular data modeling architecture for supervised and semi-supervised learning. The TabTransformer is built upon self-attention based Transformers. The Transformer layers transform the embeddings of categorical features into robust contextual embeddings to achieve higher prediction accuracy. Through extensive experiments on fifteen publicly available datasets, we show that the TabTransformer outperforms the state-of-theart deep learning methods for tabular data by at least 1.0% on mean AUC, and matches the performance of tree-based ensemble models. Furthermore, we demonstrate that the contextual embeddings learned from TabTransformer are highly robust against both missing and noisy data features, and provide better interpretability. Lastly, for the semi-supervised setting we develop an unsupervised pre-training procedure to learn data-driven contextual embeddings, resulting in an average 2.1% AUC lift over the state-of-the-art methods.

In this paper we give reconstruction algorithms for depth-3 arithmetic circuits with k multiplication gates (also known as ΣΠΣ(k) circuits), where k = O(1). Namely, we give an algorithm that when given a black box holding a ΣΠΣ(k) circuit C over a field F as input, makes queries to the black box (possibly over a polynomial sized extension field of F) and outputs a circuit C computing the same polynomial as C. In particular we obtain the following results.1. When C is a multilinear ΣΠΣ(k) circuit (i.e. each of its multiplication gates computes a multilinear polynomial) then our algorithm runs in polynomial time (when k is a constant) and outputs a multilinear ΣΠΣ(k) circuits computing the same polynomial.2. In the general case, our algorithm runs in quasi polynomial time and outputs a generalized depth-3 circuit (a notion that is defined in the paper) with k multiplication gates. In fact, this algorithm works in the slightly more general case where the black box holds a generalized depth-3 circuits.Prior to this work there were reconstruction algorithms for several different models of bounded depth circuits: the well studied class of depth-2 arithmetic circuits (that compute sparse polynomials) and its close by model of depth-3 set-multilinear circuits. For the class of depth-3 circuits only the case of k = 2 (i.e. ΣΠΣ(2) circuits) was known.Our proof technique combines ideas from [Shp07] and [KS08] with some new ideas. Our most notable new ideas are: We prove the existence of a unique canonical representation of depth-3 circuits. This enables us to work with a specific representation in mind. Another technical contribution is an isolation lemma for depth-3 circuits that enables us to reconstruct a single multiplication gate of the circuit.

It is well known that collaborative filtering (CF) based recommender systems provide better modeling of users and items associated with considerable rating history. The lack of historical ratings results in the user and the item coldstart problems. The latter is the main focus of this work. Most of the current literature addresses this problem by integrating content-based recommendation techniques to model the new item. However, in many cases such content is not available, and the question arises is whether this problem can be mitigated using CF techniques only. We formalize this problem as an optimization problem: given a new item, a pool of available users, and a budget constraint, select which users to assign with the task of rating the new item in order to minimize the prediction error of our model. We show that the objective function is monotone-supermodular, and propose efficient optimal design based algorithms that attain an approximation to its optimum. Our findings are verified by an empirical study using the Netflix dataset, where the proposed algorithms outperform several baselines for the problem at hand.