Separating Adaptive Streaming from Oblivious Streaming

Kaplan, Haim; Mansour, Yishay; Nissim, Kobbi; Stemmer, Uri

doi:10.48550/arxiv.2101.10836

“…Naor and Yogev [32] showed that Bloom filters are susceptible to attacks by an adversarial stream of queries. Kaplan et al [25] constructed a streaming problem naturally inspired by the adaptive data analysis literature, which exhibits a large separation between the space complexities in the adversarial and oblivious regimes. On the positive side, several recent works [4,23,48] present generic compilers that transform non-robust randomized streaming algorithms into efficient adversarially robust ones, for various classical problems such as distinct elements counting and 𝐹 𝑝 -sampling, among others.…”

Section: Streaming Algorithmsmentioning

confidence: 99%

Adversarial laws of large numbers and optimal regret in online classification

Alon

¹

,

Ben‐Eliezer

²

,

Dagan

³

et al. 2021

Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

Laws of large numbers guarantee that given a large enough sample from some population, the measure of any fixed sub-population is well-estimated by its frequency in the sample. We study laws of large numbers in sampling processes that can affect the environment they are acting upon and interact with it. Specifically, we consider the sequential sampling model proposed by Ben-Eliezer and Yogev (2020), and characterize the classes which admit a uniform law of large numbers in this model: these are exactly the classes that are online learnable. Our characterization may be interpreted as an online analogue to the equivalence between learnability and uniform convergence in statistical (PAC) learning.The sample-complexity bounds we obtain are tight for many parameter regimes, and as an application, we determine the optimal regret bounds in online learning, stated in terms of Littlestone's dimension, thus resolving the main open question from Ben-David, Pál, and Shalev-Shwartz (2009), which was also posed by .

show abstract

“…Naor and Yogev [32] showed that Bloom filters are susceptible to attacks by an adversarial stream of queries. Kaplan et al [25] constructed a streaming problem naturally inspired by the adaptive data analysis literature, which exhibits a large separation between the space complexities in the adversarial and oblivious regimes. On the positive side, several recent works [4,23,48] present generic compilers that transform non-robust randomized streaming algorithms into efficient adversarially robust ones, for various classical problems such as distinct elements counting and 𝐹 𝑝 -sampling, among others.…”

Section: Streaming Algorithmsmentioning

confidence: 99%

Adversarial Laws of Large Numbers and Optimal Regret in Online Classification

Alon,

Ben-Eliezer,

Dagan

et al. 2023

SIAM J. Comput.

0

View full text Add to dashboard Cite

Laws of large numbers guarantee that given a large enough sample from some population, the measure of any fixed sub-population is well-estimated by its frequency in the sample. We study laws of large numbers in sampling processes that can affect the environment they are acting upon and interact with it. Specifically, we consider the sequential sampling model proposed by Ben-Eliezer and Yogev (2020), and characterize the classes which admit a uniform law of large numbers in this model: these are exactly the classes that are online learnable. Our characterization may be interpreted as an online analogue to the equivalence between learnability and uniform convergence in statistical (PAC) learning.The sample-complexity bounds we obtain are tight for many parameter regimes, and as an application, we determine the optimal regret bounds in online learning, stated in terms of Littlestone's dimension, thus resolving the main open question from Ben-David, Pál, and Shalev-Shwartz (2009), which was also posed by .

show abstract

“…They showed it for the task of approximating L P norms but their technique stands for other tasks as well. In a recent result, Kaplan, Mansour, Nissim and Stemmer [20] showed a problem that requires polylogarithmic amount of memory in the static case but any adversarially robust algorithm for it requires exponentially larger memory.…”

Section: Related Workmentioning

confidence: 99%

Keep That Card in Mind: Card Guessing with Limited Memory

Menuhin¹,

Naor²

2021

Preprint

View full text Add to dashboard Cite

A card guessing game is played between two players, Guesser and Dealer. At the beginning of the game, the Dealer holds a deck of n cards (labeled 1, ..., n). For n turns, the Dealer draws a card from the deck, the Guesser guesses which card was drawn, and then the card is discarded from the deck. The Guesser receives a point for each correctly guessed card.With perfect memory, a Guesser can keep track of all cards that were played so far and pick at random a card that has not appeared so far, yielding in expectation ln n correct guesses. With no memory, the best a Guesser can do will result in a single guess in expectation.We consider the case of a memory bounded Guesser that has m < n memory bits. We show that the performance of such a memory bounded Guesser depends much on the behavior of the Dealer. In more detail, we show that there is a gap between the static case, where the Dealer draws cards from a properly shuffled deck or a prearranged one, and the adaptive case, where the Dealer draws cards thoughtfully, in an adversarial manner. Specifically:1. We show a Guesser with O(log 2 n) memory bits that scores a near optimal result against any static Dealer.2. We show that no Guesser with m bits of memory can score better than O( √ m) correct guesses, thus, no Guesser can score better than min{ √ m, ln n}, i.e., the above Guesser is optimal.3. We show an efficient adaptive Dealer against which no Guesser with m memory bits can make more than ln m + 2 ln log n + O(1) correct guesses in expectation.These results are (almost) tight, and we prove them using compression arguments that harness the guessing strategy for encoding.

show abstract

Separating Adaptive Streaming from Oblivious Streaming

Cited by 3 publications

References 19 publications

Adversarial laws of large numbers and optimal regret in online classification

Adversarial laws of large numbers and optimal regret in online classification

Adversarial Laws of Large Numbers and Optimal Regret in Online Classification

Keep That Card in Mind: Card Guessing with Limited Memory

Contact Info

Product

Resources

About