Online Discrepancy Minimization for Stochastic Arrivals

Bansal, Nikhil; Jiang, Haotian; Meka, Raghu; Singla, Sahil; Sinha, Makrand

doi:10.1137/1.9781611976465.169

Cited by 11 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In independent and concurrent work, Bansal, Jiang, Meka, Singla, and Sinha [8] (building on the techniques of [9]) achieve similar guarantees to the present work (with worse poly-log factors) for the online Komlós problem restricted to the setting where vectors are sampled randomly from a fixed distribution p inside the unit sphere. However [8] uses potential based techniques as in [9] and thus their results due not extend to minimizing discrepancy in the (more general) oblivious adversary model which is the primary focus of this paper.…”

Section: Concurrent and Independent Worksupporting

confidence: 58%

See 1 more Smart Citation

Discrepancy Minimization via a Self-Balancing Walk

Alweiss,

Liu,

Sawhney

2024

SIAM J. Comput.

View full text Add to dashboard Cite

We study discrepancy minimization for vectors in R n under various settings. The main result is the analysis of a new simple random process in high dimensions through a comparison argument. As corollaries, we obtain bounds which are tight up to logarithmic factors for online vector balancing against oblivious adversaries, resolving several questions posed by Bansal, Jiang, Singla, and Sinha (STOC 2020), as well as a linear time algorithm for logarithmic bounds for the Komlós conjecture. CCS CONCEPTS• Theory of computation → Online algorithms.

show abstract

Section: Concurrent and Independent Worksupporting

confidence: 58%

“…• [8,9], including improved bounds to online geometric discrepancy problems, balancing against convex bodies and multicolor discrepancy. This is discussed in Section 3.…”

Section: Consequences Of Theorem 11mentioning

confidence: 99%

Discrepancy Minimization via a Self-Balancing Walk

Alweiss,

Liu,

Sawhney

2024

SIAM J. Comput.

View full text Add to dashboard Cite

show abstract

“…The adversarial setting with interval discrepancy is discussed in [Jiang et al, 2019]. Other works focus on balancing variants such as stochastic arrival [Bansal et al, 2021] or box discrepancy [Matoušek and Nikolov, 2015].…”

Section: Enabling Memory-efficient Herding With Balancingmentioning

confidence: 99%

GraB: Finding Provably Better Data Permutations than Random Reshuffling

Lu¹,

Guo²,

De³

2022

Preprint

View full text Add to dashboard Cite

Random reshuffling, which randomly permutes the dataset each epoch, is widely adopted in model training because it yields faster convergence than with-replacement sampling. Recent studies indicate greedily chosen data orderings can further speed up convergence empirically, at the cost of using more computation and memory. However, greedy ordering lacks theoretical justification and has limited utility due to its non-trivial memory and computation overhead. In this paper, we first formulate an example-ordering framework named herding and answer affirmatively that SGD with herding converges at the rate O(T −2/3 ) on smooth, non-convex objectives, faster than the O(n 1/3 T −2/3 ) obtained by random reshuffling, where n denotes the number of data points and T denotes the total number of iterations. To reduce the memory overhead, we leverage discrepancy minimization theory to propose an online Gradient Balancing algorithm (GraB) that enjoys the same rate as herding, while reducing the memory usage from O(nd) to just O(d) and computation from O(n 2 ) to O(n), where d denotes the model dimension. We show empirically on applications including MNIST, CIFAR10, WikiText and GLUE that GraB can outperform random reshuffling in terms of both training and validation performance, and even outperform state-of-the-art greedy ordering while reducing memory usage over 100×.

show abstract

“…Manshadi et al [39] studied fair online rationing such that each arriving agent can receive a fair share of resources proportional to its demand. The fairness issue has been studied in other domains/applications as well, see, e.g., online selection of candidates [40], influence maximization [41], banditbased online learning [42][43][44], online resource allocation [45,46], and classification [47].…”

Section: Other Related Workmentioning

confidence: 99%

Fairness Maximization among Offline Agents in Online-Matching Markets

Ma¹,

Xu²,

Xu³

2021

Preprint

View full text Add to dashboard Cite

Matching markets involve heterogeneous agents (typically from two parties) who are paired for mutual benefit. During the last decade, matching markets have emerged and grown rapidly through the medium of the Internet. They have evolved into a new format, called Online Matching Markets (OMMs), with examples ranging from crowdsourcing to online recommendations to ridesharing. There are two features distinguishing OMMs from traditional matching markets. One is the dynamic arrival of one side of the market: we refer to these as online agents while the rest are offline agents. Examples of online and offline agents include keywords (online) and sponsors (offline) in Google Advertising; workers (online) and tasks (offline) in Amazon Mechanical Turk (AMT); riders (online) and drivers (offline when restricted to a short time window) in ridesharing. The second distinguishing feature of OMMs is the real-time decision-making element. However, studies have shown that the algorithms making decisions in these OMMs leave disparities in the match rates of offline agents. For example, tasks in neighborhoods of low socioeconomic status rarely get matched to gig workers, and drivers of certain races/genders get discriminated against in matchmaking. In this paper, we propose online matching algorithms which optimize for either individual or group-level fairness among offline agents in OMMs. We present two linear-programming (LP) based sampling algorithms, which achieve online competitive ratios at least 0.725 for individual fairness maximization (IFM) and 0.719 for group fairness maximization (GFM), respectively. There are two key ideas helping us break the barrier of 1 − 1/e. One is boosting, which is to adaptively re-distribute all sampling probabilities only among those offline available neighbors for every arriving online agent. The other is attenuation, which aims to balance the matching probabilities among offline agents with different mass allocated by the benchmark LP. We conduct extensive numerical experiments and results show that our boosted version of sampling algorithms are not only conceptually easy to implement but also highly effective in practical instances of fairness-maximization-related models.

show abstract

Online Discrepancy Minimization for Stochastic Arrivals

Cited by 11 publications

References 7 publications

Discrepancy Minimization via a Self-Balancing Walk

Discrepancy Minimization via a Self-Balancing Walk

GraB: Finding Provably Better Data Permutations than Random Reshuffling

Fairness Maximization among Offline Agents in Online-Matching Markets

Contact Info

Product

Resources

About