This paper explores the generalization loss of linear regression in variably parameterized families of models, both under-parameterized and over-parameterized. We show that the generalization curve can have an arbitrary number of peaks, and moreover, locations of those peaks can be explicitly controlled.Our results highlight the fact that both classical U-shaped generalization curve and the recently observed double descent curve are not intrinsic properties of the model family. Instead, their emergence is due to the interaction between the properties of the data and the inductive biases of learning algorithms.
We study the stochastic shortest path (SSP) problem in reinforcement learning with linear function approximation, where the transition kernel is represented as a linear mixture of unknown models. We call this class of SSP problems the linear mixture SSP. We propose a novel algorithm for learning the linear mixture SSP, which can attain a O(dB 1.5 K/c min ) regret. Here K is the number of episodes, d is the dimension of the feature mapping in the mixture model, B bounds the expected cumulative cost of the optimal policy, and c min > 0 is the lower bound of the cost function. Our algorithm also applies to the case when c min = 0, where a O(K 2/3 ) regret is guaranteed. To the best of our knowledge, this is the first algorithm with a sublinear regret guarantee for learning linear mixture SSP. In complement to the regret upper bounds, we also prove a lower bound of Ω(dB √ K), which nearly matches our upper bound.
Despite remarkable success, deep neural networks are sensitive to human-imperceptible small perturbations on the data and could be adversarially misled to produce incorrect or even dangerous predictions. To circumvent these issues, practitioners introduced adversarial training to produce adversarially robust models whose predictions are robust to small perturbations to the data. It is widely believed that more training data will help adversarially robust models generalize better on the test data. In this paper, however, we challenge this conventional belief and show that more training data could hurt the generalization of adversarially robust models for the linear classification problem. We identify three regimes based on the strength of the adversary. In the weak adversary regime, more data improves the generalization of adversarially robust models. In the medium adversary regime, with more training data, the generalization loss exhibits a double descent curve. This implies that in this regime, there is an intermediate stage where more training data hurts their generalization. In the strong adversary regime, more data almost immediately causes the generalization error to increase.
We develop a model to explain discontinuities in the increase of the length of a DNA plectoneme when the DNA filament is continuously twisted under tension. We account for DNA elasticity, electrostatic interactions and entropic effects due to thermal fluctuation. We postulate that a corrugated energy landscape that contains energy barriers is the cause of jumps in the length of the plectoneme as the number of turns is increased. Thus, our model is similar to the Prandtl-Tomlinson model of atomic scale friction. The existence of a corrugated energy landscape can be justified due to the close proximity of the neighboring pieces of DNA in a plectoneme. We assume the corrugated energy landscape to be sinusoidal since the plectoneme has a periodic helical structure and rotation of the bead is a form of periodic motion. We perform calculations with different tensile forces and ionic concentrations, and show that rotation-extension curves manifest stair-step shapes under relatively high ionic concentrations and high forces. We show that the jump in the plectonemic growth is caused by the flattening of the energy barrier in the corrugated landscape.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.