No abstract

The spatiotemporal oscillation patterns of the proteins MinD and MinE are used by the bacterium E. coli to sense its own geometry. Strikingly, both computer simulations and experiments have recently shown that for the same geometry of the reaction volume, different oscillation patterns can be stable, with stochastic switching between them. Here we use particle-based Brownian dynamics simulations to predict the relative frequency of different oscillation patterns over a large range of threedimensional compartment geometries, in excellent agreement with experimental results. Fourier analyses as well as pattern recognition algorithms are used to automatically identify the different oscillation patterns and the switching rates between them. We also identify novel oscillation patterns in three-dimensional compartments with membrane-covered walls and identify a linear relation between the bound Min-protein densities and the volume-to-surface ratio. In general, our work shows how geometry sensing is limited by multistability and stochastic fluctuations.

Our understanding of reinforcement learning (RL) has been shaped by theoretical and empirical results that were obtained decades ago using tabular representations and linear function approximators. These results suggest that RL methods that use temporal differencing (TD) are superior to direct Monte Carlo estimation (MC). How do these results hold up in deep RL, which deals with perceptually complex environments and deep nonlinear models? In this paper, we re-examine the role of TD in modern deep RL, using specially designed environments that control for specific factors that affect performance, such as reward sparsity, reward delay, and the perceptual complexity of the task. When comparing TD with infinite-horizon MC, we are able to reproduce classic results in modern settings. Yet we also find that finite-horizon MC is not inferior to TD, even when rewards are sparse or delayed. This makes MC a viable alternative to TD in deep RL.

No abstract