Abstract. Given an irreducible subshift of finite type X, a subshift Y , a factor map π : X → Y , and an ergodic invariant measure ν on Y , there can exist more than one ergodic measure on X which projects to ν and has maximal entropy among all measures in the fiber, but there is an explicit bound on the number of such maximal entropy preimages.
Recently, training with adversarial examples, which are generated by adding a small but worst-case perturbation on input examples, has improved the generalization performance of neural networks. In contrast to the biased individual inputs to enhance the generality, this paper introduces adversarial dropout, which is a minimal set of dropouts that maximize the divergence between 1) the training supervision and 2) the outputs from the network with the dropouts. The identified adversarial dropouts are used to automatically reconfigure the neural network in the training process, and we demonstrated that the simultaneous training on the original and the reconfigured network improves the generalization performance of supervised and semi-supervised learning tasks on MNIST, SVHN, and CIFAR-10. We analyzed the trained model to find the performance improvement reasons. We found that adversarial dropout increases the sparsity of neural networks more than the standard dropout. Finally, we also proved that adversarial dropout is a regularization term with a rank-valued hyper-parameter that is different from a continuous-valued parameter to specify the strength of the regularization.
Link to this article: http://journals.cambridge.org/abstract_S0143385701001584How to cite this article: SUJIN SHIN (2001). Measures that maximize weighted entropy for factor maps between subshifts of nite type.Abstract. Let X, Y be topologically mixing subshifts of finite type and π : X → Y a factor map. For each α ≥ 0, the weighted entropy function φ α is defined by φ α (µ) = h(µ) + αh(πµ) for each invariant measure µ on X. To investigate whether for a given α > 0 there is a unique measure which achieves sup µ φ α (µ), we use the concept of compensation functions which was first considered by Boyle and Tuncel and has been developed by Walters. We prove that if there is a certain kind (more general than summable variation) of compensation function, then for each α ≥ 0 the shift-invariant measure which maximizes the weighted entropy is unique. In particular, if the compensation function is locally constant, then the unique measure is Markov and mixing. We classify the 1-block codes from a 3-symbol subshift of finite type to a 2-symbol subshift in terms of what type of compensation function exists or does not exist, providing examples of factor maps which do and do not satisfy the hypothesis. Also we study general properties of compensation functions and the maximal weighted entropy map as a function of the weight.
Abstract. Let (X, S) and (Y, T ) be topological dynamical systems and π :For a factor code between subshifts of finite type, we analyze the associated relative entropy function and give a necessary condition for the existence of saturated compensation functions. Necessary and sufficient conditions for a map to be a saturated compensation function will be provided.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.