Variational Depth Search in ResNets

Deep MCANC: A deep learning approach to multi-channel active noise control

“…Dikov et al [17] proposed to estimate the neural architecture width and depth through BNN. Antorán et al [18] proposed to search the depth of residual networks in the efficient one-shot NAS framework, where the neural weights and architecture are jointly learned. Antorán et al [19] estimated the depth uncertainty through the probabilistic reasoning over a sequential structure of feed-forward networks.…”

“…Most of the current methods [17], [18], [18] approximate the architecture depth posterior with VI based on the meanfield assumption [8], [9], where the neural weights and depth variables are independent. The mean-field assumption can limit the approximation fidelity and introduce the rich-getricher problem, i.e., the shallow networks would dominate the search.…”