Flexible Learning of Sparse Neural Networks via Constrained L0 Regularizations

Gallego-Posada, Jose; Rios, Juan Ramirez de los; Erraqabi, Akram

doi:10.52591/202112071

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The graphical criterion of Lachapelle et al [2022], which allows complete disentan-glement, can be derived as a special case of our theory. We also propose to enforce sparsity via constrained optimization instead of regularization, following Gallego-Posada et al [2021]. We finally illustrate our theory in simulations.…”

Section: Contributionsmentioning

confidence: 99%

“…Lagrangian multiplier α, which is forced to remain greater or equal to zero via a simple projection step. As suggested by Gallego-Posada et al [2021], we perform dual restarts which simply means that, as soon as the constraint is satisfied, the Lagrangian multiplier is reset to 0. We used the library Cooper [Gallego-Posada and Ramirez, 2022], which implement many constrained optimization procedure in Python, including the one described above.…”

Section: Author Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

Partial Disentanglement via Mechanism Sparsity

Lachapelle¹,

Lacoste-Julien²

2022

Preprint

View full text Add to dashboard Cite

Disentanglement via mechanism sparsity was introduced recently as a principled approach to extract latent factors without supervision when the causal graph relating them in time is sparse, and/or when actions are observed and affect them sparsely. However, this theory applies only to ground-truth graphs satisfying a specific criterion. In this work, we introduce a generalization of this theory which applies to any ground-truth graph and specifies qualitatively how disentangled the learned representation is expected to be, via a new equivalence relation over models we call consistency. This equivalence captures which factors are expected to remain entangled and which are not based on the specific form of the ground-truth graph. We call this weaker form of identifiability partial disentanglement. The graphical criterion that allows complete disentanglement, proposed in an earlier work, can be derived as a special case of our theory. Finally, we enforce graph sparsity with constrained optimization and illustrate our theory and algorithm in simulations.

show abstract

Section: Contributionsmentioning

confidence: 99%

Section: Author Contributionsmentioning

confidence: 99%