Distilling the Knowledge from Conditional Normalizing Flows

Baranchuk, Dmitry; Алиев, В. Ш.; Babenko, Artem

doi:10.48550/arxiv.2106.12699

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These provide an alternative measure of closure that has improved convergence and convexity properties compared to the KL divergence. Other works in the ML literature that explore variants on the idea of Probability Density Distillation include [25][26][27].…”

Section: Introductionmentioning

confidence: 99%

“…However, the difference between the last two is quite small, so not adding L x (i) and L z (i) to the loss function provides a further viable loss candidate.5 In principle, the student does not have to be an IAF, it could also be a simple, fully-connected neural network[25]. However, in this case we would not have access to the LL as a measure of quality and we would not be able to train it with the additional loss terms of (2.12)-(2.15).…”

mentioning

confidence: 99%

See 1 more Smart Citation

CaloFlow II: Even Faster and Still Accurate Generation of Calorimeter Showers with Normalizing Flows

Krause¹,

Shih²

2021

Preprint

View full text Add to dashboard Cite

Recently, we introduced CaloFlow, a high-fidelity generative model for Geant4 calorimeter shower emulation based on normalizing flows. Here, we present CaloFlow v2, an improvement on our original framework that speeds up shower generation by a further factor of 500 relative to the original. The improvement is based on a technique called Probability Density Distillation, originally developed for speech synthesis in the ML literature, and which we develop further by introducing a set of powerful new loss terms. We demonstrate that CaloFlow v2 preserves the same high fidelity of the original using qualitative (average images, histograms of high level features) and quantitative (classifier metric between Geant4 and generated samples) measures. The result is a generative model for calorimeter showers that matches the state-of-the-art in speed (a factor of 10 4 faster than Geant4) and greatly surpasses the previous state-of-the-art in fidelity.

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%