The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

Abstract

We describe the outcome of a data challenge conducted as part of the Dark Machines (https://www.darkmachines.org) initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims to detect signals of new physics at the Large Hadron Collider (LHC) using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 billion simulated LHC events corresponding to 10\, fb^{-1}10fb−1 of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.

Deep Set Auto Encoders for Anomaly Detection in Particle Physics

“…Because of this, we were unable to explore a huge range of hyperparameter options. Despite this, our deep set autoencoders were among the best models in the Dark Machines Challenge [50].…”

“…The data for this work come from the Dark Machines Anomaly Score Challenge [50]. A large dataset of over 1 billion simulated proton-proton collisions with a center of mass energy of S = 13 TeV was generated [60].…”

“…Sometimes networks find certain signals better than others, but changing the parameters can trade which signals can be separated from the background. We incorporate a holistic view of the problem as done in the challenge [50,69]. For each channel, we find the total loss which allows 1%, 0.1%, or 0.01% of the background events through.…”

“…The second category of anomaly detection searches for individual events that look different than what is expected from the standard model (also known as out-of-distribution events) . The Dark Machines Anomaly Score Challenge [50] studied many methods to flag individual events as anomalous after training on a simulated set of standard model background events. This work was a contributed method to the Dark Machines challenge.…”

“…Unexpectedly, we find that β = 1 offers the best anomaly detection performance for this method. In fact, our networks with β = 1 were among the top methods submitted to the Dark Machines challenge [50]. As β = 1 does not include the reconstruction loss, there is actually no need for the decoder part of the network, and the method is only trying to map the set to a Gaussian latent space.…”