Predicting dataset popularity for the CMS experiment

Abstract

The CMS experiment at the LHC accelerator at CERN relies on its computing infrastructure to stay at the frontier of High Energy Physics, searching for new phenomena and making discoveries. Even though computing plays a significant role in physics analysis we rarely use its data to predict the system behavior itself. A basic information about computing resources, user activities and site utilization can be really useful for improving the throughput of the system and its management. In this paper, we discuss a first CMS analysis of dataset popularity based on CMS meta-data which can be used as a model for dynamic data placement and provide the foundation of data-driven approach for the CMS computing infrastructure. arXiv:1602.07226v1 [physics.data-an]

Progress in Machine Learning Studies for the CMS Computing Infrastructure

“…For reference: PR curves plot precision versus recall, where precision = TP/(TP+FP) and recall = TP/(TP+FN); ROC curves plot FPR vs TPR, with FPR = FP/(FP+TN), and TPR = TP/(TP+FN). The main result of this part of the work is that the Spark Machine Learning code has been developed and tested with the dataset popularity data sets and shows results that are comparable to those obtained previously using scikit-learn [9,10]. It was demonstrated that a Spark platform can be used with no loss whatsoever in the quality of the model and its prediction, and in addition it allows to deal more efficiently with much larger dataframe (i.e.…”

“…Widely known classifiers like XGBClassifier and RandomForestClassifier performed in the model at the level of a TPR (True Positive Rate) of 90% or more, and a TNR True Negative Rate) of around 95% [9]. The study also showed at least a factor or 2.5 of difference in favour of RandomForestClassifier in the classifier running time [10].…”

“…number of accesses to a dataset, number of users accessing it, CPU-hrs spent in accessing it, etc. A proper definition of the popularity concept in the context of preparing for the application of ML techniques has been subject of previous work [8,9]. The ability to predict the popularity of a CMS dataset is crucial as it allows to optimise the storage utilisation.…”

“…Using the DCAF framework, it may take a couple of days to collect up to 12 months of data. Dataframes from 6, 9 and 12 months in the past were used in a test to train the classifiers, and a dedicated study demonstrated that 6 months is a sufficiently large time window in terms of quality of the model built with this data: extending to a large window would have increased the risk of unwanted overfitting effects [9,10]. As a consequence, the model was trained on 6 months worth of data to predict the popularity of CMS datasets in the up-coming week; then, this week was added to the training set and the first week in the past was dropped (thus yielding the time windows to indeed "slide"); then, the classifiers were re-trained and the predictions were extracted for the following week; and so on.…”

Prototype of Machine Learning “as a Service” for CMS Physics in Signal vs Background discrimination

“…Today machine learning (ML) is becoming ubiquitous in HEP applications, most notably in final offline analyses, but it is also increasingly used in both online and offline reconstruction and particle identification algorithms, in the classification of reconstruction-level objects, such as jets, as well as in sectors that did not exploit ML up to 2-3 years ago, like the analysis of computing metadata for resource usage optimization [1,2]. It is difficult to predict how this will evolve in HEP.…”