Towards Green Big Data at CERN

Abstract

High-energy physics studies collisions of particles traveling near the speed of light. For statistically significant results, physicists need to analyze a huge number of such events. One analysis job can take days and process tens of millions of collisions. Today the experiments of the large hadron collider (LHC) create 10 GB of data per second and a future upgrade will cause a tenfold increase in data. The data analysis requires not only massive hardware but also a lot of electricity. In this article, we discuss energy efficiency in scientific computing and review a set of intermixed approaches we have developed in our Green Big Data project to improve energy efficiency of CERN computing. These approaches include making energy consumption visible to developers and users, architectural improvements, smarter management of computing jobs, and benefits of cloud technologies. The open and innovative environment at CERN is an excellent playground for different energy efficiency ideas which can later find use in mainstream computing.

Explicit feedback for congestion control in Science DMZ cyberinfrastructures based on programable data-plane switches.

“…Current scientific applications, such as collaborative research experiments, astronomical observations, weather prediction, and hyperspectral imaging in medicine or biology, may quickly generate Tera bytes (TB) of data daily. For instance: (i) the Belle 2 High Energy Physics experiment is expecting to collect at least 250 PB of raw data in its first five years of operation [1]; (ii) during the second run, the CERN computing center has saved up to 10 GB of data per second, this information was transmitted to globally distributed computing centers by using the grid computing paradigm [2], and (iii) instruments used in the earth remote sensing could generate hyperspectral image datasets in terms of GB [3]. The Chilean scientific community also faces this overflowing growth in generating scientific data.…”