The LHCb Online system in 2020: trigger-free read-out with (almost exclusively) off-the-shelf hardware

Colombo, T.; Amihalachioaei, A.; Arnaud, Keith A.; Alessio, F.; Brarda, L.; Cachemiche, J.-P.; Cámpora, Daniel; Cap, S.; Cardoso, L. Granado; Cindolo, F.; Daoudi, M.; Durante, P.; Duval, P.Y.; Faerber, C.; Fernández, Pablo; Frank, M.; Galli, D.; Gaspar, C.; Hachon, F.; Jevaud, M.; Jost, B.; Gac, R. Le; Manzali, M.; Marconi, U.; Mohammed, H.; Neufeld, N.; Pisani, F.; Promberger, L.; Quast, Christina; Réthoré, F.; Sborzacchi, Francesco; Schwemmer, R.; T’Jampens, S.; Valat, S.; Barbosa, Joao; Vöneki, B.; Vouters, G.

doi:10.1088/1742-6596/1085/3/032041

Cited by 8 publications

(10 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…InfiniBand has been proven to be the most performant and reliable for this use case, where the same link speeds could be used. The related extensive evaluation supporting this statement has been documented in [4][5][6][7][8][9][10]. This stage is currently deployed.…”

Section: Distribution Of Events To the Filtering Farmmentioning

confidence: 92%

See 1 more Smart Citation

Ethernet evaluation in data distribution traffic for the LHCb filtering farm at CERN

et al. 2021

View full text Add to dashboard Cite

This paper evaluates the real-time distribution of data over Ethernet for the upgraded LHCb data acquisition cluster at CERN. The system commissioning ends in 2021 and its total estimated input throughput is 32 Terabits per second. After the events are assembled, they must be distributed for further data selection to the filtering farm of the online trigger. High-throughput and very low overhead transmissions will be an essential feature of such a system. In this work RoCE (Remote Direct Memory Access over Converged Ethernet) high-throughput Ethernet protocol and Ethernet flow control algorithms have been used to implement lossless event distribution. To generate LHCb-like traffic, a custom benchmark has been implemented. It was used to stress-test the selected Ethernet networks and to check resilience to uneven workload distribution. Performance tests were made with selected evaluation clusters. 100 Gb/s and 25 Gb/s links were used. Performance results and overall evaluation of this Ethernet-based approach are discussed.

show abstract

Section: Distribution Of Events To the Filtering Farmmentioning

confidence: 92%

“…The present CERN LHCb upgrade [1,2] involves a significant increase in the throughput in the filtering farm [3][4][5]. The new version of the LHCb data acquisition cluster will implement a fully software-defined selection.…”

Section: Introductionmentioning

confidence: 99%

Ethernet evaluation in data distribution traffic for the LHCb filtering farm at CERN

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Those invalid data should be skipped not creating "holes" in the output stream. For event reconstruction in the triggerless DAQ, the data must be assigned to a particular time period 1 . Therefore, an essential requirement is that the concentrator disturbs the time-ordering of the input data as little as possible.…”

Section: Formulation Of the Problemmentioning

confidence: 99%

“…The readout system does not need to perform complex local processing of data. Extraction of interesting events is done in further stages of the DAQ ("event builder" and "event filter" [1] or "event selector" [2]). The responsibility of the readout system is different in this configuration.…”

Section: Introductionmentioning

confidence: 99%

Beneš Network Based Efficient Data Concentrator for Triggerless Data Acquisition Systems

Gumiński¹,

Kruszewski²,

Zabołotny³

et al. 2023

Preprint

View full text Add to dashboard Cite

The concentration of data from multiple links to a single output is an essential task performed by High-Energy Physics (HEP) Data Acquisition Systems (DAQs). At high and varying data rates combined with the large width of the concentrator’s output interface, this task is non-trivial. This paper presents a concentrator based on the Bene&scaron; network, which provides efficient concentration without using a high-frequency clock internally. It warrants that empty data are eliminated and does not disturb the data time-ordering if the data rates significantly differ between inputs. Additionally, it is well suited to FPGA implementation. It is based on simple data-routing primitives and may be fully pipelined.

show abstract

“…After the major upgrade [1] done in view of Run 3, the Large Hadron Collider Beauty (LHCb) experiment [2] will read out the entire detector at a frequency of 30 MHz, resulting in a data rate of 40 Tb/s. One of the main goals of the LHCb upgrade is the transition to a full software event trigger [3]. This requires a new readout system, a new event builder, and a substantially upgraded purely software-based event filter system in order to acquire, assemble, and select events at the full rate of 30 MHz.…”

Section: Introductionmentioning

confidence: 99%

The LHCb HLT2 Storage System: A 40-GB/s System Made of Commercial Off-the-Shelf Components and Open-Source Software

Cifra

Sborzacchi

Neufeld

et al. 2023

IEEE Trans. Nucl. Sci.

View full text Add to dashboard Cite

The LHCb (Large Hadron Collider beauty) experiment is designed to study differences between particles and anti-particles as well as very rare decays in the charm and beauty sector of the standard model at the LHC. With the major upgrade done in view of Run 3, the detector will read-out all events at the full LHC bunch-crossing frequency of 40 MHz. The LHCb data acquisition system will be subject to a considerably increased data rate, reaching a peak of 40 Tb/s. The second stage of the two-stage filtering consists of more than 10000 multithreaded processes which simultaneously write output files at an aggregated band-width of 100 Gb/s. At the same time, a small number of file-moving processes will read files from the same storage to copy them over to tape-storage. This whole mechanism must run reliably over months and be able to cope with significant fluctuations. Moreover, for cost reasons, it must be built from off-the-shelf components. In this paper we describe LHCb's solution to this challenge. We show the design, present reasons for the design choices, the configuration and tuning of the adopted software solution, and present performance figures.

show abstract

The LHCb Online system in 2020: trigger-free read-out with (almost exclusively) off-the-shelf hardware

Cited by 8 publications

References 6 publications

Ethernet evaluation in data distribution traffic for the LHCb filtering farm at CERN

Ethernet evaluation in data distribution traffic for the LHCb filtering farm at CERN

Beneš Network Based Efficient Data Concentrator for Triggerless Data Acquisition Systems

The LHCb HLT2 Storage System: A 40-GB/s System Made of Commercial Off-the-Shelf Components and Open-Source Software

Contact Info

Product

Resources

About