Fast Columnar Physics Analyses of Terabyte-Scale LHC Data on a Cache-Aware Dask Cluster
The development of an LHC physics analysis involves numerous investigations that require the repeated processing of terabytes of data. Thus, a rapid completion of each of these analysis cycles is central to mastering the science project. We present a solution to efficiently handle and accelerate phy...
Gespeichert in:
Hauptverfasser: | , , , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The development of an LHC physics analysis involves numerous investigations
that require the repeated processing of terabytes of data. Thus, a rapid
completion of each of these analysis cycles is central to mastering the science
project. We present a solution to efficiently handle and accelerate physics
analyses on small-size institute clusters. Our solution is based on three key
concepts: Vectorized processing of collision events, the "MapReduce" paradigm
for scaling out on computing clusters, and efficiently utilized SSD caching to
reduce latencies in IO operations. Using simulations from a Higgs pair
production physics analysis as an example, we achieve an improvement factor of
$6.3$ in runtime after one cycle and even an overall speedup of a factor of
$14.9$ after $10$ cycles. |
---|---|
DOI: | 10.48550/arxiv.2207.08598 |