Robust benchmarking in noisy environments
We propose a benchmarking strategy that is robust in the presence of timer error, OS jitter and other environmental fluctuations, and is insensitive to the highly nonideal statistics produced by timing measurements. We construct a model that explains how these strongly nonideal statistics can arise...
Gespeichert in:
Hauptverfasser: | , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | We propose a benchmarking strategy that is robust in the presence of timer
error, OS jitter and other environmental fluctuations, and is insensitive to
the highly nonideal statistics produced by timing measurements. We construct a
model that explains how these strongly nonideal statistics can arise from
environmental fluctuations, and also justifies our proposed strategy. We
implement this strategy in the BenchmarkTools Julia package, where it is used
in production continuous integration (CI) pipelines for developing the Julia
language and its ecosystem. |
---|---|
DOI: | 10.48550/arxiv.1608.04295 |