The IceProd (IceCube Production) Framework

IceProd is a data processing and management framework developed by the IceCube Neutrino Observatory for processing of Monte Carlo simulations and data. IceProd runs as a separate layer on top of middleware or cluster job schedulers and can take advantage of a variety of computing resources including...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of physics. Conference series 2014-01, Vol.513 (3), p.32026-6
1. Verfasser: Díaz-Vélez, J C
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:IceProd is a data processing and management framework developed by the IceCube Neutrino Observatory for processing of Monte Carlo simulations and data. IceProd runs as a separate layer on top of middleware or cluster job schedulers and can take advantage of a variety of computing resources including grids such as EGI, OSG, and NorduGrid as well as local clusters running batch systems like HT Condor, PBS, and SGE. This is accomplished by a set of dedicated daemons which process job submission in a coordinated fashion through the use of middleware plug-ins that serve to abstract the details of job submission and job management. IceProd can also manage complex workflow DAGs across distributed computing grids in order to optimize usage of resources. We describe several aspects of IceProd's design and it's applications in collaborative computing environments. We also briefly discuss design aspects of a second generation IceProd, currently being tested in IceCube.
ISSN:1742-6588
1742-6596
DOI:10.1088/1742-6596/513/3/032026