Systems and methods for distributed log file processing

A log file processing system includes a data storage device and a processing circuit configured to retrieve log file data from the data storage device and to create a bundle including a plurality of data chunks using the log file data. The processing circuit is configured to assign the bundle to a f...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	SIKKA NAMIT, GUPTA ASHISH, VASILEVSKI YURI, MELVILLE NAVIN REGINALD, HE SHAN, JIANG HAIFENG, GANESHAN VINAYAK
Format:	Patent
Sprache:	eng
Schlagworte:	CALCULATING COMPUTING COUNTING ELECTRIC DIGITAL DATA PROCESSING PHYSICS
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	A log file processing system includes a data storage device and a processing circuit configured to retrieve log file data from the data storage device and to create a bundle including a plurality of data chunks using the log file data. The processing circuit is configured to assign the bundle to a first processing pipeline of a set of multiple processing pipelines in response to a request from the first processing pipeline. The processing circuit is configured to store a global available timestamp for the bundle upon assigning the bundle to the first processing pipeline. The global timestamp is a time threshold before which the bundle is ineligible to be assigned to any of the multiple processing pipelines.