DIRECT CLOUD STORAGE INTAKE AND UPLOAD ARCHITECTURE

A data gathering and query method for collecting ongoing updates to large, unstructured databases performing data collection from multiple sites. A large repository of unstructured or semi-structured data according to a JSON syntax receives periodic updates from an enterprise site for gathered event data. A bigdata cloud store receives the additions as columnar parts that arrange the data in a columnar form for storing similarly named fields consecutively. The enterprise site generating the event data arranges the block files containing the columnar data, and header files containing metadata. Incremental time and/or size triggers the periodic part upload, and a query server in network communication with the cloud store integrates the incoming additions by receiving the header files and updating a catalog of collections in the cloud store, without downloading the larger block files containing the actual columnar data. Query requests operate on the cloud store without moving the block files.