HDFS (Hadoop Distributed File System) mass small file storage method suitable for more-read less-write scene
The invention provides an HDFS (Hadoop Distributed File System) mass small file storage method suitable for a more-read less-write scene. The method is used for solving the technical problems of high NameNode occupancy rate and low access efficiency caused by small file storage of an existing HDFS....
Gespeichert in:
Hauptverfasser: | , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention provides an HDFS (Hadoop Distributed File System) mass small file storage method suitable for a more-read less-write scene. The method is used for solving the technical problems of high NameNode occupancy rate and low access efficiency caused by small file storage of an existing HDFS. The method comprises the following steps: a construction stage: combining small files into larger data files, and constructing a primary index and a secondary index for each data file; in the use stage, if the query operation is carried out, the index is queried to return a result, and if the update, deletion and addition operations are carried out, the operations are uniformly converted into addition, and the small file update, deletion and addition operations are realized. According to the method, the first-level index combining two index technologies is adopted, so that efficient access can be ensured, and modification, updating and deletion operations of small files can be assisted to be realized; the small fil |
---|