DISTRIBUTED INDEX GENERATION METHOD AND DEVICE

The present invention discloses a distributed index generation method and device, wherein said method comprises: determining, in accordance with the data volume of original data, the number of map operations within Hadoop; allocating post-processed data, which has passed through each map operation,...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
1. Verfasser: HAN, BINGWEI
Format: Patent
Sprache:chi ; eng ; fre
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The present invention discloses a distributed index generation method and device, wherein said method comprises: determining, in accordance with the data volume of original data, the number of map operations within Hadoop; allocating post-processed data, which has passed through each map operation, to a plurality of reduce operations, and generating an index repository corresponding to every reduce operation, wherein the number of reduce operations and the corresponding relationship between every reduce operation and one or a plurality of map operations is completed as preconfigured; merging the index repositories corresponding to every reduce operation. The technical solution of the present invention achieves a high-efficiency, rapid indexing of very large amounts of data. Cette invention concerne un procédé et un dispositif de génération d'index distribué, ledit procédé consistant : à déterminer, en fonction du volume des données d'origine, le nombre des opérations de mise en correspondance dans Hadoop ; à