Generating method and device for distributed indexes
The invention discloses a generating method and device for distributed indexes. According to the method, the number of map jobs in Hadoop is determined according to the data volume of original data; data processed through the map jobs are distributed to multiple reduce jobs, and an index database co...
Gespeichert in:
1. Verfasser: | |
---|---|
Format: | Patent |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention discloses a generating method and device for distributed indexes. According to the method, the number of map jobs in Hadoop is determined according to the data volume of original data; data processed through the map jobs are distributed to multiple reduce jobs, and an index database corresponding to each reduce job is generated, wherein the number of the reduce jobs and the corresponding relation between each reduce job and one or more map jobs are pre-configured; the index databases corresponding to the reduce jobs are combined. According to the technical scheme, mass data are efficiently and quickly indexed. |
---|