Data mining accelerator for efficient data searching

A data mining accelerator is used with network processor technology to enable real time pattern searching of large databases. The classification and search capability of a processor element array inside the network processor is used to format database records having variable length fields in random order into ordered data packets containing fixed length fields. The contents of the fields are hashed and formatted into binary key values. Searching can be by parallel processing of multiple database records or distributed processing of a single record for multiple match conditions. A classification engine is used to sort records from a single database into separate streams based on one or more special fields, or to sort records from different databases into separate search streams for routing to search engines dedicated to each stream. The search engine collects and matches statistics in real time or searches for new, statistically significant match conditions.