System and method for performing fast statistical pattern hints detection

A system for performing fast data pattern searches, comprising a computerized device consisting of at least one processor and associated memory, which are adapted to automatically create condensed elementary units to be used, based on the data patterns or based on known elementary unit data; create an indexed dictionary containing the data patterns to be searched; select a primary key for each pattern using the statistics and secondary keys counting on the primary key value, the dictionary having entries that are balanced over all array entries, based on the numbers of the secondary keys; for each searched pattern, select a primary key to search for a first fragment of the searched pattern to be compared; create a secondary key to search for a second fragment of the searched pattern, the secondary key being greater that the size of the primary key to both directions, the secondary key functions as a filtering element, where only entries for which there is a match in the secondary key value and the corresponding distance will move to the next processing stage; process typical search data, based on the condensed elementary units and preforming a training stage to thereby obtain the fastest optimal combination on a primary key size and a secondary key size, to gather statistics regarding the probability of identifying data patterns; revoke entries options that are used more often according to a predetermined threshold; create third multiple keys for multiple fragments of the searched pattern, to be compared, and if there were matches, performing full comparison; at least one register for performing full or partial comparisons. The distance of the secondary key before or after the primary key is predetermined and wherein each entry of the array defines multiple entries to be compared as secondary keys, each one of the secondary keys with its unique defined distance from the primary key.