A METHOD AND SYSTEM FOR LOSSY COMPRESSION OF LOG FILES OF DATA

A method and apparatus for log files of data compression are disclosed. The method comprising: classifying each of a plurality of lines in a plurality of the log files of data with at least two levels hierarchy clustering comprising identifying a plurality of strings repeated in the plurality of lines of the plurality of log files of data. Creating a table matching each of the plurality of strings to a unique value. Creating a vector encoding the unique value matched to each of the plurality of strings using the table. Assigning each of the encoded unique values in the vector, a security relevance score according to the classification of the plurality of lines; and selecting a subset of the encoded unique values such that the encoded unique values in the vector are filtered according to the security relevance score of each unique value.