On the Automatic Analysis of the Practical Resistance of Obfuscating Transformations

A method is developed for assessing the practical persistence of obfuscating transformations of programs. The method is based on the calculation of the similarity index for the original, obfuscated, and deobfuscated programs. Candidates are proposed for the similarity indices, which are based on such program characteristics as the control flow graph, symbolic execution time, and degree of coverage for symbolic execution. The control flow graph is considered the basis for building other candidates for program similarity indicators. On its basis, a new candidate is proposed for the similarity index, which, when calculated, finds the Hamming distances between the adjacency matrices of the control flow graphs of the compared programs. A scheme for estimating (analyzing) the persistence of obfuscating transformations is constructed. According to this scheme, the characteristics of the original, obfuscated, and deobfuscated programs are calculated and compared in accordance with the chosen comparison model. In particular, the developed scheme is suitable for comparing programs based on similarity indices. This paper develops and implements one of the key units of the constructed scheme, which is a block for obtaining the program characteristics compiled for the x86/x86_64 architecture. The developed unit allows finding the control flow graph, the time for symbolic execution, and the degree of coverage for symbolic execution. Selected results of operation of the constructed block are provided.