Recognizing malicious software behaviors with tree automata inference

We explore how formal methods and tools of the verification trade could be used for malware detection and analysis. In particular, we propose a new approach to learning and generalizing from observed malware behaviors based on tree automata inference. Our approach infers k -testable tree automata from system call dataflow dependency graphs. We show how inferred automata can be used for malware recognition and classification.