Detection of malicious system calls

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting malicious system calls. In one aspect, a method includes monitoring a function vulnerable to a buffer overflow attack; receiving a call to the function, the call associated with a call stack, the call stack including one or more base pointers, and a destination buffer associated with the function; identifying a first critical memory address vulnerable to the buffer overflow attack comprising: determining the first critical memory address based on a base pointer of the one or more base pointers, wherein the base pointer address is greater than an address of the destination buffer; identifying a first address based on the base pointer of the one or more base pointers; and determining that the first address is a critical memory address in response to the first memory address is greater than the address of the destination buffer.