Dealing with Process Overload Attacks in UNIX

A process overload attack is an attack on a shared computer system in which a user repeatedly forks new processes and hence makes the system unusable for others. The specific problem we address is seen in an academic environment where student programs create unintentional process overload attacks in UNIX systems by careless coding. Instead of rebooting the system or manually examining and killing the processes, our approach to dealing with these attacks was to build a process load monitoring tool to detect and kill these processes automatically. This paper focuses on what we learned about the behaviors of different fork bombs, how we classified them based on their self-replicating capabilities, and our experience with detecting, killing and cleaning these unwanted processes.