OomKiller -- Out-Of-Memory Process Killer

By default, the Linux kernel is configured to never say no when application processes ask for more memory. The assumption is that the applications will not actually use all the memory they ask for--this is called overcommitting memory. Hotels and airlines do the same thing when accepting bookings: the assumption is that not everybody who makes a booking will actually turn up to take their room/flight.

However, every now and then, the assumption is wrong--you have an overbooking, and somebody has to get bumped. When Linux runs out of memory, it starts killing processes in order to free some up. Of course, the processes are chosen according to a heuristic (which is a technical term meaning "you can't please everyone"), and so invariably the kernel is going to kill something you consider important, thereby leading to much wailing and gnashing of teeth.

This default behaviour can be changed via the sysctl(8) parameter vm.overcommit_memory.

Good explanation at LWN.