Buffer overflows are when you read more data into a buffer than it can hold, normally overwriting the data of whatever the next thing is in memory. This has been exploited to great effect in recent years due to sloppy C/C++ coding.

The main problem with exploiting buffer overflows is that the stack grows "down" (ie, to lower addresses), and you write to buffers "upwards" (towards higher addresses). This means that if a buffer is stored on the stack, then you can overwrite whatever else is on the stack including the return address of a function. Thus, normally exploits are set up so when a function returns it returns into the address of the buffer, and then arbitary (usually malicious) code can be executed.

Recently there has been a great deal of discussion about making the stack non executable so that this attack will fail. This however (IMHO) is rather pointless, since you can just set up the stack in a way where it runs a sequence of libc functions anyway. Admittidly you can't do conditionals, but you can make it return into for instance the beginning of "system(3)" with the parameter "rm -rf /" on the stack where system would expect it's argument.