ESP is the protocol used by IPSec when it wants to encrypt the packets traversing the tunnel. It does not provide authentication as such, so any program using ESP should either also use AH or allow for authentication via some other method. Without authentication, a ManInTheMiddle attack becomes trivial and the usefulness of having the data encrypted is nullified.
More analysis is available at 
http://www.freeswan.org/freeswan_trees/freeswan-1.99/doc/ipsec.html#ESP.ipsec.
ESP is defined in RFC 2406.
4 pages link to ESP:
