ESP is the protocol used by IPSec when it wants to encrypt the packets traversing the tunnel. It does not provide authentication as such, so any program using ESP should either also use AH or allow for authentication via some other method. Without authentication, a ManInTheMiddle attack becomes trivial and the usefulness of having the data encrypted is nullified.
More analysis is available at 
http://www.freeswan.org/freeswan_trees/freeswan-1.99/doc/ipsec.html#ESP.ipsec.
ESP is defined in 
RFC:2406
4 pages link to ESP:
