In PublicKeyEncryption, the PrivateKey is the portoin of the key that is kept secret. If the PrivateKey is lost, stolen or compromised (or the PassPhrase protecting it forgotten), then the messages encrypted with the PrivateKey cannot be decrypted.
Compare: PublicKey
