Some of the more esoteric terms and concepts you may come across in MPEG.

“Streams” in MPEG are actually called packetized elementary streams (PES), because the data is divided up into packets, each of which begins with a PES header. Each PES header can contain a presentation time stamp (PTS) which synchronizes the time at which that data is to be decoded and presented to the viewer, along with corresponding data from the other streams.

Not every PES packet has to have a PTS, but there must be at least one every 0.7 seconds, in order to ensure the decoder clock stays in sync. The PTS is a 33-bit unsigned integer, in units of a 90kHz clock.

A packet belonging to one stream (say, video) will invariably be immediately followed by another belonging to another stream (say, audio), with around the same PTS. This interleaving or multiplexing of data minimizes the amount of buffering the decoder has to implement in order to provide smoothly synchronized playback.