Software Development Kit

// previous: Why MxPEG? // next: Finding the right streaming parameters

How does it work?

The first frame in an MxPEG stream is always a valid JPEG frame (with two application-specific comments included, called "M1IMG" and "MX Frame Header MXF").

Subsequent frames are not complete JPEG frames. These so-called MxPEG frames are JPEG-like frames consisting of the tiles (read: MCUs) that have changed, compared to the preceding frame.

The position of the changed tiles within the frame is communicated by means of a bitmask. This bitmask is included in another JPEG comment, called MXM.

There is an algorithm that determines a tile's degree of change based on the YUV image representation. This algorithm belongs to the implementation details of the MxPEG encoder which are not covered here.

If a tile has changed above a user-definable threshold (mxpeg_minchange), the tile will be included in the MxPEG frame.

In order to get a regular update of every tile even if its change is below the threshold, there's a user-definable parameter (mxpeg_cyclic) that forces the encoder to send cyclic updates of the allegedly unchanged tiles.

In the following diagram, the encoder's operating principle and the resulting frames are sketched (leaving out cyclic update, though).

(We expect you to be familiar with JPEG encoding. Please refer to appropriate documentation if not. A very brief one is here)

When decoding MxPEG, all operations have to be inverted, except for the special case when MxPEG is converted into Motion JPEG. Then (if you use the same tables) you will be able to reuse the runlength- and Huffman-encoded AC components of the changed tiles. The DC components have to be Huffman- and DPCM-decoded first before inserting the tiles into the complete frame. After insertion, DPCM and Huffman encoding of the complete frame's DC components has to be carried out.

In the mxg2mjpg sample application, we decoded and re-encoded the AC components, too. This is because mjpeg_pipe uses the more general mxcpc library. Since this library is also being used for rendering tiles for screen etc., we only wrote a catch-all decoding procedure and not an MJPEG-specific. To write the "reuse-AC MxPEG to Motion JPEG" converter is assigned to the reader as an easy excercise (to be completed by next Friday ;-) .

// previous: Why MxPEG? // next: Finding the right streaming parameters