Global Sources
EE Times-Asia
Stay in touch with EE Times Asia
EE Times-Asia > Controls/MCUs

Managing latency in video encoding, decoding

Posted: 16 Jan 2008 ?? ?Print Version ?Bookmark and Share

Keywords:latency of a video codec? digital video compression? H.264 latency video encoding?

Early digital video compression solutions primarily focused on applications that do not require real-time interaction (i.e. in TV broadcast, video-on-demand and DVD playback). In these applications, the latency between the source and the decoded video was not important and could extend to several seconds.

However, in applications where there is a closed feedback loop (i.e. video conference and videophone), latency is the most crucial aspect of the system, as it determines whether the system will be stable or not. Keeping the latency of video as minimal as possible is the proper approach for such systems. In many such applications, latency measured in sub-10ms is crucial. Moreover, it takes a radically different approach to achieve low latency implementation of the H.264/MPEG-4 AVC (Part 10) video coding standard.

In video conferencing and telephony, noticeable delay makes a conversation impossible, unless a "walky-talky" like protocol is strictly followed. This makes the conversation unnatural and cumbersome. Sub-33ms latency for the video codec is required in these applications.

Home networks
An emerging application with high sensitivity to latency is wireless video networking in the home. This application has recently gained a lot of interest from CE manufacturers, and aims to eliminate the HDMI cable between the HDTV set and video source (i.e. STB, DVD player or game box). A similar compelling case exists for the computer industry, where the cable between laptop and flat-panel monitor is replaced by a wireless connection.

In these applications, user interaction via the remote control, game pad, keyboard or mouse, must result in instant screen updates; otherwise the solution is rendered useless. Since transmission at multigigabit per second rates over a highly unpredictable RF link is impractical, video compression is required. In these applications, sub-10ms latency for the video codec is a critical requirement.

In multiple-camera-tracking, video feeds from several cameras are stitched together chronologically into a single feed, which tracks one or more moving objects of interest. Too much latency in the video feeds makes stitching these together a complicated task and renders the application useless for rapid response action. Surveillance applications require sub-10ms latency for the video codec.

A less obvious example is in electronic newsgathering. Cameras in the field capture live action and transmit the video for live broadcast to a nearby satellite uplink truck, where it is edited in real-time prior to uplinking. Video feeds from multiple cameras and camera panning/zooming actions need to be interpreted in real-time by the production crew.

Very low latency in the video feeds is necessary to provide inherent synchronization between all the different video feeds and with panning/zooming actions of the cameras. In this application, sub-33ms latency for the video codec is highly desirable.

Simplify systems design
"Zero" latency can drastically simplify systems design in applications where added latency due to other parts in the system (i.e. transmitters, receivers, video capture and rendering subsystems) is negligible. In these cases, complicated A/V time stamping and synchronization schemes are not needed. This is because the extremely low latency of the video stream with respect to the audio stream provides inherent synchronization between the two streams. Such extremely low latency A/V systems strongly mimic the way A/V communication occurs in the natural worldwithout complicated time stamping and synchronization.

In traditional approaches, the encoding process starts when a complete frame of video is present, introducing at least 33ms of latency at the encoder and another 33ms at the decoder. Combined with multipass motion estimation, multipass rate control and frame-based noise filtering, traditional methods of implementation can easily exhibit in excess of 200ms encode-decode latency.

Since transmission at multigigabit per second rates over a highly unpredictable RF link is impractical, video compression is required. In these applications, sub-10ms latency for the video codec is a critical requirement.

W&W Communications has taken a different approach to the implementation of H.264 video codecs with its Taos architecture. By using inherent pipelining in the H.264 video coding standard, Taos is optimized to achieve extremely low encode-decode latencies of sub-2ms, between the source video and the rendered decoded video. The low latency does not compromise the video quality. No "corner-cutting" takes place in the encoder and decoder, and the resulting high-quality video is within 2-5 percent of the International Telecommunication Union Joint Video Team's Joint Model results.

- Kishan Jainandunsing
VP of Marketing
W&W Communications Inc.

Article Comments - Managing latency in video encoding, ...
*? You can enter [0] more charecters.
*Verify code:


Visit Asia Webinars to learn about the latest in technology and get practical design tips.

Back to Top