Taking a look at the embedded vision market

The never-ending megapixel race among CMOS image sensors no longer defines how imaging technology progresses. As market focus shifts to "vision" processing, the industry has drawn a new battle line鈥攐ver how fast and how accurately a processor can capture, dissect, and interpret data in a manner comprehensible to an embedded system.

In the embedded vision world, what matters is not so much you, the photographer, who wants to take better photos. Instead, the technology now exists to cater to embedded systems that need to watch you, recognize who you are, analyze your behavior, and process data they think you need.

You might call this just the plain reality of technology progress in machine vision or computer vision. Maybe so. But I confess that some of the embedded vision plots hatched by marketers are disturbing enough to make me cringe.

None of this stuff, of course, is more worrisome than the NSA's electronic spying programs. But the very notion of a bunch of sensors physically watching me鈥攕olely to make a commercial gain at my expense鈥攇ives me, at least, a slight case of the willies. At worst, it's a reminder of the increasingly Orwellian society we live in.

Over a cup of coffee in Tokyo, I recently sat down with Tom Wilson, VP of business development at CogniVue, a Quebec-based embedded vision technology developer. Wilson tried to convince me that automotive isn't the only market being targeted by vision processing technology developers such as CogniVue.

Wilson pointed out that Mirriad, a developer of ad platforms, is one company working on such a project. "The plan is to couple this type of ad insertion with viewer preference," he explained. In fact, a set-top box with eyes isn't such a far-fetched idea. Mirriad recently signed a deal with Pace, a set-top box vendor, to trial this in the UK, according to Wilson.

While explaining the digital product placement scheme, Wilson joked that this is partly why he doesn't own a TV. But he made sure that I understood the far-reaching ramifications of embedded vision applications and how the competition among embedded vision IP vendors, both software and hardware, has been escalating in recent years.

CogniVue, Mobileye, CEVA and Tensilica (now a part of Cadence) are just a few examples of IP companies enabling embedded vision technologies. The newest member to join the fray is Imagination Technology, which announced its PowerVR Raptor ISP (image signal processing) architecture.

Chip companies such as Freescale, Texas Instruments and STMicroelectronics are also rolling out purpose-built vision processors, often taking advantage of their partnership/licensing deals with embedded vision IP vendors.

For the time being, though, automotive is the primary market for all these vision processors, since embedded vision is playing a key role in Advanced Driver Assist System (ADAS). Carmakers are banking on ADAS, advocating safety features such as lane departure warnings, collision mitigation, self-parking and blind-spot notification.

"Vision processing still remains as a very hard problem to solve," stated Jeff Bier, founder of the Embedded Vision, "despite the number of man-years spent developing a host of embedded vision algorithms."

CogniVue's Wilson agreed. Processing a huge amount of real-time data demands intense compute power. To do a 3D sensor map in a robust manner, especially in a low-power consumer device, is especially tough, he added.

Asked why a 3D sensor map, he described it as "essential" to solve fundamental limitations in 2D computer vision. He noted that 2D, for example, has problems with segmentation (separating foreground from background), illumination (for face recognition), relative position (placing objects in the scene) and occlusion (hands in front of the face). Noting that different approaches for 3D sensing are fraught with tradeoffs, Wilson said that CogniVue is working on an algorithmic way to efficiently compute disparity maps for low-cost 3D sensor vision.

Designing hardware that can efficiently run different vision algorithms is a huge challenge for system designers. Options for system vendors looking for imaging/video processing solutions range from keeping it all in the CPU to offloading imaging to the GPU, or adding hardwired logic dedicated to imaging functions.

With Imagination entering the vision market, the race among IP vendors and chip suppliers has only gotten even more intense. There is no question that it's going to be a "Brave New World."