Global Sources
EE Times-Asia
Stay in touch with EE Times Asia
EE Times-Asia > Embedded

Enabling augmented reality in automotives

Posted: 21 Jan 2015 ?? ?Print Version ?Bookmark and Share

Keywords:Surround view monitoring? parking assistance system? ADAS? 3D? R-Car?

Surround view monitoring will become a common feature in vehicles. This functionality is part of the parking assistance system. From a niche market first driven by Asian car makers, it has become an option offered by the majority of car manufacturers, with the consequence of a higher requirement in terms of driver experience and solution scalability. Renesas is already supporting surround view requirements in their early phase. Today, Renesas provides a new generation of SoC touted to answer global market needs with a scalable and innovative approach.

Surround view with R-Car Gen2 families
The purpose of surround view monitoring is to display a panoramic view of the car's immediate surroundings. This representation, at 360 degrees with 2D perspective from the sky, is called "bird view" or "top view". The different views are stitched together thanks to the correct geometric alignment of the cameras. The brightness and colour of the different cameras' videos are modified for the harmonisation of the surround view [1] [2].

Nevertheless, displaying only this representation does not generally help the driver during the parking process. To facilitate this manoeuvre, additional information can be shown to the driver as 2D overlays or rear view [1]. A complementary approach is to improve the driver apprehension of the distances with a 3D representation of the car's surroundings. The target is to use 2D cameras around the car to create a 3D comprehensive representation of its immediate vicinity with a 3D generated car as a driver perspective reference. It should reflect a realistic representation of the distances to nearby elements (pedestrians, cars and buildings). The 3D sphere perspective should dynamically change according to the car movement. The model car has to be properly integrated in the overall scene with light or reflection on the model car [2].

This level of application drives the performance required in terms of 3D graphics and computer vision in an automotive embedded platform. Renesas created the SoC family called R-Car in order to enable this level of applications. The second R-Car generation was first officially released in March 2013 and supports a wide variety of applications, such as connectivity, entertainment expansion and ADAS. This family provides outstanding performance with optimal power consumption capabilities [3] and a common API for reducing customer development efforts. From this family, two devices support surround view application: the R-Car H2 and the R-Car V2H.

3D experience with R-Car H2
R-Car H2 is the first device released in March 2013 and tailored to integrated cockpit solutions with the 3D surround application. For this utilisation, we first need to consider 3D graphic engine performance requirement. We should particularly pay attention to the two parts of the scene: the texture mapping of the 2D camera images on a 3D sphere and the 3D car representation. The polygon count of the scene depends on the deformation of the 3D sphere and the rendering effects on the car model. For better rendering, the graphic engine must be able to process a significant polygon count in a short time.

Moreover, as the application can use different shader programs for one scene, the graphic engine must come with a powerful shader engine. Those performance requirements must be supported by a high GPU frequency, which will allow fast data processing. All these performance aspects justify Renesas' decision to integrate an outstanding 3D graphics engine in R-Car H2. Indeed, its 3D graphic engine provides similar performance than the latest iPad Air 3D graphic engine.

The path towards augmented reality
The sensing of the scene in 3D is the other important aspect required to provide easy to understand content. This can be achieved with two techniques. The first is human-like stereo vision, although it has the disadvantage of double the camera cost and integration effort. The other option is to create the Structure from Motion (SfM) of the car, thus providing stereo vision over time. Renesas has implemented vision-dedicated hardware accelerators into the R-Car family to power this algorithm on the four cameras in real-time, meeting both performance and low power consumption requirements.

1???2???3?Next Page?Last Page

Article Comments - Enabling augmented reality in automo...
*? You can enter [0] more charecters.
*Verify code:


Visit Asia Webinars to learn about the latest in technology and get practical design tips.

Back to Top