Recently there have been a few interesting projects that are involving using AR and VR technology in cars. Two such examples are the ride-sharing service, Lyft, and Audi’s announcement at CES 2019. What has fascinated people about both of these updates is how they could possibly redefine travel. Furthermore, they could expand entertainment and educational options and even attempt to resolve the motion-sickness issue commonly accustomed to virtual reality design.

A Shift in Perception

People have noticed that virtual, augmented and mixed reality are no longer confined to games and entertainment. “VAMR” is bound to touch many aspects of daily life, from shopping to media consumption. Autonomous vehicles are a new advancement showing how VAMR is currently being used. With simulated testing grounds and complex algorithms, the future of driving is completely changed. Developers are trying to compensate for the motion of an automobile on highways as well as easing the queasiness of walking around a room in a headset.

The Simulated Testing Ground

MR Prototyping is providing a safe testing ground for these autonomous vehicles. Although these are yet to be perfected, the Mixed Reality Lab at USC has started using MR Prototyping to explore human-machine teaming. This basically means that the lab has successfully paired people with autonomous drones. Yes, actual drones.

Historically, traditional research and design would pair real drones with real people and although this was a pretty cool development, it was also dangerous. Now, the algorithms and math is all sorted out. The virtualized aspects of pairing has been perfected within any virtual environments imaginable.

The Idea of Mixed Reality

Holoride is where VR meets the real world. It is now part of a subset of VR known as mixed-reality (MR), because it is mixing virtual and real worlds. The specialty it possesses is that it does not enter the terrain famously known to augmented reality. Promotion of this concept showed a user put on a headset for accessing different experiences. This includes driving on the streets of a brightly colored made up city, flying through a Jurassic landscape as a pterodactyl and even shooting missiles from a careening space fighter.

Autonomous Vehicles
If these cars stop at a pedestrian crossing the road, the passenger in VR, is able to play a whack-a-mole-type game with the birds that walk across it. If the car drives along a winding road, the driver can veer and dip through the woods or through space. Holoride has claimed that it is "turning vehicles into moving theme parks". Since autonomous driving is still up and coming, companies are having trouble marketing to the masses. An idea that is simple in theory, its inner workings have still not been explained clearly to the public.

Civil Maps is another company working for this promotion. It’s a startup that focuses on making crowdsourced maps for autonomous cars and has implemented AR to develop solutions. Basically, this new technology will display the “brain” of an automated vehicle. It is a standard practice, but Civil Maps is even providing a visual representation of the eyes of the car through a monitor.

The Science behind these Vehicles

There is an order that goes into producing these types of complex images. Millions of pixels make them up at the required frame rates and extremely powerful computing elements are needed. With a fast and reliable storage architecture, many VR and AR applications have been using sophisticated 3D models, and even stream 4K-quality video. They have made dynamic changes to the display in real time with the levels of detail that were not needed before.

Another thing that makes VR and AR science so exciting albeit challenging is the visual processing demands like fast graphic processors (GPUs). These technical issues are an addition to fast regular processors (CPUs), and storage components that can keep up with the complicated requirements for VR and AR.

The Latest Example Set By Volvo

Most recently Varjo has announced an impending partnership with Volvo. This will enable safe driving using an AR headset that users can wear while operating a car. It can give them a visual alert by using computer graphics to highlight any dangers on the road. Volvo is employing Varjo’s XR-1 developer edition headset, so users can see what is going on in the outside world through high quality cameras. Volvo in Sweden is also investing in Varjo.

It has been pointed out that this kind of technology will eventually make it difficult for people to tell if they are sitting in a fake car or in a real one. Their terrain will also be hard to distinguish from reality. Engineers at the Swedish carmaker have already begun to work on this.

Varying Forms of Reality

Within these new high level simulations the notion of “reality” can vary. It has been noted that geo-specific terrain can be used if needed so the design parameters and algorithmic problems would be to a minimum. The association with human-drone pairing can now be resolved in the virtual space. The reality will then get sort of mixed where some of the elements are in both the virtual and the physical space. The humans will remain exclusively in the virtual space. After a few more tweaks, the system can go fully physical and the algorithms will be well behaved.