Why Mixed Reality is Messed Up, for Now

Expectations of AR and VR have been high, perhaps too high. While the idea of creating immersive experiences for enterprise applications inspires all sorts of fascinating use cases, AR and VR have not taken off as rapidly or pervasively as some expected. Part of the problem is focusing on the possibilities without considering the practical aspects such as cost, safety, comfort and relative ROI.

Even the technologies themselves can be confused in conversations. AR superimposes virtual elements on real-world elements via a smart visor, helmet or glasses. The interfaces may be touch, gesture or voice controlled, although the latter two are preferable since touching glasses seems counterintuitive if not messy without gloves.

Popularly-referenced use cases are architecture and interior design. For example, a home buyer intending to build a house on a vacant lot can view different house designs as applied to the space. Similarly, after the house is built, different interior designs can be applied to bare rooms. Sotheby’s now has an AR-enabled staging app. Using VR, a client can walk through a house design.

AR supplements what exists, VR creates a new and immersive 3D reality.

How VR is used today

VR’s most popular use cases are gaming and military training, although enterprises also use the technology now. For example, AR and VR agency Friends With Holograms clients include Coca-cola, Accenture, Verizon, and Unity. The agency created a virtual training program on behalf of Accenture for social workers so they’re better equipped to handle the range of experiences they’ll encounter.

“VR allows enterprises to train people in ways they couldn’t otherwise be safely trained. For example, Verizon trains people how to climb a telephone pole and do wire repairs. You can’t just have them read a book or watch a video and send them out into the field,” said Cortney Harding, founder and CEO of Friends with Holograms.

Insurance companies are using VR to help adjusters handle flood-damage claims more effectively. Rather than reading a manual or watching a video about flood inspections, the adjuster can experience a virtual flooded environment first-hand. The result is fewer calls for help while out in the field and at least some personal experience spent in a flooded environment, even though it’s virtual.

Courtney Harding

Similar benefits apply to surgery. For example, Orthopedic Surgeon Michael Ast, MD, and the Hospital for Special Surgery in NYC have been evaluating the Osso VR platform for about a year. They are using the system to teach surgeons in training about surgical procedures and equipment.

“The point is to try to get new surgeons to go through, learn and practice the procedure in a virtual environment,” said Ast. “New surgeons or experienced surgeons can practice it on their own hundreds of times before ever performing the actual procedure on a patient.”

The point is to make surgical training simpler and quicker as well as safer for patients. Of course, doing virtual surgery does not feel the same as doing actual surgery, which is why VR supplements, rather than replaces, traditional practices such as dissecting a cadaver.

“The goal for this technology is to remove the learning curve of new procedures and devices. If we can shift the learning of those aspects into the virtual world, we can allow surgeons to concentrate on the more individualized aspects of surgical care when they do perform the surgery on a patient,” said Ast.

Michael Ast

Friends with Hologram’s Harding said her agency sees a 75% increase in learning quality and retention with experiential learning and a 76% increase in learning recall when using virtual teaching methods over traditional methods.

“If you go to a training seminar and you’re falling behind, you’re not likely to ask questions. With VR, you can go over the same thing 20 times at no additional cost because you’re not paying for trainers or space,” said Harding. “Also, when people are watching a video, they tend to be distracted, checking their phone, checking email. When you’re using VR for training, they have to pay attention because the experience is immersive.”

VR challenges

Cost is one of VR’s main drawbacks. There’s the cost of the headsets and the cost of creating the environment and characters. Harding said organizations can save money by building an environment once and then populating it with different characters over time.

Interestingly, what enterprises might not consider is the amount of money they spend on classroom and video training, and how effective those significant investments are versus VR. In other words, the approach to ROI isn’t as comprehensive as it should be.

VR can also cause negative physical effects. The headsets tend to be large and somewhat clunky, so they may not be comfortable to wear. Users also can become disoriented in and nauseated by the virtual environment.

“We estimate 10 million units have been sold, but people put them on for five or ten minutes and then stop using them,” said John Fan, founder, chairman, CEO and president of electronic component manufacturer Kopin Corp. “Even gamers are not using it. For VR to come back, the headsets need to be smaller and lighter, and the experience has to be there.”

John Fan

In other words, Hollywood’s portrayal of VR and the true state of VR are two different things. In addition to headset design issues, Fan said the environments themselves can be problematic because their scale is not the same as the real world. The virtual world (buildings. etc.) tend to be larger than their physical counterparts.

Designers may also cram so many content elements into an experience that it becomes too difficult or frustrating to keep the average user engaged.

“If you design it correctly, anyone should be able to use it,” said Friends with Hologram’s Harding. “Companies think it needs to be complicated or gamified, but really, it needs to be easy to use.”

AR faces challenges too

AR faces two problems akin to VR, which are the headset and the environment. Like VR, Fan said AR headsets are not necessarily comfortable, which is problematic for a wearable. Also, the virtual elements can get in the way of the real-world elements, cluttering the experience or even causing disorientation if the scale of the virtual element is larger than the physical element.

Another issue is aesthetics. People want to look good while wearing AR glasses. Of course, the use case impacts design. Fighter pilots’ helmets need to be designed for functionality, not aesthetics.

The interface can also be problematic. While voice and gesture interfaces are expected to dominate AR wearables, some of them use touch, which is counterintuitive.

In addition, AR, like VR, can be overdesigned to the point where the user is overwhelmed by the amount of information.

“It’s humans first, technology second, and that’s hard to describe to people,” said Fan. “People are trying to feed too much computer or sensor information to a user, but the user can’t consume it.”

Fan sees the demand for AR building in the sports industry for swimmers, cyclists, runners and others who want hands-free, up-to-the-minute performance information. For now, AR is being used in factories, and more medical use cases are emerging. Physical security is also a viable use case because smart glasses can use facial recognition to identify suspicious actors and activity.

How to approach AR and VR

AR and VR are sometimes approached as solutions looking for a problem, when the ROI depends on the outcome. Enterprises should start with the outcome in mind and then decide what technology or technologies will likely deliver the greatest amount of value versus other options. It may be that AR or VR is a supplement to, rather than a replacement form], what exists.

“People ask what they could do for $10K which is nothing,” said Friends of Hologram’s Harding. “They don’t understand the cost, and they don’t have a sense of what they want to do. You need a general sense of what you want to accomplish such as how to handle a natural disaster emergency.”

Also consider the fact that the interface may suggest use cases. For example, Harding said police departments want to use voice-activated VR to train officers while lawyers want to prepare clients for depositions.

The opportunities for AR and VR are virtually endless, but their ultimate success requires a confluence of enabling technologies, compelling product designs and results that prove the technology is a good investment. For now, enterprises are wise to learn the capabilities and limitations of AR and VR so they can better understand what can actually be accomplished with them.