Amazon Patents Have Designs On Augmented-Reality Living Rooms

8 Ways You're Failing At Data Science

During its history, Amazon has managed to conquer the book market, traditional retail, and, through Amazon Web Services, cloud computing. Now, thanks to two patents it was awarded on Dec. 1, it seems the company has more plans for your living room than just its Echo voice-controlled device.

The US Patent and Trademark Office granted the Seattle company a patent for technologies that can project an augmented reality representation into a room (#9,204,121) and another patent that provides a way to navigate that world (#9,201,499).

The patents were submitted by Rawles LLC of Wilmington, Del., in February 2013 and officially assigned to Amazon this week.

One of the patents issued on Dec. 1 concentrates on determining the depth of an object in a scene.

It describes a system that includes a single light source with an attached shutter, and ellipsoidal reflection, and a camera. The light source is directed to the scene and toward the reflector. The camera captures the first source light as well as the reflected light. Comparison of both light beams will yield a determination of the depth of an object within the scene. It discusses at some detail how an augmented reality functional node (ARFN) can be used to generate and augmented reality environment. There may be multiple ARFNs within the environment, which can interact.

The other Dec. 1 patent tracks the movement of an object in a 3D environment -- such as a hand -- which is then used for "gesture input to an augmented reality system." The 3D images are captured and can be used to generate 2D contours of that object. The overlap in the 2D regions of the object can then determine if the object is the correct one to track.

It is also tied to the use of the technique inside ARFN-enabled environments.

When used together, these two technologies outlined in the patents could simultaneously generate an augmented reality environment (by depth determination) and track user input by the gestures used in that environment.

To gain some insight into what Amazon may or may not be doing with these patents, I emailed SmartUp Legal, a legal firm that follows patents in the IT and technologies market. Mikhail Avady, the company's co-founder and current CMO, wrote that these patents relate back to Lab126, Amazon's subsidiary for research and development. Avady noted:

Amazon seems to be using an entity called "Rawles LLC" for their Lab126 division (the team responsible for Kindle, Dash, Echo etc…). The way we determined that these two organizations are related are through the patent filings. A couple of patents surfaced with Amazon and Rawles as co-applicants, as we dug deeper we found that the inventors on each of these applications are current or past employees of Lab126. We also noticed that most if not all the patents from Rawles were transferred over to Amazon on 11/12/15.

This isn't the first time that Amazon has dipped its toes into augmented reality, or AR, for one business purpose or another.

In October, Amazon was granted another patent (#9,158,115) for a head-mounted display that featured a way to toggle from opaque to transparent when interacting with a tablet. It may be that this is not going to be a public-facing product, since such a device could be more useful in Amazon fulfillment centers, for example, as a way for workers to scan barcodes.

The newest patents take a totally different approach, projecting the AR into physical space and then allowing the user's gestures to manipulate the AR.

[Read how Wikipedia is using AI and machine learning to edit.]

In some ways, this effort is similar to what Microsoft has done with the RoomAlive project, which uses projectors and Kinect technology to construct an AR environment.

"We do not know if it [the patents] is a response to Microsoft, but, it does seem to fit their vision," Avady added. "As we saw with Echo and Dash, Amazon wants to be in your living room and these patents were invented by the same division as those."

**New deadline of Dec. 18, 2015** Be a part of the prestigious InformationWeek Elite 100! Time is running out to submit your company's application by Dec. 18, 2015. Go to our 2016 registration page: InformationWeek's Elite 100 list for 2016.