Xbox Kinect Converted To 3D Video Camera

Xbox Kinect

A University of California Davis researcher has figured out how to turn the Microsoft Xbox Kinect into a 3D video camera.

The device was hacked a little over a week ago and then researcher Oliver Kreylos spent three days working on capturing objects in three dimensions using the Kinect's two cameras. He then reconstructed the objects in real time "from scratch" in C++ using his own Vrui virtual reality (VR) toolkit for 3D rendering management and interaction.

A 3-D model of the room and all of the objects in it was created when Kreylos merged data from one camera providing a photographic video stream with data from a second camera providing the depth video stream. The software interpreted points created by the depth camera to help create a 3D image that could be moved around.

Some observers have theorized that a clearer image could be now achieved with the use of additional Kinect systems and an algorithm to mesh the four streams.

Kreylos said he based his efforts on Hector Martin's (marcan42 on Twitter and YouTube) reverse engineering work. Last week, Martin won $3,000 from open source hardware developers Adafruit Industries after he was declared the first person to successfully create an open source driver for the Kinect, a motion-sensitive controller. Microsoft said it did not "condone the modification of its products," which prompted Adafruit to increase the prize to $2,000 and then eventually $3,000, CNET reported.

"I didn't use any of his code, but the 'magic incantations' that need to be sent to the Kinect to enable the cameras and start streaming," Kreylos wrote on his website. "Those incantations were essential, because I don't own an Xbox myself, so I couldn't snoop its USB protocol."

A virtual reality development toolkit's mission, according to Kreylos, is "to shield an application developer from the particular configuration of a VR environment," so that applications can be developed in a quick, portable, and scalable manner.

Kreylos said his next project will be to create an augmented reality environment by incorporating real and computer-generated imagery. His plan is to figure out if he can use the 3D views and insert them into another 3D environment to blend the realistic people he developed with the Kinect with computer-generated imagery.

Source code is now available on Kreylos' download page, although the software will not build until Vrui-2.0 is released, which he said should happen soon. He said he thought the source "might still be of interest to some people."