Boeing's 3D Projects Invigorate IT, Manufacturing

In one of Boeing's Washington factories, where the company makes a military tanker airplane, employees are trying out a very strange-looking tablet PC. The tablet itself is normal, but it has more than a dozen quarter-inch silver balls connected to it.

In this awkward setup, Boeing sees the future of augmented reality on the assembly line.

Those silver balls do the very difficult job of tracking objects in 3D space. Six infrared cameras track the balls to know precisely where that tablet is. Patent-pending algorithms written by Boeing make sure the image and reality match.

Looking through that tablet, a mechanic can see the real-world torque box unit he or she is assembling, and see digital parts, arrows, or instructions to supplement that real-world view. Move the tablet, and the digital elements adjust so they're properly aligned with the real-world view.

CIO Ted Colbert

The 3D augmented reality system is only a pilot project today and not part of everyday aircraft production. But the Boeing mechanics who've tried the system instantly see the potential.

"As soon as you pick up the tablet and see this, there's a huge 'wow' factor," says Paul Davies, a technical fellow in the Boeing Research and Technology group, who's worked on the project more than four years.

The augmented reality system is an example of what CIO Ted Colbert sees as an emerging new level of tech innovation, one which has the potential to markedly improve Boeing operations and financials.

Boeing's Digital Strategy

Colbert thinks Boeing's IT team is "feeling a bit of a rebirth."

Like many IT organizations, Boeing IT for many years focused on building its operational foundation -- ERP, corporate intranets, and the first rounds of mobile technology. That work never ends, but Colbert says he feels it's strong enough that IT can "lean forward and really drive business enablement and innovation in a way that we have never been able to do it before."

The Internet of Things, advanced analytics, additive manufacturing, and mobility all offer huge potential for cost savings, increased output, and improved safety for a global manufacturer like Boeing. "I'm not talking every day per se about unit cost performance or mobile device challenges," Colbert says. "It's business leaders pulling on our organization to help enable the digital strategy, or the next-generation business models. That's exciting."

[ Want more innovation ideas? Read about all the Elite 100 winners. ]

Now, Colbert says he wants to see his IT team challenging business unit colleagues with new ideas. If an advanced analytics capability or a mobile collaboration tool worked well for one part of the company, he wants his IT leaders embedded in business units to push for its broader use. "If there isn't a big, huge challenge in front of them, they get entrenched in the business and begin to order-take a little bit," he says. "You have to sort of stir up those folks."

An augmented reality view, with digital designs on a real component

But building breakthrough technology, and getting it to perform reliably on the global scale that Boeing needs, isn't quick or easy. The augmented reality project shows how such work takes investment, time, and commitment.

The Limits Of Augmented Reality

Moving augmented reality from concept to aircraft factory floor requires a technological leap. "You see a lot of concept videos, and they look really flashy, but when you actually experience it a lot of times it doesn't live up to the hype," Davies says.

Two major obstacles remain.

First, there's no ubiquitous way, like a WiFi network, to blanket an area for 3D tracking. Boeing uses the infrared cameras in the torque box bay, but that's not practical on a massive scale, as in examining the entire fuselage of a 747. "Tracking something in 3D space is a very difficult thing to do," Davies says.

Second, the 3D display options aren't ideal. "A lot of folks want wearable technology," Davies says, but none of the head-mounted displays are robust enough today. The dream state would be a heads-up display built into the safety glasses that people already wear on the production line, but nothing's close to that today. For now, tablets are a reasonable starting point for Boeing. Mechanics pick up and put down tools all day long, so this could be treated the same way.

Software provides the third building block of augmented reality -- graphic rendering, CAD models, and user interfaces. Davies considers this area a solved problem, and not a significant obstacle.

To test the benefits of augmented reality, Boeing partnered with Iowa State University last year. ISU put about 45 students through assembly projects using either paper instructions, instructions on a handheld PC, or instructions using augmented reality. Boeing tapped ISU for its experience running controlled experiments, so the results carried more credibility. The company found that the augmented reality group had significantly fewer errors, and they took less time to build than the other groups. That data provided the business case to move the pilot into a factory and a step closer to real-life testing.

Boeing thinks augmented reality could drive down costs and increase first-time quality on many fronts -- reducing training time, speeding up build times, making it easier to shift employees across tasks, and aiding the transfer of "tribal knowledge" from retiring workers to newbies. But the process has only just begun.

"It's about whether the technology is ready, but also is the culture ready," says Kristy Heffernan, an IT director who acts as the IT liaison with Boeing military aircraft.

Boeing has gone through such culture shifts before, for example, moving from paper to more digital designs and instructions. Boeing has a hands-on culture, so one key is showing employees new tools and how they will help get the job done. "The unknown is what they fear more than the known," Heffernan says. Mechanics weren't so sure about using tablets in the production environment, she says, "and now they won't let them go."

Augmented reality technology isn't ready for a live assembly line yet. But Davies thinks people are increasingly open to an interface that's more than a 2D glass screen, and are ready to accept 3D displays that place digital images in the real world in some way. Now, he and the rest of the Boeing R&D, IT, and manufacturing teams just have to get augmented reality reliable and flexible enough for a high-stakes factory environment. Says Davies, "We're only a centimeter or an inch along a journey that's going to be a mile long. There's so much work to do here, and so much benefit."