Given all of the headwinds Sears faces, should its CTO be spending much of his time building a startup? Sears Holdings had operating losses four of the last five quarters. Sears chairman Eddie Lampert, whose hedge fund owns more than 60% of the company, started his chairman's letter back in February with this assessment: "Our poor financial results in 2011, culminating in a very poor fourth quarter, underscore the need to accelerate the transformation of Sears Holdings."

In that environment, launching a technology startup is risky, given the potential for distraction from meeting the IT needs of a $42 billion-a-year retail business that includes Sears and Kmart. With the MetaScale venture, CTO Phil Shelley is looking to take advantage of Sears' broad experience with using the Hadoop big data platform. MetaScale sells subscription services to manage large data sets using Hadoop, and it offers big data consulting.

Shelley doesn't see such a startup as risky. Lampert, he says, sets the expectation that executives need to make such moves. "It's a very innovative environment," Shelley says. "The concept of generating new business, a new business model, a whole new business, is very much encouraged."

Sears is doing cutting-edge work when it comes to Hadoop and big data management. Some of the most practical work--work that other big companies might buy as a service--is moving big batch processing data loads off of mainframes, cutting hours of processing time. Applied to Sears' own business, eliminating mainframes could save tens of millions of dollars.

But when it comes to applying big data tactics to change Sears, it feels like Sears could be doing more and moving faster. One of the most promising areas is for customized marketing promotions, using Sears' growing loyalty card program to know what customers bought in the past and what they're now buying, and to give them an intensely relevant offer to get them to buy more. Sears is just beginning to do that kind of personalization at scale.

"You're starting to see that much more personal, targeted, digital engagement," Shelley says. "It's a big company to change, so it will take awhile, but it is changing."

A critical advantage of Hadoop, Shelley says, is its ability to let companies keep and analyze all of their data. Whereas Sears used to analyze 10% of data on customers to figure out which promotions might work, now it can analyze all data on them. Because it's cheaper and faster to keep and analyze data, he says, it's collecting more of it--such as data coming into Sears' call center about which appliances are breaking and how often. But Shelley doesn't offer a clear example of how Sears is putting that kind of data to profitable use.

One argument in favor of Sears doing a startup like MetaScale is that Amazon.com, the most feared company in retail today, is doing its own tech startups. The e-commerce giant's Amazon Web Services arm pioneered the sale of commodity infrastructure-as-a-service, letting companies buy computing capacity by the hour with only a credit card. Sears is attacking a niche in the cloud computing market: high-end, specialized Hadoop workloads.

A more powerful argument in favor of MetaScale is that it forces Sears to stay on the leading edge of big data management and analytics, and lets it learn from big companies that are innovating in other industries while also driving some revenue. Shelley won't say how many clients MetaScale has, but he refers to a major healthcare company and another in financial services.

Retail Must Change

You can't walk into a Sears store today and really feel how Shelley's big data efforts have changed the shopping experience. But Sears isn't alone--up against this challenge is every single big retailer: Best Buy, J.C. Penney, Wal-Mart, Target, Home Depot, Lowe's. Every one of them needs to figure out how to make in-store shopping so appealing that customers come to their stores to make purchases rather than to just look around and then buy from discount competitors online.

It's no exaggeration to say that Sears' survival hinges on its ability to figure out how to serve customers across store, Web, and mobile channels.

Lampert, in his chairman's letter, listed the five pillars of Sears' business (see p. 22). One is "reinventing the company continuously through technology and innovation." Lampert said he spends more of his time on that pillar than any other. He realizes that people will soon, instinctively, reach for their smartphones as they shop in stores. "How people shop today is changing, and it isn't just the younger generation that is benefiting from iPads, Facebook, and online retail," Lampert wrote.

Retailers have yet to take truly daring steps to create this cross-channel experience. But imagine stores geofenced by Wi-Fi, so that loyalty card customers' smartphones automatically notify the store when they walk in. Sounds creepy at first, but that's exactly what many people have set up on Amazon. Give people a compelling reason to set that kind of functionality up in a physical store--say, to receive customized offers on their phones--and they will.

How about changing prices multiple times a day? Sears is building the data analytics necessary to make those kind of dynamic price and inventory decisions; like other retailers, it's also experimenting with digital price signs in stores that would make such changes feasible. Again, dynamic pricing would be a dramatic change, but perhaps one needed to compete with online retailing.

Retailers' survival depends on these kinds of changes in the mobile e-commerce era. If Sears' MetaScale work helps it figure out the omni-channel shopper, its startup will succeed. If it doesn't, or it distracts Sears from this mission, it will have failed because there won't be a Sears or Kmart left.

Download a free PDF of InformationWeek magazine
(registration required)