Server Consolidation Delivers

Overhauling a data center and consolidating servers can mean big benefits, but it also can be dangerous. Just ask the Chicago Tribune, which undertook a major server-consolidation project--and survived a big glitch.

The nation's seventh-largest newspaper last year moved its critical applications from a mishmash of mainframes and older Sun Microsystems servers to a new dual-site data-center infrastructure based on Sun 15K servers. In a unique server-consolidation project, the Tribune clustered the servers over a two-mile distance, lighting up a 1-Gbps dark-fiber link--optical fiber that's in place but not being used--between two data centers. This configuration let the newspaper spread the processing load between the servers while improving redundancy and options for disaster recovery.

Unfortunately, the cutover to the new server environment hit a snag: A small piece of software written for the transition contained a coding error that caused the Tribune's editorial applications to experience intermittent processing failures. As a result, the Tribune was forced to delay delivery to about 40% of its 680,000 readers and cut 24 pages from a Monday edition, costing the newspaper nearly $1 million in advertising revenue. Don't ever let anybody tell you that data-center renovation isn't risky.

More than nine months later, the Tribune's IT department is still sold on server consolidation, despite the high-profile outage. "One five-hour delay doesn't change the fact that this project has brought us significant cost savings and improved our system reliability for the long run," says Darko Dejanovic, VP and chief technology officer of Tribune Co., which owns the Chicago Tribune, the Los Angeles Times, Long Island's Newsday, and about a dozen other major metropolitan newspapers. "There's no question that server consolidation was the right move for us."

Following the outage, the Tribune's IT staff re-examined its processes and made a few changes. CCI Europe A/S, the Tribune's editorial-applications vendor, now maintains a more regular on-site presence in the data center. Problem-escalation policies have been changed to help users raise concerns more quickly. Application-testing processes have been modified to incorporate more thorough testing by business users. And to the IT staff's credit, the consolidation project not only stayed on track, it finished a few weeks ahead of schedule and about 10% under budget.

After editorial applications were stabilized, the Tribune proceeded to migrate apps for operations--that is, the physical production and printing of the newspaper--and circulation to the new Sun server environment. Both made the transition without a hitch. By the end of August, all the applications scheduled to run on the new servers were operational.

Though the Tribune's consolidation project is largely complete, that of its parent is just getting started. For one, Tribune Co. has consolidated all its mainframe applications at its Long Island, N.Y., data center, decommissioning its one remaining IBM System/390 in Chicago. Mainframe processing, mostly for billing applications that don't run on Solaris, is centralized on Long Island, and Chicago staffers access them through remote 3270 terminal connections.

"As we gradually took [other] applications off the mainframe, we realized that we were incurring very high costs in maintaining underutilized mainframes at two different locations," Dejanovic says. "By moving from two locations to one, we've achieved several million dollars in cost savings."

The consolidation effort is beginning to reach into other Tribune properties as well. One of the company's other newspapers is planning a move similar to the clustered Sun server configuration in the near future, Dejanovic says, though he declines to say which one. Several of the newspapers have begun to explore ways to use one another as backup sites, though the lack of WAN-distance dark fiber limits the newspapers' ability to do clustering, as the Chicago Tribune did.

"There are some gigabit-speed services available in the WAN, but they're very expensive and there aren't many choices out there," says Milind Dere, one of the project managers for the server-consolidation effort.

And while it's consolidating hardware, Tribune Co. also is looking to consolidate software, Dejanovic says. Currently, each newspaper maintains its own applications for classified advertising and billing, which means the parent company must support about 10 billing packages and the same number of classified-ad programs.

"Most newspapers feel that they have their own unique processes for handling those functions, but we've found that most of the business processes can be standardized," Dejanovic says. So far, the Tribune newspapers have standardized about 95% of their classified-ad processes and about 90% of their advertising-sales processes. Over the next three years, Tribune Co. will replace the disparate billing and ad applications across the company with a single package that will be used by all the business units.

Back at the Chicago Tribune, some new projects are in the works as well. One is the implementation of Sun Solaris 10, which will let individual apps run on partial CPUs, freeing up processor power and making more efficient use of the Sun server cluster. Solaris 10 will make server consolidation an even more attractive option for Sun users, Dere predicts. "That capability will bring us closer to realizing the full potential of the Sun environment, both in terms of server capacity and clustering," he says. Sun is shipping Solaris 10, but the Chicago Tribune won't do the migration until more of its specialized newspaper-industry applications are available on the new operating system.

What advice does the Tribune's IT staff have for other IT departments considering data-center renovation? "Don't hesitate," Dejanovic says. "We were convinced it would be cheaper to consolidate, and we realized those savings. And don't underestimate dealing with one vendor. We've saved a lot by negotiating good maintenance contracts and lowering our overall cost of support."

Scott Tafelski, director of publishing technology for the Tribune Co., suggests that server consolidation will become a popular trend among large enterprises. "Tomorrow's data center will look a lot like yesterday's data centers, in that a lot of the technology will be centralized," he says. "From a cost perspective and a management perspective, it's a direction that makes a lot of sense."