InformationWeek

But Schwab's IT management isn't backing off its mainframe-oriented E-commerce strategy; it's moving full-steam ahead. Schwab is also keeping its aggressive E-commerce stance and extending the services it offers over the Internet. At the same time, the company must keep up with transaction volume that has grown dramatically in recent months and is subject to volatile swings on any given trading day. These are problems most companies are only now thinking about.

"We're getting to solve some really big problems that, frankly, other companies are going to face," says Schwab CIO Dawn Lepore. "We're getting to them first."

Next quarter, Schwab will begin a migration from batch processing to real-time processing of trades on its mainframes. Schwab will also redesign its Web site and add services and technology for active traders. By May, the company will add two mainframes to the three IBM and three Hitachi mainframes in its primary data center. It will also dramatically increase the scale of its Web system from 88 IBM RS/6000 Web servers to 154.

Over the next year, the company will finish rewriting most of its back-end software programs, such as those supporting balances, positions, and trading, mostly in Cobol wrapped in object-oriented Java code. Also, Schwab plans to work with IBM on advancing the vendor's Sysplex technology, which Schwab hopes will eventually allow it to share processing across mainframes at both its primary and backup data centers. Schwab currently uses Sysplex to perform what is essentially load balancing across the six mainframes at its operational data center, with six mainframes in its backup data center providing redundant capacity.

With $200 billion in online assets, Schwab dwarfs online rivals such as E-Trade Group Inc., which holds $15 billion in online assets. Customers checking on their Schwab assets generated more than 1 billion hits on the company's Web servers in January-not a minor matter for a site with the security and authentication requirements of a financial services company.

"Relative to the competition, they've had fewer problems than most and they process more trades than anybody," says Bill Burnham, an online brokerage expert at investment bank CS First Boston. "For the volumes they're processing, they've done a good job."

Competition Rises
Still, Schwab can't afford more outages. Due to increased competition, Schwab's share of online trades slipped from 30% in the third quarter to 27% in the fourth quarter last year, according to CS First Boston. Schwab's share could slip further if the brokerage doesn't meet expectations for high-quality service implicit in its $29.95 commission rate, which is higher than most competitors' fees-and company executives know it.

"We want to have a superior customer experience where any downtime will be transparent to customers," says Schwab co-CEO David Pottruck. That means getting backup systems to kick in after only a few minutes, instead of the 15 to 20 minutes it currently takes, says Pottruck.

Schwab IT execs hope using IBM Sysplex technology to share database processing across data centers will eventually provide that seamless backup, even if one entire data center goes down. But they aren't there yet. When complications from adding the third Hitachi mainframe caused online trading to halt on Feb. 24, Schwab IT executives elected not to use their backup mainframes because they thought they could solve the configuration problem relatively quickly. "We would rather have had a 20- to 30-minute outage than deal with the transition to the backup system and the transition back" to the primary data center, says Frederick Matteson, executive VP of IT. "It turned out to be a 90-minute outage. There was a lot of learning in that."

That and another recent failure came as company executives were boasting that their mainframe architecture was more reliable than competitors' client-server systems. CIO Lepore said in an interview on Feb. 26 that despite the outage two days earlier, Schwab had avoided the persistent problems plaguing its competitors. But when markets opened the next trading day, Schwab customers were unable to submit trades for about 20 minutes via the Web because of a problem with IBM utility software as Schwab added storage discs to one of its IBM mainframes.

"Schwab did take it on the chin just as they were crowing that their mainframes were strong," says John Robb, a principal at Gomez Advisors, a consulting firm that ranks online brokerage firms.

Still, Schwab is betting that big iron will carry the company as it adds to the 2.2 million accounts already Internet-enabled among its 5.7 million active accounts. Schwab is working to better integrate its IBM RS/6000 Web servers with its mainframe technology. For example, Schwab is upgrading its mainframes from the MVS operating system to OS/390 so they can handle direct TCP/IP connections from the servers. Also, Schwab is looking to move a layer of middleware that handles HTML-to-CICS conversion off the RS/6000 servers and back onto the mainframe, according to Jan Hier-King, senior VP of electronic brokerage technology.

Friendlier For Users
To deal with criticism of its Web site, Schwab will begin a major redesign of its home page next month that executives say will organize content in a more intuitive fashion. "We haven't done so well at creating a complete experience," admits Gideon Sasson, president of the electronic brokerage division.

A more ambitious initiative will come later in the second quarter. Schwab has already moved many accounting functions, such as those for cash debits and credits, from batch to real-time processing on the mainframe. It will soon begin the critical task of reconciling securities trades in real time. "Where we want to get to is where the system runs 24 hours a day and there's no real differentiation between what goes on in the day and what goes on in the night," says Geoff Penney, executive VP of financial products and international technology. "We're a couple of years away, but we're getting closer and closer."

Adding mainframes and Web servers while reworking software and Web sites requires additional resources. CIO Lepore says Schwab will increase IT staff this year from 2,000 to about 2,400. Co-CEO Pottruck says IT spending may rise from 16% of revenue to around 18% over the next few years.

Schwab's aggressive moves to add capacity and deal with online traffic are rippling out to other companies engaged in E-commerce. "We don't hesitate now when it comes to struggling with decisions to scale our site," says Bob Perkins, executive VP of ticketing at Ticketmaster Online-CitySearch Inc. "We know it's not just us. It makes it easier for us to justify the expense of adding bandwidth, adding additional servers."