InformationWeek.com

Welcome Guest. | Log In| Register | Membership Benefits

December 7, 1998

Related stories:

Tools For Growth In E-Commerce

Web Service: Ready Or Not

Changing The Rules

And from our sister sites:

TechWeb:WebTools Servers And E-Commerce Toolbox

Network Computing:LocalDirector 420 Balances The Load

Network Computing:Log-Analysis Tools

Network Computing:Seven Web Load Balancers Score With Round-The-Clock Access

Mega Web Sites

Big sites are becoming the norm for companies doing commerce on the Web-bringing a new set of IT challenges

By Justin Hibbard

Illustration by
August Stein

he rise of Internet commerce has a corollary in IT departments: the emergence of large, complex, and fast-growing back-end systems that handle the workload.

As traffic on business Web sites grows, the IT infrastructures required to support that growth are taking on the scale of full-fledged data centers. Busy Web sites--with their unpredictable traffic spikes and 24-hour availability re- quirements--bring a unique set of challenges. Few companies have mastered the problem.

The most popular Web sites handle in excess of 100 million page views a day. Market forecasts make it clear that those types of numbers will become commonplace. Revenue for most online retailers is growing by more than 200% annually, with total online sales topping $13 billion this year, according to Boston Consulting Group Inc. Over Thanksgiving weekend, traffic on CDNow Inc.'s music site tripled, while orders on Amazon.com's book and music store were up 400% over last year.

The implication for IT departments is clear. The demands on Web sites are going to keep escalating, which for many organizations will entail a steep learning curve. "We all have limited experience with high-performance, highly integrated Web sites," says George Favaloro, Compaq's director of marketing for Internet solutions.

AMR Corp.'s Travelocity has learned that one key to success is capacity planning. "You need to dedicate a good amount of resources to do nothing but look at your growth, your capacity needs, and the growth of the industry overall," says Chuck Geiger, Travelocity's VP of product development.

Travelocity served 50 million page views in June, up 500% from last year. In one record week this year, the site booked $7 million in travel-related sales. Travelocity was ready because it learned from past mistakes. "You get burned a couple times from spikes that you didn't anticipate," says Geiger.

System and software vendors are increasingly marketing products specifically for heavy-duty Web traffic. And they're not just selling Unix and Windows NT servers. A recent IBM ad touts the System/390 mainframe for use as a Web server. "These organizations can't have a weak link," says Brian Walton, director of E-business with IBM's S/390 division.

Executives overseeing major Web sites are tuned in to the IT issues. Jeff Killeen, president of barnesandnoble.com, a subsidiary of Barnes & Noble, says tapping into the parent company's existing distribution systems has been a huge advantage for the Web start-up. "Logistics, supply-chain management, and all the aspects of back-end execution are just critical success factors for an E-commerce company with any kind of grand aspirations," Killeen says.

A Business Surprise
System upgrades are a fact of life for many Web site managers. Apparel retailer J. Crew Inc. has been reinforcing its site to handle growing traffic since it opened its store on the Web 17 months ago. "The surprise was how much business was waiting for us," says Marc Hansen, VP of systems architecture at J. Crew.

When launched, J. Crew's site ran on two Sun Microsystems SparcServer 1000 servers. The machines were overloaded in eight days. The company upgraded to a pair of Sun SparcCenter 2000s, only to swap in two Enterprise 3000s three months later. J. Crew upgraded CPUs and memory in the 3000s until they were filled to capacity. In October, it traded up to four Sun Enterprise 5500s, adding two boxes for redundancy.

The hardware upgrades were necessary because of a steady rise in traffic, a trend Hansen expects to continue. "When I max out the 5500s, I'll put in 6500s. When I max those out, I'll put in Sun's Starfires," he says. The J. Crew site serves 30,000 page views per hour during peak periods.

J. Crew's work is made easier because system upgrades generally involve unplugging disks from one set of Sun servers and plugging them into more powerful servers without changing software. This ability to upgrade by swapping disks is popular with Web businesses because traffic jumps with little warning. On the Web, "the whole notion that demand levels are predictable goes out the door," says Anil Gadre, VP of marketing at Sun. "Handling unpredictability becomes of paramount importance."

J. Crew's servers run Netscape's Enterprise Web software and Sybase's database-management system, which are linked by custom-developed Common Gateway Interface scripts. When the Netscape server gets a request for a page, it invokes a CGI script, which requests data from the Sybase database. Another script plugs the retrieved data into a Web page, and still another script stamps the page with a unique session ID, which lets the site keep track of items in a user's shopping cart. Finally, the Netscape server dishes out the finished page to the user's browser.