Cloud
Commentary
12/4/2009
09:39 PM
Charles Babcock
Charles Babcock
Commentary
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Ready To Launch? Lack Of Scalability A Killer

Yahoo Pipes is a way to mash up RSS feeds and Web pages into combined information, apply rules and filters, and publish it. When it launched Feb. 7, 2007, the Pipes API worked fine in principle. But as "Daniel" said from the audience at a San Francisco panel this week, Pipes wasn't ready to scale.

Yahoo Pipes is a way to mash up RSS feeds and Web pages into combined information, apply rules and filters, and publish it. When it launched Feb. 7, 2007, the Pipes API worked fine in principle. But as "Daniel" said from the audience at a San Francisco panel this week, Pipes wasn't ready to scale.The gathering was a panel at the GigaOm network offices in San Francisco, an online publisher that produces blogs on several key niches, including Apple and cloud computing. The question before the panel was: what should a startup do, build out its own data center or contract with an infrastructure service provider?

The somewhat predictable answer from a panel of service providers was that startups with intentions of growing fast should find a service provider to their liking and let the service handle the infrastructure issues. With a service, whether an application building and deployment site, such as EngineYard, or a server infrastructure provider, such as Joyent, the service provider will handle the issues of fast growth better and spread the expense across multiple users.

Jason Hoffman, CTO of Joyent, cited the possibility of the TechCrunch or Slashdot effect on a new company, where its notice and commended in a popular opinion broker site, then crashes as traffic swarms to the newcomer. You can avoid missing out on all those prospective customers, he advised. "If you think you are good at building infrastructure, then go ahead, build it yourself. But I think I'm pretty good at building infrastructure," he said.

Tom Morini, CTO of EngineYard, the Ruby on Rails deployment platform, declined to go along with the consensus, however. "I'd like to dispute the myth that the TechCrunch (a popular blogging site) or SlashDot effect is a great surprise to startups. In many cases, they can predict when they'll see a surge in traffic and are well-prepared for it.

At that point, Daniel, who was probably Daniel Raffel, a member of the original Yahoo Pipes team, interrupted the proceedings from the back of the crowd. "We thought we'd experience a gradual ramp up over the course of a year. We were just a small team," he said of the developers who produced the popular mash-up service. With the February 2007 launch of the new service, their Pipes site was overwhelmed and couldn't come close to satisfying the download demand, he recounted.

"I kept running into people over the next year, who, when they heard I worked on Pipes, would say, 'Yeah, we went there, but it was down." The moral of his story: some people are going to be caught before they're ready to address the scalability issue, if they do their own infrastructure.

Yahoo has since announced it is building a private cloud for more elastic, internal operations and external services.

Comment  | 
Print  | 
More Insights
2014 Next-Gen WAN Survey
2014 Next-Gen WAN Survey
While 68% say demand for WAN bandwidth will increase, just 15% are in the process of bringing new services or more capacity online now. For 26%, cost is the problem. Enter vendors from Aryaka to Cisco to Pertino, all looking to use cloud to transform how IT delivers wide-area connectivity.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - July 22, 2014
Sophisticated attacks demand real-time risk management and continuous monitoring. Here's how federal agencies are meeting that challenge.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
A UBM Tech Radio episode on the changing economics of Flash storage used in data tiering -- sponsored by Dell.
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.