Yahoo Pipes is a way to mash up RSS feeds and Web pages into combined information, apply rules and filters, and publish it. When it launched Feb. 7, 2007, the Pipes API worked fine in principle. But as "Daniel" said from the audience at a San Francisco panel this week, Pipes wasn't ready to scale.
Yahoo Pipes is a way to mash up RSS feeds and Web pages into combined information, apply rules and filters, and publish it. When it launched Feb. 7, 2007, the Pipes API worked fine in principle. But as "Daniel" said from the audience at a San Francisco panel this week, Pipes wasn't ready to scale.The gathering was a panel at the GigaOm network offices in San Francisco, an online publisher that produces blogs on several key niches, including Apple and cloud computing. The question before the panel was: what should a startup do, build out its own data center or contract with an infrastructure service provider?
The somewhat predictable answer from a panel of service providers was that startups with intentions of growing fast should find a service provider to their liking and let the service handle the infrastructure issues. With a service, whether an application building and deployment site, such as EngineYard, or a server infrastructure provider, such as Joyent, the service provider will handle the issues of fast growth better and spread the expense across multiple users.
Jason Hoffman, CTO of Joyent, cited the possibility of the TechCrunch or Slashdot effect on a new company, where its notice and commended in a popular opinion broker site, then crashes as traffic swarms to the newcomer. You can avoid missing out on all those prospective customers, he advised. "If you think you are good at building infrastructure, then go ahead, build it yourself. But I think I'm pretty good at building infrastructure," he said.
Tom Morini, CTO of EngineYard, the Ruby on Rails deployment platform, declined to go along with the consensus, however. "I'd like to dispute the myth that the TechCrunch (a popular blogging site) or SlashDot effect is a great surprise to startups. In many cases, they can predict when they'll see a surge in traffic and are well-prepared for it.
At that point, Daniel, who was probably Daniel Raffel, a member of the original Yahoo Pipes team, interrupted the proceedings from the back of the crowd. "We thought we'd experience a gradual ramp up over the course of a year. We were just a small team," he said of the developers who produced the popular mash-up service. With the February 2007 launch of the new service, their Pipes site was overwhelmed and couldn't come close to satisfying the download demand, he recounted.
"I kept running into people over the next year, who, when they heard I worked on Pipes, would say, 'Yeah, we went there, but it was down." The moral of his story: some people are going to be caught before they're ready to address the scalability issue, if they do their own infrastructure.
Yahoo has since announced it is building a private cloud for more elastic, internal operations and external services.
2014 Next-Gen WAN SurveyWhile 68% say demand for WAN bandwidth will increase, just 15% are in the process of bringing new services or more capacity online now. For 26%, cost is the problem. Enter vendors from Aryaka to Cisco to Pertino, all looking to use cloud to transform how IT delivers wide-area connectivity.
Server Market SplitsvilleJust because the server market's in the doldrums doesn't mean innovation has ceased. Far from it -- server technology is enjoying the biggest renaissance since the dawn of x86 systems. But the primary driver is now service providers, not enterprises.
. We've got a management crisis right now, and we've also got an engagement crisis. Could the two be linked? Tune in for the next installment of IT Life Radio, Wednesday May 20th at 3PM ET to find out.