Infrastructure // PC & Servers
Commentary
10/29/2008
06:05 PM
Allen Stern
Allen Stern
Commentary
Connect Directly
RSS
E-Mail
50%
50%

Are Bots On Twitter Hurting Overall System Performance?

If you use the "real-time short messaging service" Twitter, you know that over the last year there have been many performance issues. So many, in fact, that users have created a name for the outages, called the "fail whale." I started to poke around and began to wonder if bots running on the Twitter network could be causing some of the system performance issues on the service.

If you use the "real-time short messaging service" Twitter, you know that over the last year there have been many performance issues. So many, in fact, that users have created a name for the outages, called the "fail whale." I started to poke around and began to wonder if bots running on the Twitter network could be causing some of the system performance issues on the service.A bot basically performs a function or functions. In the case of Twitter, most of the bots either post links to content from across the Web or provide a reply service for certain phases.

To get an idea of just how many bots there are on the Twitter service, over the past couple of weeks I analyzed blog posts from more than 10 popular tech blogs. While this won't account for all of the reply service bots, it did provide good insight into the massive number of link-posting bots. For the purposes of this post, the bots listed below come from one post on the tech blog TechCrunch. Here's a list of the bots that reposted a link to the post on TechCrunch along with the number of subscribers (i.e., followers) to each one:

  • top_post - 104
  • tc2tw - 25
  • techupdates - 171
  • RSS_TechCrunch - 350
  • techupdate - 31
  • TechFeed - 207
  • twittfeed - 116
  • RossRSS - 70
  • SocialMedia411 - 51
  • TechGlance - 449
  • top_blog - 112
  • nuws - 132
  • readmashcrunch - 96
  • wiredgnome - 43
  • techcrunchtweet - 47
  • balduaashish - 28
  • tinycomb - 31
  • readburner - 348
  • normman - 89
  • crazyengineers - 25
  • TwitLinksRSS - 392
  • seomasterlist - 207
  • simplynews - 56
  • tweetmeme - 643
  • memebot - 48
  • foobot - 42
  • readburnerRSS - 112
  • techgurus - 225
  • linksgoogle - 300
The bots listed above only represent one post from one tech blog. If you start to multiply this out across blogs and posts, you can see where this might present some level of performance issues for Twitter. Are these bots actually needed? How many are actually being used anymore? With so few subscribers on the bots listed above, why not offer one feed bot and reduce the server load overall?

What's really interesting is that in the case of TechCrunch, its Twitter account has the full feed in the account, so none of these bots are even necessary for TechCrunch, although some of the bots listed also scrape feeds from other tech sites as well.

As Twitter works on implementing a revenue model, I can only imagine that bot traffic could be the first hit.

Comment  | 
Print  | 
More Insights
Server Market Splitsville
Server Market Splitsville
Just because the server market's in the doldrums doesn't mean innovation has ceased. Far from it -- server technology is enjoying the biggest renaissance since the dawn of x86 systems. But the primary driver is now service providers, not enterprises.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 20, 2014
CIOs need people who know the ins and outs of cloud software stacks and security, and, most of all, can break through cultural resistance.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.