Topics: Digital Life

Are Bots On Twitter Hurting Overall System Performance?

Posted by Allen Stern, Oct 29, 2008 06:05 PM

If you use the "real-time short messaging service" Twitter, you know that over the last year there have been many performance issues. So many, in fact, that users have created a name for the outages, called the "fail whale." I started to poke around and began to wonder if bots running on the Twitter network could be causing some of the system performance issues on the service.

A bot basically performs a function or functions. In the case of Twitter, most of the bots either post links to content from across the Web or provide a reply service for certain phases.

To get an idea of just how many bots there are on the Twitter service, over the past couple of weeks I analyzed blog posts from more than 10 popular tech blogs. While this won't account for all of the reply service bots, it did provide good insight into the massive number of link-posting bots. For the purposes of this post, the bots listed below come from one post on the tech blog TechCrunch. Here's a list of the bots that reposted a link to the post on TechCrunch along with the number of subscribers (i.e., followers) to each one:

top_post - 104
tc2tw - 25
techupdates - 171
RSS_TechCrunch - 350
techupdate - 31
TechFeed - 207
twittfeed - 116
RossRSS - 70
SocialMedia411 - 51
TechGlance - 449
top_blog - 112
nuws - 132
readmashcrunch - 96
wiredgnome - 43
techcrunchtweet - 47
balduaashish - 28
tinycomb - 31
readburner - 348
normman - 89
crazyengineers - 25
TwitLinksRSS - 392
seomasterlist - 207
simplynews - 56
tweetmeme - 643
memebot - 48
foobot - 42
readburnerRSS - 112
techgurus - 225
linksgoogle - 300

The bots listed above only represent one post from one tech blog. If you start to multiply this out across blogs and posts, you can see where this might present some level of performance issues for Twitter. Are these bots actually needed? How many are actually being used anymore? With so few subscribers on the bots listed above, why not offer one feed bot and reduce the server load overall?

What's really interesting is that in the case of TechCrunch, its Twitter account has the full feed in the account, so none of these bots are even necessary for TechCrunch, although some of the bots listed also scrape feeds from other tech sites as well.

As Twitter works on implementing a revenue model, I can only imagine that bot traffic could be the first hit.

« Imation Finds Scary Data On "Recertified" Tapes | Main | IBM's Enterprise Content Management Push »

This is a public forum. United Business Media and its affiliates are not responsible for and do not control what is posted herein. United Business Media makes no warranties or guarantees concerning any advice dispensed by its staff members or readers.

Community standards in this comment area do not permit hate language, excessive profanity, or other patently offensive language. Please be aware that all information posted to this comment area becomes the property of United Business Media LLC and may be edited and republished in print or electronic format as outlined in United Business Media's Terms of Service.

Important Note: This comment area is NOT intended for commercial messages or solicitations of business.