The InformationWeek -- Blogs
Digital Life

Topics:   Digital Life

  • Email this page E-mail this page
  • Print this page Print this page
  • Bookmark and Share
  • icon

Are Bots On Twitter Hurting Overall System Performance?


Posted by Allen Stern, Oct 29, 2008 06:05 PM

If you use the "real-time short messaging service" Twitter, you know that over the last year there have been many performance issues. So many, in fact, that users have created a name for the outages, called the "fail whale." I started to poke around and began to wonder if bots running on the Twitter network could be causing some of the system performance issues on the service.


A bot basically performs a function or functions. In the case of Twitter, most of the bots either post links to content from across the Web or provide a reply service for certain phases.

To get an idea of just how many bots there are on the Twitter service, over the past couple of weeks I analyzed blog posts from more than 10 popular tech blogs. While this won't account for all of the reply service bots, it did provide good insight into the massive number of link-posting bots. For the purposes of this post, the bots listed below come from one post on the tech blog TechCrunch. Here's a list of the bots that reposted a link to the post on TechCrunch along with the number of subscribers (i.e., followers) to each one:

  • top_post - 104
  • tc2tw - 25
  • techupdates - 171
  • RSS_TechCrunch - 350
  • techupdate - 31
  • TechFeed - 207
  • twittfeed - 116
  • RossRSS - 70
  • SocialMedia411 - 51
  • TechGlance - 449
  • top_blog - 112
  • nuws - 132
  • readmashcrunch - 96
  • wiredgnome - 43
  • techcrunchtweet - 47
  • balduaashish - 28
  • tinycomb - 31
  • readburner - 348
  • normman - 89
  • crazyengineers - 25
  • TwitLinksRSS - 392
  • seomasterlist - 207
  • simplynews - 56
  • tweetmeme - 643
  • memebot - 48
  • foobot - 42
  • readburnerRSS - 112
  • techgurus - 225
  • linksgoogle - 300
The bots listed above only represent one post from one tech blog. If you start to multiply this out across blogs and posts, you can see where this might present some level of performance issues for Twitter. Are these bots actually needed? How many are actually being used anymore? With so few subscribers on the bots listed above, why not offer one feed bot and reduce the server load overall?

What's really interesting is that in the case of TechCrunch, its Twitter account has the full feed in the account, so none of these bots are even necessary for TechCrunch, although some of the bots listed also scrape feeds from other tech sites as well.

As Twitter works on implementing a revenue model, I can only imagine that bot traffic could be the first hit.

« Imation Finds Scary Data On "Recertified" Tapes | Main | IBM's Enterprise Content Management Push »



Sign Up Now
For InformationWeek News Alerts




This is a public forum. United Business Media and its affiliates are not responsible for and do not control what is posted herein. United Business Media makes no warranties or guarantees concerning any advice dispensed by its staff members or readers.

Community standards in this comment area do not permit hate language, excessive profanity, or other patently offensive language. Please be aware that all information posted to this comment area becomes the property of United Business Media LLC and may be edited and republished in print or electronic format as outlined in United Business Media's Terms of Service.

Important Note: This comment area is NOT intended for commercial messages or solicitations of business.




 
Digital Life Video

 

  1. Sequential Programming: Like Eating Peas with a Straw.
  2. Biomolecular device using self-assembled DNA nanostructures?
  3. Coreinfo v2.0: A Simple Utility to Understand the Manycore Complexity in Windows


Join The InformationWeek Group On LinkedIn


                           


  1. Too Much Netbook For Too Litl?
  2. Sprint And T-Mobile Headed The Wrong Direction
  3. More Reasons Why Linux Misses The Desktop
  4. Windows 7 Is Broken, So What?


  1. Florida Hospital Dials Up iPhones For Nurses
  2. Is Antivirus Software Dead?
  3. Securing The Cyber Supply Chain
  4. CIO Profiles: Christopher Rence, Chief Information And Business Transformation Officer Of FICO
  5. InformationWeek Analytics Research: Federated Search
  6. Practical Analysis: The Fastest-Growing Security Threat

 

  Ars Technica
Boing Boing
Channel 9 Forums
CRN Blogs
Dr.Dobb's Portal: Blogs
Engadget
Gizmodo
GrokLaw
  Lifehacker
Schneier on Security
Slashdot
TechCrunch
Techdirt
Techmeme
Valleywag

  DECEMBER 2008
NOVEMBER 2008
OCTOBER 2008
SEPTEMBER 2008
AUGUST 2008
JULY 2008
JUNE 2008
MAY 2008
  APRIL 2008
MARCH 2008
FEBRUARY 2008
JANUARY 2008
DECEMBER 2007
NOVEMBER 2007
OCTOBER 2007
SEPTEMBER 2007