Clay Shirky Busts The Myth Of Information Overload At Web 2.0 Expo

It's hard to write a news article about a Clay Shirky presentation. The conclusion of an article is supposed to be at the top, in the headline and lead sentence, which is supposed to pack such a wallop that the reader feels compelled to read. But the conclusion of a Shirky presentation isn't the best part. With Shirky, the entire talk is a unified whole, and the best part is listening to Shirky walk through his thought process.Another reason reporting on Shirky is hard: He makes every sentence and every word count. Other speakers have lots of marketing fluff, irrelevant asides, and throat-clearing, which gives a poor journalist time to rest his pen and flex his fingers. But with Shirky, it's scribble, scribble scribble, and my hand aches by the time he's done speaking. And I still don't get every word, because once or twice in any Shirky talk, something he says brings me up short and I have to pause to actually think about what he's saying, rather than simply recording it.

Shirky is an adjunct professor at NYU who studies social media, and author of the book Here Comes Everybody: The Power of Organizing Without Organizations, which looks at how ad hoc groups that form using social media like Facebook and Twitter are challenging traditional institutions like Big Business and government.

During his keynote Thursday at Web 2.0 Expo in New York, Shirky examined the problem of information overload. The problem is not that we have too much information to process. The problem is that our filters are inadequate. And privacy breakdowns are a similar problem -- privacy is threatened because the filters we relied on to keep our private data confidential are broken, and we haven't evolved good mechanisms to replace those filters yet.

Shirky started by displaying a chart of the growth of information production worldwide. It's an exponential curve, with hockey stick growth. The chart he used came from IDC; many other companies have produced similar charts, and they always look the same, Shirky said.

People love those charts. They tell themselves, "This is why I'm not getting anything done -- I'm suffering from information overload," Shirky said.

Likewise, journalists for the past 15 years have constantly been rewriting the same article about information overload, Shirky said.

So why does information overload still come as a surprise, he asked.

The problem is much older than the Internet age. It started with Gutenberg's invention of the printing press in the 15th century, Shirky said. After that, for the first time, there were more books in the world than one person could read.

And the printing press introduced a new kind of business, publishing, with new risks. Printing presses were expensive, and, to keep from going out of business, publishers had to print books the public would want to buy. Publishers had to become information filters. Publishing combined the physical manufacturing of books with decisions on what information to filter for the public, Shirky said.

In retrospect, it wasn't intuitively obvious that those two functions should go together, but that's the way it's been done since the 15th century, and, when TV and radio came along, they followed the same model -- the people who owned the transmission facilities made the filtering decisions, Shirky said.

The Internet broke that model -- it made publishing cheap, and filtering broke down. "Proof of this concept, I'll leave to you, but I recommend you start with LiveJournal," Shirky quipped.

(Shirky has discussed this transition before, where he noted that bloggers and other online communities have stepped in to fulfill the filtering component of the publishing cycle. Bloggers sift through the mass of Internet content and point their audiences at the content worth checking out. The model used to be "filter, then publish," now it's "publish, then filter.")

Shirky next turned his attention to spam. Junk e-mail occasionally gets overwhelming -- a form of information overload -- but that's not really a function of volume. Relatively small changes in the volume of spam can make us feel like spam is exploding. We're overwhelmed by spam when our filters break down.

Spam is a typical example of information overload. And spam filters are a good model for all kinds of information filters; we require both automated and manual filters to cut through spam. Automated filters quarantine and tag spam, and then we have to examine our e-mail for errors. There's no such ting as "set and forget" spam filters, Shirky said.

Shirky then turned his attention to Facebook. A friend of his broke off her engagement recently, and, in addition to all the other hassles and heartbreaks that go along with that, she had to do a 21st century ritual -- changing her relationship status on Facebook from "Engaged" to "Single."

And yet she wanted to control how the information got out to her network of friends, which included real friends; pseudo-friends, like former co-workers and people she went to high school with; and friends of her now-ex-fiance, who were now her friends, too, but who really should be told about the breakup by her fiance.

So she investigated Facebook's privacy policies and carefully set the privacy controls, and then changed her status -- and watched the information go out to all her Facebook friends, which was not what she wanted. Shirky said, "E-mails started coming in, the phone started ringing, IMs came in -- it was out of control."

Who should be blamed for the error, Shirky asked rhetorically. Was his friend stupid? No, in fact she did everything she should have done to protect her privacy -- and she's an expert on social networks, she did academic studies comparing Facebook, Friendster, and Meetup.com.

Blame Facebook? No, Facebook actually has a very advanced privacy protections, Shirky said,.

The problem is that we only recently started having to manage privacy, Shirky said. "The problem is we're moving from an evolved system to an engineered system," he said. Managing privacy is an unnatural act. His friend wanted to let the news of her broken engagement get out the traditional way -- tell four or five friends and they'd tell some people and word would trickle out slowly that way. Instead, Facebook sent out the information in a burst.

People used to have a thing called a "personal life," which was private, Shirky said. You could walk down the street and have a private conversation without fear of it being overheard, let alone recorded and disseminated worldwide. "The inefficiency of information flow was not a bug, it was a feature," Shirky said. His friend is an extreme example, as an aggressive social media user, but it's a problem that we all face.

Privacy problems occur when outbound information filters break, Shirky said. Information overload occurs when inbound filters break.

Another Facebook anecdote: When Chris Avenir started classes at Ryerson College, he did what a lot of college students do: He formed a study group. He formed the group on Facebook, called it the Dungeon, which is the name of the room where study groups meet, and it accumulated 146 members.

Ryerson College said the Facebook group was a form of cheating, since students would be sharing information. They charged Avenir with 147 violations of the school academic code, one for forming the group, and 146 for each member.

Avenir said the charges were nonsense. If the Facebook group was an example of cheating, then so were all the study groups, mentoring, and tutoring programs the college offers.

The incident was a clash of metaphors, Shirky said. Facebook isn't like anything else in the world -- it's not like a fax machine or a study group or anything but itself.

Both sides have good points, he said. Avenir was simply extending the accepted model of study groups to a new medium.

And the college -- while it overreacted -- had a good point, too. Study groups are designed for collaboration, everybody works and everybody learns. Because study groups are small, and meet face-to-face around a table, they're self-policing -- freeloaders are easily identified and kicked out. But a Facebook study group with 146 members will inevitably contain many freeloaders, people who get the answers without doing the work. And doing the work is the point -- we don't ask students to derive the formula for hydrochloric acid because we need to find the formula, Shirky said. We already know the formula. We ask students to derive the formula because it's important to their learning.

Shirky concluded: Former Israeli Prime Minister "Yitzhak Rabin, a man who should know, said if you have had the same problem for a long time, maybe it's not a problem, maybe it's a fact." The problem of information overload is one we've had a long time -- since the 1500s. The problem that we have now is that our good filters are broken. So when you think you're suffering from information overload, you should look instead for broken filters.

And that was Shirky's presentation.

What do you think? Are information overload and privacy breakdown simply a problem of broken filters? Discuss among yourselves while I ice down my aching fingers.