A few weeks ago I was watching an April 2007 episode of Bill Moyers Journal called "Buying the War. Moyers' program served as a critical look at the media's handling of the run-up to the Iraq war and examines how the media got the story "so wrong". So how does this relate to mobility?While Moyers' analysis and interviews made for great TV, it was a quote from Walter Isaacson, former chairman and CEO of CNN, that really sparked my interest. Isaacson said, "One of the great pressures we're facing in journalism now is, it's a lot cheaper to hire thumb suckers and pundits and have talk shows on the air than actually have bureaus and reporters. And in the age of the Internet when everybody's a pundit, we're still gonna need somebody there to go talk to the colonels, to be on the ground in Baghdad and stuff, and that's very expensive."

"Is technology journalism becoming like the mainstream media?" I wondered. In some ways, I believe that technology journalism is indeed mirroring the trends posed in Isaacson's comments. A February 2008 article in eWeek provides a fine example of the rise of techno-punditry. In an article entitled "Duke Is Investing in a Half-Baked 802.11n Standard," Steven J. Vaughan-Nichols provides a lengthy commentary on how Duke is misguided in its decision to deploy Cisco's new 1250 draft 2.0 802.11n access points. Vaughan-Nichols comments that draft 2.0 isn't truly a standard, that there is no promise for interoperability and that the only way laptops from various vendors will truly interoperate is for them to fall back on the older, ratified, 802.11g standard.

What Vaughan-Nichols fails to mention is any role of the Wi-Fi Alliance, which has provided a draft 2.0 certification for 802.11n. Through the Wi-Fi Alliance, there is promise that Wi-Fi-certified products will interoperate with each other. There also is precedence for the Wi-Fi Alliance's actions; when the industry needed quality of service features in order to support applications like voice, the Wi-Fi Alliance took a draft of 802.11e and created the WMM specification. When the industry needed to address the security concerns of WEP, the Wi-Fi Alliance published the WPA specification before the ratification of 802.11i.

The problem is that, despite the flaws in Vaughan-Nichols' commentary (including the assertion that IT buyers would be better off buying the SoHo grade Linksys 802.11g access points, which I find laughable for any serious deployment like Duke's), other bloggers (including Network Instruments' marketing blog) latched onto the eWeek commentary as "proof" that the standard isn't ready. Don't get me wrong, those who decide to install 802.11n today will likely face some pain with their deployments as bugs are worked out. InformationWeek's own testing of Cisco's 1250 access point wasn't without its own troubles. But even with those kinks, our lab experiences aren't nearly as "Chicken Little" as Vaughan-Nichols makes the state of the industry out to be.

Testing by media outlets gets the story wrong sometimes, too. For instance, a recent article from CRN pointed out that Cisco's 1250 access point "did not come close to Cisco's claims of its top-level performance of 300-Mbps throughput". Then again, Ruckus and Meru also didn't come close to the magical 300-Mbps number, either.

One problem with CRN's analysis is that it's misunderstanding the difference between data rate and throughput. 802.11 always has been an inherently chatty protocol and, despite protocol improvements, conventional wisdom says that you'll get roughly half (give or take) of the stated data rate as your actual throughput due to protocol overhead. For 802.11a and 802.11g, the data rate is 54 Mbps and our lab tests generally peg 802.11a and 802.11g's throughput at somewhere between 22 and 25 Mbps. With 802.11n, the 300 Mbps data rate number comes from 40-MHz channels (double the channel size of 802.11a and 802.11g).

Thus, users can expect throughput using 40-MHz channels to come in at about 150 Mbps and 75 Mbps using 20-MHz channels. CRN's own testing methodology stated that it tested only in 2.4-GHz channels and used only 20-MHz channel bandwidths. I'm not sure why it believed that it could wring out the best performance from an 802.11n deployment with this configuration. As a point of comparison, InformationWeek's testing of the Apple Airport Extreme in October 2007 clocked in results of 112 to 137 Mbps using 40-MHz channels. Average throughput for the Cisco 1250, based on our March 2008 testing, came in at 154.9 Mbps using 40-MHz channels in 5 GHz, right around the "1/2 data rate" number conventional wisdom expects.

At the end of the day, there was some redeeming value in what CRN did. Leaving its less-than-stellar analysis aside, it encountered a few issues with 802.11n that users will likely face. One issue of contention with its testing methodology, based on my reading and the reader comments, is that the reviewers refused to upgrade to the latest 802.11n client drivers. The authors believed this scenario represented more "real world" conditions. I don't necessarily mind the idea of testing with off-the-shelf drivers, but the fact of the matter is that drivers are being updated all the time to address performance issues, particularly with a new technology like 802.11n. We've seen this in our own lab experiences. While it's "real world" to have stock drivers in your network, call any tech support line (I say this having worked in technical support) and the first response you'll get is to update your drivers if you complain about less-than-stellar performance.

CRN's article also showed the value in Wi-Fi certified equipment. When reviewers removed a Linksys WPC4400N Wireless N client card which lacked Wi-Fi certification from the test matrix, magically, all of the performance numbers went up. My take is that IT administrators need to be careful to use equipment that's Wi-Fi certified -- it's likely to yield the best performance for your setup. Of course I'm drawing these conclusions on my own; they went unstated in CRN's analysis.

As Isaacson points out, as a society there is value in having reporters overseas and talking to sources. Likewise, there is value in our industry in conducting our form of "boots on the ground" reporting through true independent testing. The problem is that both scenarios are very expensive. As the mainstream media have emphasized punditry over investigative reporting, the technology media have followed suit. We have moved away from critical testing-based analysis and toward providing "commentary." As a response to this shift, I've begun to see more and more whitepapers released containing the type of comparative testing, conducted by vendors or by "independent" testing houses, which you used to see in print magazines. And in the drive to provide more commentary, journalists and bloggers alike pick up on these whitepapers and report the results as if they were news.

At the end of the day, our goal as an industry is to educate consumers. IT vendors publish whitepapers to educate consumers in order to sell more products. Journalists write articles in order to "report the news," but my own view is that this is also a form of consumer education. The problem is when whitepapers rely too much on smoke and mirror tactics or when journalists' commentary isn't based in sound understanding of the technology at hand. This only serves to confuse consumers. I can only hope that things continue to get better within our industry, particularly around the area of wireless which, despite four revisions of the physical radio standard, is clearly still misunderstood. The last thing we need is the IT equivalent of CNN's Crossfire.