5 min read

Soccer Wonks Learn Tough Big Data Lesson

Recent gameday experiment shows that in sports and in business, even the most detailed big data analysis is worthless if the questions don't make sense.
Correlation Does Not Equal Causation

Rival Chelsea has collected more than 32 million data points over the course of 12,000 to 13,000 games, according to Mike Forde, the team's performance director, in the same Financial Times article.

It took time for team managers and their statistics gurus to understand which of those data points to pay attention to. The percentage of completed passes compared to the number of interceptions is a good indication of consistent victory, as is an error rate lower than 18%.

Traditional metrics such as the number of tackles, on the other hand, can lead teams to ignore potentially invaluable players, however, according to Soccernomics, the European-football version of Moneyball.

For example, retired team Milan star Paolo Maldini, who Aslett describes as "arguably one of the greatest defenders the world has ever seen," rarely made more than one tackle every other game. Viewed only by that statistic, he would probably be out of a job.

Judged according to details of his positioning, movement, and comparisons with players whose advance he was trying to prevent, it became obvious "he positioned himself so well he didn't need to tackle," according to Soccernomics.

That particular error was due to the tendency to measure the performance of individuals even while they engage in a team activity--corporate project teams, for example, as well as soccer.

Analyzing the results of such out-of-context metrics shows that judging players as stars according to their individual accomplishments could very well hurt the performance of the whole team by ignoring those whose team play makes the whole group successful, according to a study published in 2010 by Jordi Duch, Joshua S. Waitzman, and Luis A. Nunes Amaral, researchers at Northwestern University.

"Whereas there are contexts in which simple measures or statistics may provide a very complete picture of an individual's performance--think of golf, baseball, or a track event--for most situations of interest, objectively quantifying individual performances or individual contributions to team performance is far from trivial," the authors wrote.

"In the context of a soccer, where quantification has always been challenging, we are able to demonstrate that flow centrality provides a powerful objective quantification of individual and team performance," they concluded.

In other words: examining, highlighting, and rewarding players (or employees) according to their individual accomplishments can be anything from ineffective to counterproductive in situations in which the goal can only be accomplished with contributions from many players.

While the Northwestern researchers could not extrapolate their results into specific guidelines on how corporate managers could encourage more effective team play by their own employees, they did suggest the result should also apply to situations other than soccer.

The value of teamwork and team building is widely accepted within business-management circles, though it is unclear how many incentive systems genuinely reward teamwork at the expense of individual accomplishment.

Studies describing the value of teamwork and techniques to encourage it are mostly anecdotal and advisory, rather than quantitative, so it's difficult to extrapolate from them. However, it's easy to extrapolate how big data analytics can be misunderstood or misused by people trying to extract lessons from it.

Garbage In, Garbage Out

Using big data analytics to identify points of inefficiency, gaps in automation, and other elements in a specific business process may also affect an entire business, according to Bill Franks, chief analytics officer of global alliance programs at Teradata.

Corporate processes tend to depend on one another, not exist independently. So improving one process using data on only that process is just as likely to cause glitches in related activities and gum up the whole works, Franks wrote, as it is to improve things overall.

Even doing all the analytics correctly and improving systems in a coordinated way won't do much to improve a company that didn't pay as much attention to the quality of data going into a big data system as it does to the answers coming out, according to Steve Sarsfield of big data analytics vendor Talend.

Even something as simple as having one call center operator who occasionally misspells Main St. as "Mian St." will reduce the quality of the data overall and make it harder to depend on the kind of personalized, location-dependent recommendations a big data system might deliver for a single customer who seems to live part of the time on Main St. and part of the time in a location the map-search engine can't identify.

Even with top-quality data and analytics, making the results available to a marketing staff (for example) with too little training in statistics can cause those too enthusiastic about the results to adjust tactics every time they see a change on their dashboards "and end up changing direction so often that they lose sight of their goals," according to an article titled "Marketers Flunk the Big Data Test" in the Harvard Business Review.

The lesson seems to be that big data, no matter how great the potential or how ambitious the goal, can't point a team or a corporate staff in the right direction if there are flaws in the quality of the data, the questions, or how the answers are eventually used.

Without those three, no matter how big the data or how sophisticated the analytics, it may be more accurate to just guess the answer, rather than extracting the wrong one using advanced techniques and expensive technology no one quite understands how to use.

Attend Online Marketing Summit 2012 and gather the insights and strategies you need to make the right online marketing choices to deliver the most value for your business. The summit--in Santa Clara, Calif., Oct 22-25--offers four days of inspiration, connections, and practical learning. Attend Online Marketing Summit 2012 using code QJBQSA01 and receive a 25% discount on conference passes or a Free Expo Pass.