Big Data Will Stymie Local Government

State and local agencies have only a fraction of the computing resources and personnel needed to handle expected data volumes, survey finds.
New York's 32-Story Data 'Fortress'
New York's 32-Story Data 'Fortress'
(click image for slideshow)
Despite the potential of big data analysis to improve governmental decision making, state and local IT pros have less than half of the computing resources and a third of the personnel they need to capitalize on the expected surge in data over the next four years.

The resulting gap and limited overall awareness about big data are among the reasons relatively few state and local agencies are taking steps to harness and analyze big data sets, according to a study by MeriTalk, a government IT networking group. The report, sponsored by NetApp, is based on a survey of 150 state and local CIOs and IT managers.

The survey found that state and local IT pros are struggling to keep up with data demands. Just 59% of state and local agencies are analyzing the data they collect and less than half are using it to make strategic decisions. State and local agencies estimate that they have just 46% of the data storage and access, 42% of the computing power, and 35% of the personnel they need to successfully leverage big data.

In addition, 57% say their enterprise architecture is not able to support big data initiatives. One in three state and local agencies has a dataset that has grown too large to work with, given their capacity limitations, the study found.

[ Can technology make your city a better place to live? Read IT Strategies For Future Cities. ]

Complicating the data management picture is confusion over who owns data: 47% of survey respondents believed that IT departments own the data, while 31% said ownership belongs to the department that generated it.

It's not that officials don't see the potential for analyzing big datasets. Survey respondents cited the ability to improve overall agency efficiency (57%), increase the speed and accuracy of decisions (54%), and achieve a greater understanding of citizen needs and how to meet them (37%) as the top advantages of tapping into big data.

But the advantages aren't universally understood or appreciated. The study found that 44% of state and local agencies aren't discussing big data; and 39% are just beginning to learn about this technology.

And while 94% of respondents said their agency can benefit from big data, 79% estimate it will be at least three years before they are able to take full advantage of it.

In a report issued last year, the National Association of State Chief Information Officers advised its members to develop an enterprise architecture and data governance policies to maximize the potential of big data in their agencies.

Urban transformation requires IT innovation. Here's how five U.S. cities are forging ahead. Also in the new, all-digital Future Cities issue of InformationWeek Government: Video surveillance provided valuable clues to the Boston Marathon bombings, serving as a lesson to other cities. (Free registration required.)