With so many big data projects now underway across the enterprise, the next big challenge facing IT organizations is how to effectively manage that amount of data.
Big data governance has suddenly become one of the hottest topics in IT as organizations turn their attention once again to the fundamentals of data management; except this time, it's at a scale no one previously thought possible, or, for that matter, advisable.
Establishing big data governance policies is critically important because the amount of data involved not only threatens to overwhelm IT organizations, but how that data gets used will actually determine the success of such projects. Specifically, organizations need to understand what type of data is valuable enough to keep, as opposed to data that is expendable.
Unfortunately, a recent survey of 3,500 IT leaders conducted by TEKsystems, a provider of IT services, found that 66% of IT leaders and 53% of IT professionals said their data is stored in disparate systems and that they need new platforms to accommodate these increased data management needs.
In addition, 81% of IT leaders said their organization lacks the skills on staff to plan, build and run big data initiatives. That's problematic because a separate survey of 1,000 Fortune 1000 business and IT leaders conducted by NewVantage Partners found that half the executives surveyed said they would be spending more than $10 million on a big data project by 2016.
"Companies are appointing chief data officers to drive innovation," Randy Bean, founder and managing partner for NewVantage Partners, told Enterprise Efficiency. "That means there's also multiple tiers of governance with a lot of active IT participation."