InformationWeek is more impressed by the testimonials of companies that are getting real value out of big data platforms and analysis techniques. Pfizer and Merck, for example, are developing more effective and affordable drugs thanks to big data techniques that are leading to more targeted treatments and more productive manufacturing processes. GE and others are demonstrating improvements in industrial equipment performance, uptime, and safety thanks to Internet of things-style applications.
And then there are the pioneers like The Weather Company and Facebook that say they just couldn't run their data-driven businesses without new platforms, even if they still have a place for more conventional tools like relational databases.
Here are five trends witnessed over the last year that point to progress in big data analysis:
1. SQL meets Hadoop Hadoop is here to stay, so every data management vendor worth its salt must have a SQL-on-Hadoop or SQL-access-to-Hadoop option. Here are five of our most-read stories in the SQL-meets-Hadoop vein:
Just remember that SQL is not designed to find correlations among variably structured data sets. Nor does it support machine learning, many advanced analytics techniques, or other approaches often associated with big data analysis. If SQL solved everything, we wouldn't need new platforms.
2. Platforms mature Every other week in 2014, or so it seems, Hadoop software distributors and NoSQL database vendors announced new management consoles, security systems, data management capabilities, search engines, or high-availability features. Here's a sampling of what we're talking about:
These and other big data vendors are trying to reassure enterprise IT types that these products are secure and reliable as 30-year-old database management systems. Let's just say that more than a few grizzled IT veterans are still used to working with favored and familiar tools and still need some convincing.
3. Educational options proliferate Nature abhors a vacuum, so into the void of data science and big-data platform knowledge and expertise have rushed vendors,
Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio
How Enterprises Are Attacking the IT Security EnterpriseTo learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
2017 State of IT ReportIn today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.