Big data is a big part of many enterprise IT operations today. According to an IDC forecast, it will be $187 billion big by 2019. It's a critical part of the analysis that forms the basis of both machine and human business intelligence and decision-making. Since it's obvious that you can't have any sort of data -- big, little, or exactly right -- without some sort of infrastructure, it's worth taking a look at some of the factors that go into building a successful big data architecture.
I decided to take a look at seven factors that can make a big difference in the effectiveness of your big data infrastructure. Some might seem obvious, while others are a bit more subtle. In practice, all will work together to have a huge impact on the analysis and action your big data systems will support.
[See 9 Hot Big Data and Analytics Startups to Watch.]
It's not that these seven factors are the only things that have an impact on the way your big data infrastructure will work. Making big data work for an enterprise is complex.
There are scores, if not hundreds, of bits and pieces that go into a big data system -- any one of which can end up having a large impact on the work data scientists can do. But these seven deserve your consideration because they underlie so many other pieces and processes.
At this point, it's likely that you're involved with big data, even if you work in a small company. That's part of the power of the infrastructure pieces now available -- many of them are accessible to even the smallest IT operations.
With that accessibility comes the possibility of confusion and frustration for those smaller staffs that might not have data science expertise on board. If you're in that position, this list won't relieve all your confusion, but it might provide a place to start asking some pointed questions of potential service providers and suppliers.
If you're involved in a big data project, I'd love to hear from you regarding the infrastructure choices you've made. What do you think of this list? Is there something you'd swap in, or should the entire list be tossed out and started again from scratch? I'll be hanging out in the comments to see what you have to say.