Big Data // Big Data Analytics
News
3/15/2013
03:19 PM
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Facebook On Big Data Analytics: An Insider's View

Facebook's Jay Parikh talks about fixing Hive, real-time platforms and how traditional companies can 'thread the needle' of big data success.

13 Big Data Vendors To Watch In 2013
13 Big Data Vendors To Watch In 2013
(click image for larger view and for slideshow)
IW: Are there graph-analysis possibilities for ordinary companies and is the technology very mature?

Parikh: Graphs are not new, but there are definitely more technologies available now in terms of commercial and open-source graph databases. It's yet another cool piece of technology that lets you derive insight, but it's not going to supplant enterprise applications like fraud-detection or e-commerce that already highly optimized on relational databases.

The ecosystem around graph technology is very under-developed, and I don't think it will ever become as developed as the relational world because it's not general purpose. Graphs will develop, but it's going to be just yet another piece of technology that lets companies carve off and optimize a few key applications.

IW: Do you have any advice for enterprise IT shops venturing into big data?

Parikh: You're going to have the big-data Hadoop-Hive world, and then you're going to have some specialized real-time systems and you're going to have some specialized graph processing engines. Most IT shops, if they're good and they have a lot of applications to deal with, are going to end up in this world.

Everybody is dealing with scale today, and it's getting to be a more difficult challenge in terms of the amount of data that people want to collect and analyze. Sometimes companies are collecting data and they don't know what to do with it yet, or they're collecting data that they don't even know they have. The fundamental problems are how do you store it, how do you process it and how do you derive useful insights? If you aren't careful as you build out big data applications, you stand to waste a lot of money or you stand to miss huge opportunities in your business. Threading that needle is what every tech company in the world has to do, and most companies won't be able to do it well.

IW: Why not?

Parikh: It's very hard to manage the balance between storing too much and then trying to find something valuable or partitioning your data among different business units and not being able to get insight across the business. We're in an early phase of this technology. It's not something that's insurmountable and people are figuring it out. But storing the data, determining what you do with it, writing the applications and responding to the insight from the data is the balancing act that every tech organization is going to work on.

IW: The "wasting a lot of money" danger is pretty clear -- too much data, too little value. Any advice on how not to miss the opportunity?

Parikh: It's crucial to understand the data that you're collecting and to react to it to change your business. If you're just focused on the tip of the data, you may be missing a longer-term trend. You might be fixated on just a couple of bits of data and not looking at other bits that might be significant. You need a micro, laser focus on impact, but you also need to have a broad perspective on where you're going with all the data.

You may be focused on decisions with real-time data, but are you missing a longer-term impact on your business if you're not looking at your entire data set? It takes a lot of iteration and experimentation to succeed. It's an exciting time and there are lots of cool things for enterprises to try, but it's hard work and the technologies are still maturing.

The Enterprise Connect conference program covers the full range of platforms, services and applications that comprise modern communications and collaboration systems. Hear case studies from senior enterprise executives, as well as from the leaders of major industry players like Cisco, Microsoft, Avaya, Google and more. Register for Enterprise Connect 2013 today with code IWKPREM to save $200 off a conference pass or get a free Expo Pass. It happens March 18-21 in Orlando, Fla.

Previous
3 of 3
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
TomMcGrath
50%
50%
TomMcGrath,
User Rank: Apprentice
5/6/2013 | 12:55:42 PM
re: Facebook On Big Data Analytics: An Insider's View
I also thought he has developed a huge team in order to maintain big data at larger extent.
http://www.bigdatacompanies.co...
PThakuria
50%
50%
PThakuria,
User Rank: Apprentice
3/26/2013 | 4:29:29 PM
re: Facebook On Big Data Analytics: An Insider's View
Great article, Doug. Glad you brought the viewpoints of the true pioneers, adopters and practitioner's viewpoints for the benefit of the mainstream enterprise. It was very interesting to read how they push front end code twice a day for analysis and Scuba. Any reason why they did not go with established in-memory databases - a technology which is pretty matured when they adopted MySQL for other purposes?
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
3/18/2013 | 4:55:29 PM
re: Facebook On Big Data Analytics: An Insider's View
Parikh is pretty up front about the limitations of Hive that Facebook is tying to overcome, but he makes it clear it will take a yet-to-be-announced new platform -- expected this summer -- to address real-time analysis needs. Given the many real-time initiatives now underway in the Hadoop community, it will be interesting to see whether Facebook's new platform is embraced the way Hive was embraced way back when.
Laurianne
50%
50%
Laurianne,
User Rank: Author
3/18/2013 | 2:25:03 PM
re: Facebook On Big Data Analytics: An Insider's View
Sounds like he has developed a team with a large amount of Hadoop expertise. I wonder if they are hiring up a storm from outside, or grooming people who were already there.

Laurianne McLaughlin
InformationWeek
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 20, 2014
CIOs need people who know the ins and outs of cloud software stacks and security, and, most of all, can break through cultural resistance.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.