As I've made the case before, the social world, by dint of a billion people engaging with each other around the clock, is now the richest source of open innovation, product ideas, marketing and sales opportunities, customer care capacity, and much more. One thing we've learned in the last eight years of the mass collaboration era is that, whatever an organization cares about, crowds can help us conceive of it, build it, test it, market it, support it, and fix it--and do all of that at scale.

For most of the social media era, we've not been able to manage this data effectively when the social platforms were not built and controlled by us. Even when we controlled the platform, the unstructured, informal, and otherwise messy nature of human conversation has been quite a hindrance in using automated tools to scale up. This made it difficult to maintain an updated and integrated picture of what was happening and which conversations really mattered to the business.

We now have a much better idea of how this challenge, namely the ability to establish a working feedback loop from the social world back to us, is going to be resolved. This largely falls under the rubric of Big Data, an umbrella term for a loosely related set of technologies, some old and some new, that let us quickly processes the vast data streams in our social environments, meaningfully analyze the freeform information within them, and zero in in the events, situations, and trends that are interesting to us and our businesses.

I've previously broken down the moving parts of Big Data as well as articulated how it supports the missing social business intelligence capability within most of our organizations. I've even stressed how these must be put into the hands of the average worker in order for real value to emerge.

To address this, an entire cottage industry of social analytics and BI tools have sprung up in the last couple of years to directly enable this scenario. You have seen the acquisitions by established players of companies like Radian6 (Salesforce), ProximalLabs (Jive), and CoreMetrics (IBM). Now it's time for practitioners to close the loop and get the ROI by tapping into the deep wells of knowledge and engagement that can drive forward very real business outcomes.

To do this, you will need data scientists that understand your business, technical capability (which may or may not be possible to outsource), and willingness to overhaul business processes like marketing, sales, customer care, and product development so that they'd be infused with the very latest intelligence and insight from the real world. This is harder than it sometimes looks, as I've explored in the cultural aspects of social business transformation. But as a new piece on Big Data last month on Harvard Business Blogs shows, disciplined and forward thinking organizations have much to gain.

Every company needs a social networking policy, but don't stifle creativity and productivity with too much formality. Also in the debut, all-digital Social Media For Grownups issue of The BrainYard: The proper tools help in setting social networking policy for your company and ensure that you'll be able to follow through. (Free with registration.)