Regulatory Oversight vs. Crowdsourcing: The Best Approach for Quality

You are in the analytics business. The goods or services you produce have an impact on individuals, businesses, governments, the nation, or the world. Perhaps your analytics will affect public policy on funding for new schools, clean drinking water or automobile safety. Perhaps your analytics will foster the development of a new cancer drug. Perhaps your analytics prevent terror attacks or help defend national interests abroad. Perhaps your analytics will help retail sales, financial investments or guide fiscal/monetary policy. Perhaps your analytics allow you to sleep easily, knowing that you helped make this world just a little bit better. Question- who should be assigned the task of verifying and validating your goods and services -- an auditing/regulatory commission or your customer base? Let's consider the pros and cons of regulatory oversight versus crowdsourcing by transparency and access.

Regulatory oversight is the best way to verify and validate analytic goods and services. Auditors and regulators are trained subject matter experts and thought leaders. They are able to discern overt and covert anomalies. They can be held accountable for their actions or lack thereof. They are objective in their decisions and have no vested interests in your success or failure.

Crowdsourcing by transparency and access is the best way to verify and validate analytic goods and services. The customer base consists of daily practitioners who have vested interests in your products. They have formed user groups, established blogs, and attend conferences to discuss your products. They may even have pens, t-shirts, and coffee mugs with your logo. They find problems with your products before your developers do. And with the power of the Internet, they can grow or collapse your business in a heartbeat.

So is it preferable to have a limited set of independent trusted advisors or use customer-friendly transparency and easy access (i.e., open source code, published business rules and methods, social media, etc.) to verify and validate analytic goods and services? Both approaches have merits. From my own experiences, especially when IT contractors were hired in the 1990s to build health care fraud and abuse detection systems, I think the answer depends on costs versus the benefits. The people who provide regulatory oversight may be best equipped and have the backing of the law to do their work. However, the amount spent to pay them may dwarf the dollar value of the malfeasance discovered. As it relates to health care fraud, we found more cases of bad training or conflicting state/federal rules than actual cases of fraud. There were cases where health care providers filed claims for dead people. But for the most part, we earned more money than the amount of actual fraud discovered. Crowdsourcing using the customer base does not provide a concrete point of accountability, but it is a frugal method of anomaly detection.

But to be even-handed, though the transparency and access approach is often deemed more cost efficient than regulatory oversight, it is not necessarily more cost effective. Trusting the customer base prevented neither the 2008 crash nor the 2015 stock market flash crashes. The implementation of the Dodd-Frank legislation and 'after trading hour' monitoring have imposed the kinds of controls that would not be possible with simply feedback from the customer base.

So what's the answer? Is it the thesis of regulatory oversight of the antithesis of crowdsourcing by transparency and access? Or is it in the synthesis of both approaches? Is the answer not 'either/or' but rather 'both/and'? Do we need to find ways of combining the efforts external agents and internal stakeholders for verifying and validating analytic goods and services? Is all of this a non-issue?