Organization structures for IT departments follow the same principles as those for any other department: structure, processes, culture and controls, all hovering around people, of course. The CIO has the same challenges as any other department head: recruiting and retaining talent, fostering teamwork, balancing accountability with reward, and driving motivation and innovation.

Yet, IT departments have a unique challenge: those rogue, shadow IT groups that sprout all over the company, usually unannounced and unheralded, yet curiously self-sufficient, well funded and with strong sponsorship and support.

What gives? In an era of strong information governance, tight management controls and stringent security policies--not to mention closely watched operational budgets and an increasing emphasis on centralized centers of excellence--how can groups of business-technologists create these undetected oases of expertise and grow them into powerful quasi-IT organizations in their own rights?

Proponents of shadow IT groups make one very clear point: There are regional and satellite sales groups so what's wrong with satellite IT groups? Business intelligence vendors and practitioners like to flaunt the mantra of self-sufficiency so why crib and cavil if, say, the Customer Service group brings in a BI tool it likes, and unleashes it on sales and CRM systems data, giving the group agility and self-control?

To understand the advent of shadow IT you have to look back at the history of computing.

The seeds of the shadow IT approach were sown, in part, when the tightly controlled and centralized mainframe computing paradigm gave way to minicomputers--computing capabilities that weren't just serving localized corporate groups, they could actually be budgeted and managed locally. The advent of personal computing gave further impetus; and data connectivity and adapters, and user-friendly analytic technologies like QlikView and Tableau, and of course the ubiquitous Microsoft Excel, made it easier than ever (and ever so tempting) to quickly reach out to the data you need and harness it to construct meaningful, focused local data stores and reports.

In other words, blame it on technology! But if that seems like not quite the full story, you're right.

Shadow IT groups serve a useful purpose, cutting short the time between making a request of IT and getting the answer (especially for a data extract or a few reports). But they also undercut good governance, reducing operational efficiencies, creating avoidable expenses and increasing exposure to risk.

Global CIOs: A Site Just For You Visit InformationWeek's Global CIO -- our online community and information resource for CIOs operating in the global economy.

The shortcomings of shadow IT groups are many.

Procurement optimization and vendor management: Vendors are adept at detecting opportunities to "go directly to the user" and sell products and services in ways that compromise--if not outright bypass--centralized corporate procurement practices. The result weakens an IT organization's ability to obtain better pricing and purchasing terms with the vendor.

Enterprise architecture: Enterprise architecture groups, if they exist in your organization (and I sincerely hope they do!), exist for the purpose of defining architectural standards and best practices that best suit the company, and ensure that technologies procured and deployed align with a long-term architectural vision. Unregulated deployment of technology can disrupt architectural consonance leading to headaches down the road for all concerned. Implementing non-standard technologies can also lead to future architectural divergence, with more serious consequences. (How rigorously should EA standards and policies enforced? That's a topic for another time.)

Technology support: Technology support, including hardware, software, networking, storage, and telecommunications, can and usually do consume a significant portion of corporate IT budgets. Technology management is inherently fraught with uncertainty and complexity; balancing difficult-to-forecast requirements for infrastructure with myriad technology solutions and infrastructure management practices is far more difficult than it seems. And there is often a trade-off between cost and complexity; for example, savings accomplished using virtualization is often at the cost of increased complexity of managing virtual machines and storage and application deployment thereon.

Corporate risk: Fostering localized fiefdoms of data and reporting creates big information security and compliance risks. Notwithstanding our years of experience with managing and disseminating information, maintaining strict information security is complex, expensive and remains an elusive goal. "Rogue" or "underground" data repositories and reporting mechanisms increase the risk that information may be shared inappropriately and with the wrong people. The consequences of such risk can be substantial, if not downright disastrous, as for instance in this case at Stanford Hospital.

Is there a solution? Can we get rid of shadow IT groups entirely?

That's not likely. Most office jobs are now so information-driven that satisfying and regulating these information needs is next to impossible. However, defining clear data governance practices and establishing meaningful dialogs with these groups is a good first step toward achieving a balance between individual and group needs for information and the corporate need for information security and control. How exactly do we go about doing that? That's also a topic for another day.

Rajan Chandras has more than 20 years of experience advising and leading business technology initiatives, with a focus on strategy and information management. Write him at rchandras at gmail dot com.

IT's jumping into cloud services with too much custom code and too little planning, our annual State of Cloud Computing Survey finds. The new Leap Of Cloud Faith issue of InformationWeek shows you what to be aware of when using the cloud. Also in this issue: Cloud success stories from Six Flags and Yelp, and how to write a SAN RFI. (Free registration required.)