Government // Enterprise Architecture
Commentary
2/16/2012
10:05 AM
Rajan Chandras
Rajan Chandras
Commentary
Connect Directly
RSS
E-Mail
50%
50%

The Shadow IT Threat

Rogue IT groups outside the main IT organization pose both pros and cons--and won't disappear anytime soon.

Organization structures for IT departments follow the same principles as those for any other department: structure, processes, culture and controls, all hovering around people, of course. The CIO has the same challenges as any other department head: recruiting and retaining talent, fostering teamwork, balancing accountability with reward, and driving motivation and innovation.

Yet, IT departments have a unique challenge: those rogue, shadow IT groups that sprout all over the company, usually unannounced and unheralded, yet curiously self-sufficient, well funded and with strong sponsorship and support.

What gives? In an era of strong information governance, tight management controls and stringent security policies--not to mention closely watched operational budgets and an increasing emphasis on centralized centers of excellence--how can groups of business-technologists create these undetected oases of expertise and grow them into powerful quasi-IT organizations in their own rights?

Proponents of shadow IT groups make one very clear point: There are regional and satellite sales groups so what's wrong with satellite IT groups? Business intelligence vendors and practitioners like to flaunt the mantra of self-sufficiency so why crib and cavil if, say, the Customer Service group brings in a BI tool it likes, and unleashes it on sales and CRM systems data, giving the group agility and self-control?

To understand the advent of shadow IT you have to look back at the history of computing.

The seeds of the shadow IT approach were sown, in part, when the tightly controlled and centralized mainframe computing paradigm gave way to minicomputers--computing capabilities that weren't just serving localized corporate groups, they could actually be budgeted and managed locally. The advent of personal computing gave further impetus; and data connectivity and adapters, and user-friendly analytic technologies like QlikView and Tableau, and of course the ubiquitous Microsoft Excel, made it easier than ever (and ever so tempting) to quickly reach out to the data you need and harness it to construct meaningful, focused local data stores and reports.

In other words, blame it on technology! But if that seems like not quite the full story, you're right.

Shadow IT groups serve a useful purpose, cutting short the time between making a request of IT and getting the answer (especially for a data extract or a few reports). But they also undercut good governance, reducing operational efficiencies, creating avoidable expenses and increasing exposure to risk.

Global CIO
Global CIOs: A Site Just For You
Visit InformationWeek's Global CIO -- our online community and information resource for CIOs operating in the global economy.

The shortcomings of shadow IT groups are many.

Procurement optimization and vendor management: Vendors are adept at detecting opportunities to "go directly to the user" and sell products and services in ways that compromise--if not outright bypass--centralized corporate procurement practices. The result weakens an IT organization's ability to obtain better pricing and purchasing terms with the vendor.

Enterprise architecture: Enterprise architecture groups, if they exist in your organization (and I sincerely hope they do!), exist for the purpose of defining architectural standards and best practices that best suit the company, and ensure that technologies procured and deployed align with a long-term architectural vision. Unregulated deployment of technology can disrupt architectural consonance leading to headaches down the road for all concerned. Implementing non-standard technologies can also lead to future architectural divergence, with more serious consequences. (How rigorously should EA standards and policies enforced? That's a topic for another time.)

Technology support: Technology support, including hardware, software, networking, storage, and telecommunications, can and usually do consume a significant portion of corporate IT budgets. Technology management is inherently fraught with uncertainty and complexity; balancing difficult-to-forecast requirements for infrastructure with myriad technology solutions and infrastructure management practices is far more difficult than it seems. And there is often a trade-off between cost and complexity; for example, savings accomplished using virtualization is often at the cost of increased complexity of managing virtual machines and storage and application deployment thereon.

Corporate risk: Fostering localized fiefdoms of data and reporting creates big information security and compliance risks. Notwithstanding our years of experience with managing and disseminating information, maintaining strict information security is complex, expensive and remains an elusive goal. "Rogue" or "underground" data repositories and reporting mechanisms increase the risk that information may be shared inappropriately and with the wrong people. The consequences of such risk can be substantial, if not downright disastrous, as for instance in this case at Stanford Hospital.

Is there a solution? Can we get rid of shadow IT groups entirely?

That's not likely. Most office jobs are now so information-driven that satisfying and regulating these information needs is next to impossible. However, defining clear data governance practices and establishing meaningful dialogs with these groups is a good first step toward achieving a balance between individual and group needs for information and the corporate need for information security and control. How exactly do we go about doing that? That's also a topic for another day.

Rajan Chandras has more than 20 years of experience advising and leading business technology initiatives, with a focus on strategy and information management. Write him at rchandras at gmail dot com.

IT's jumping into cloud services with too much custom code and too little planning, our annual State of Cloud Computing Survey finds. The new Leap Of Cloud Faith issue of InformationWeek shows you what to be aware of when using the cloud. Also in this issue: Cloud success stories from Six Flags and Yelp, and how to write a SAN RFI. (Free registration required.)

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
trustedadvisor
50%
50%
trustedadvisor,
User Rank: Apprentice
11/25/2013 | 12:54:52 PM
There is a Solution to Shadow IT..
I just want to make everyone aware of a new software solution company that has developed an elegant solution for Shadow IT. This software helps organizations make better and faster technology buying decisions while enabling a more effective solution to deal with decentralized technology spend and Shadow IT. The company is SelectHub.com

SelectHub saves months of work during the requirements compilation, vendor research and evaluation process. You can also leverage the community within SelectHub to help you evaluate technology vendors - based on their previous experience with them.

Below are actual statements from real CIOs of prominent companies on their website:

"SelectHub is huge! It takes the unstructured process of ad-hoc decision making that companies use in expensive technology selections and turns it into a fine tuned methodology." -Rajeev Ankireddypalli, CIO, Advanced Energy

"SelectHub is the only product of its kind in the market that has a real time system of record, optimizes spend, and improves the time to market for the business around the enterprise IT vendor selection process." -Jeff Kuckenbaker – CIO, Stanley Black & Decker , Inc.

"SelectHub fills a real void around Shadow IT Projects and IT Procurement best practices. It provides real time insight and visibility." -Keith Sherwell, CIO, Sears Holdings Corp.

"SelectHub provides the speed of real-time collaboration on technology procurement in a world where the old pace of doing things won't get the job done. SelectHub accelerates IT unlike any other tool." -Rob Meilen, CIO, Hunter Douglas

steve2insd
50%
50%
steve2insd,
User Rank: Apprentice
3/1/2012 | 1:36:35 AM
re: The Shadow IT Threat
Dictators vs Rogues. Sounds like yet another MMORPG. Sounds like Anon has gone through the long, fun process of flushing out all of his Rogues. It would be interesting to hear from a former Rogue, and maybe other "customers" in his own enterprise rate the results. He might very well have it running smoothly by now. If so, I would guess that communication between IT and the other departments is very good. If not, and the enterprise isn't benefiting from the control he has managed to implement, I would expect abrupt change once the bottom line reflects that.

These days IT is an important gear in the machine. The smoother it meshes with all the other important gears, the longer and smoother the machine will run. Nothing new, but re-learned often.
Sam Iam
50%
50%
Sam Iam,
User Rank: Apprentice
2/27/2012 | 7:37:52 AM
re: The Shadow IT Threat
Agree, the primary complaint against IT has nothing to do with IT, it has to do with customer service. In a large organization, shadow IT is often necessary just to enable critical tasks to be completed in their customers (external customers) timeframes. IT should understand this better than most. How many times has an IT department asked one of their vendors to create a workaround or a shadow process to meet their requirements in their timeframe?
Sam Iam
50%
50%
Sam Iam,
User Rank: Apprentice
2/27/2012 | 7:18:01 AM
re: The Shadow IT Threat
Imagine, for a second, that your response was coming from one of your service providers, SAP, Cisco, IBM, Oracle or whoever, in response to your new requirements. How long would you be doing business with that provider if they told you that bringing in Juniper firewalls or a new DB is not consistent with their standards and you need to operate within their architecture? Even if they had a point, you would be outraged. Who are they to tell you your requirements? That is essentially what you are telling your "internal customers." It is why shadow IT springs up. Not because marketing or accounting really wants to be in IT, but because they need to get things done and cannot be told what is and is not required.
Sam Iam
50%
50%
Sam Iam,
User Rank: Apprentice
2/27/2012 | 6:40:25 AM
re: The Shadow IT Threat
There are a whole bunch of reasons for shadow IT, but the two primary reasons, IMO, are that enterprise IT departments generally are: 1) Not immediately responsive to change, certainly not proactive 2) When they finally accept changes, they are too slow to implement those changes.

Large IT departments have their standards and occasionally there is valid reason, but more often than not it is just about preserving the status quo (e.g. reluctance to do anything non-Microsoft for client side). Shadow IT will stop appearing when IT can move at the speed of business. I am a proponent of giving IT some competition. It keeps them on their toes and makes their internal "customers" true customers. You are only a customer if you have the option to go elsewhere if your needs are not being met. That is why shadow IT springs up. Not because supply chain or sales, for instance, wants to be in IT, but because they need to get things done. It is true that these shadow groups create all sorts of enterprise architecture, data "silo", and management issues, but what good is having enterprise architectures, master data management, etc if they move too slowly to actually provide services in required timeframes.
ANON1233861206246
50%
50%
ANON1233861206246,
User Rank: Apprentice
2/21/2012 | 3:27:08 PM
re: The Shadow IT Threat
Just a few replies to "Tecknical" regarding his comments:

1) Regarding the re-write of the custom application: You state "But the IT department is easily migrating the data and rewriting the application to support it themselves." I might point out that "easily" is a relative term. Was this a planned project for the IT department? Doesn't sound like it was. Was this a budgeted project for the IT department? Probably not. How business critical is this application? If it is business critical, why wasn't it done in partnership with IT in the first place? Had the project been developed with IT involved from the start, there would be no rewrite project needed at all. I don't envy the IT manager having to explain to his/her boss why he needs money and people to work on a re-write of a custom application that was so productive and important that it was written and implemented without any IT involvement. Must have been a fun conversation.

2) How ignorant and arrogant you are to suggest "Anon, like so many similar CIOs, is not so much interested in the overall business, as in the efficiency of the precious IT empire. They always cite horror stories to sustain their dictatorship." You obviously don't work in IT nor do you understand the value of IT and how it supports a business. Your claim that "IT departments cannot understand the business as well as the respective departments" is proof you have no clue what you are talking about. I actually know my business exceedingly well, because I support and provide service to every department in the business. This is exactly what I was talking about when I mentioned how the business functions as an enterprise and not as a disjointed collection of "activity silos". IT is not just the "computer guys" who place a workstation on your desk and show you how to logon to the network. But based on your comments, you wouldn't know that, would you?

3) Finally, your perception about SAP just proves my point that you should not comment on what you don't know. You state that "We just installed SAP, not because the business needed it, but because it made life easier for the IT department." IT departments do not implement SAP. Businesses implement SAP. IT provides service and support through technology and functional business process knowledge via detailed business process flows as they move through SAP (or any other ERP solution). This is why it's called Enterprise Resource Planning; because it supports the entire enterprise. I can see how you, someone who has obviously never implemented SAP and doesn't know what ERP is, would think it was only an IT "tool".

I suspect you've never had a job in IT. My guess is you're one of these folks who wish they could get a job in IT, but you can't because you don't get it. So, you sit in your "innovation incubator" and bluster about your own self-importance and throw stones at those who do understand IT and work hard to make sure it works well for their business.

I've had my say. If you'll excuse me now, I have a dictatorship to attend to.
Tecknical
50%
50%
Tecknical,
User Rank: Apprentice
2/21/2012 | 12:56:24 PM
re: The Shadow IT Threat
I object to the term "rogue', and prefer to call them innovation incubators. I prefer to call the dictator CIOs rogues. We had an excellent application/database developed by an expert engineer which served the business well and saved millions in business efficiency, not to mention saving the heavy overhead our IT department imposes on the business. It also saved procurement costs (although there was, and still is, nothing like it on the market). The business got six years of productive value from the application.

Then the unthinkable happened. The developer retired with a terminal illness. At this point Anon would salivate and say "Told you so." But the IT department is easily migrating the data and rewriting the application to support it themselves. And this too is saving the business, with no need for BAs or procurement or user training. The application was well established and the new developers have an excellent prototype to work from. The whole exercise saved millions and was most educational for the IT department.

I would go so far as to suggest Anon, like so many similar CIOs, is not so much interested in the overall business, as in the efficiency of the precious IT empire. They always cite horror stories to sustain their dictatorship.

I agree that security needs to be centralized, as does architecture, most hardware, etc. But a good CIO will encourage a certain level of innovation and support it from a distance without stifling it to death. IT departments cannot understand the business as well as the respective departments. Any CIO who thinks she is the sole dispensary of wisdom, and he is always going to be correct is a menace to the business.

We just installed SAP, not because the business needed it, but because it made life easier for the IT department. This is the danger of Nazi IT departments. SAP has proven to be counter-productive for the business. We were better off before, but there will be no telling this to the IT department. It's forbidden.
Bprince
50%
50%
Bprince,
User Rank: Apprentice
2/21/2012 | 12:32:26 AM
re: The Shadow IT Threat
Good points by IndTecSvcs and Anon. Any type of IT blind spot poses a risk to an organization in my opinion. But it also sounds as if these groups are being created out of necessity in some cases. So how can that be addressed?
Brian Prince, InformationWeek/Dark Reading Comment Moderator
ANON1233861206246
50%
50%
ANON1233861206246,
User Rank: Apprentice
2/19/2012 | 3:57:33 PM
re: The Shadow IT Threat
As an IT manager that needs to support 300 users, manage over 40 different systems (including SAP), run a helpdesk, and implement solutions and manage projects (all with a staff of 2, including myself) I can tell you that I do not allow rogue IT groups to exist. Why? Because I am the individual who is personally *accountable* for the control of IT. All of IT, I might add.

For those who favor the existence of such groups, I ask the following rhetorical question: When the rouge "IT" group in Accounting has one of their analysts, Fred "The Database Expert", implement a solution in the Accounting "silo", who supports the solution when Fred leaves the company? You can bet that no one in Accounting will know how to support his solution, nor will they want to take ownership of the problem. Who do you think gets a phone call when this happens? I can tell you, it's not the Accounting manager. Allowing the existence of rogue IT groups is bad policy. If there is an IT solution to be crafted, then IT needs to be involved. If the company realizes a valid business need for this solution, then the company will find the assets and resources to get it done...with IT's oversight and management. To combat the "spawning" of rogue groups, I have implemented numerous network access controls, procedures and policies at my company.

Imagine that a user wants to put a USB color ink-jet printer on their desk because it will make them "so much more efficient". However, the *company* (not a single group within the company) made an earlier business decision to centralize print services to control costs and save money. Why spend the time to centralize print services just to let groups do what they want? Doing so creates support issues and increases cost, because now that group is buying ink cartridges and has limited expertise (or time) to work on problems. And when the printer breaks, who do you think they will call? That's right, the IT group. To combat this at my company, IT controls access to all ports on all PCs. If a user should decide to buy a desktop printer on their P-Card (or bring one in from home) they will not be able to use it on company IT equipment. This is communicated to the users during orientation and throughout the year. It has been very successful in eliminating a rouge mentality as it pertains to print services.

What if a user wants to put their new personal laptop on the network? The answer is no. IT has the network locked down via IP reservations tied to MAC address. Nothing gets on the network unless IT has allowed it. For security purposes, no personal computer equipment is allowed on the company network. IT policies and controls also prevent any user from installing software of any kind on their company workstations. All software distribution is managed by IT, as it should be.

Why behave in such a Draconian fashion? In a word: control. Control of processes, procedures and policies. Businesses are at risk if they are not in control of their processes. Money can be stolen, equipment can be damaged, and people can even lose their lives if there is a loss of control over processes. In addition, companies need to operate as a single organism. People need to view the business as an enterprise that moves forward as one unit. Successful businesses do not operate in separate silos of activity. Anyone who has implemented an ERP solution, such as SAP, will tell you this. This mentality goes for groups within an enterprise. You cannot have a singular vision for IT and move IT forward as one unit if you have these rogue groups operating outside the scope of the mandated IT organization.

I also look at this from a personal perspective. I was hired to run IT for the company. I cannot be in control of IT processes or infrastructure if I allow rogue groups to exist. How can I be in control of my network and its related security if I allow anyone outside of the IT group to add a device to the network? The answer is simple: I cannot. Assume I allowed a rogue group to operate and implement a solution in Marketing that somehow caused the network to crash. Does anyone actually believe the president of the company will ask the Marketing manager why the network crashed? Of course not. She will ask me. And what do you suggest I tell her? "You need to talk to Marketing's IT group"? Give me a break.

Allowing these rogue groups to exist is bad for your company, bad for your legitimate IT organization, and potentially bad for your career. It is not how an enterprise should operate, at least an enterprise that wants to move forward as a collective in control of its processes.
IndTecSvcs
50%
50%
IndTecSvcs,
User Rank: Apprentice
2/17/2012 | 9:16:06 PM
re: The Shadow IT Threat
Despite the shortcomings you've listed, "shadow IT" groups are essential to conducting business. Highly bureaucratic, inefficient corporate IT departments stifle innovation and actually inhibit day-to-day business activities. In my 20 plus years working in and with IT, I have had the opportunity to work on both sides of the fence. I understand the risks that you've outlined, and they are valid concerns. I also understand that businesses must conduct business. It's the responsibility of IT to facilitate that process - not to unnecessarily stifle it.

As an IT consultant, the vast majority of customer complaints I have heard have absolutely nothing to do with valid IT concerns. Instead, the primary complaint is that IT groups use red tape to simply shelve difficult requests in hopes that they'll just go away. I've heard this numerous times through the years, and I've witnessed it first-hand. IT must remain relevant by working with customers to further the goals of the organization while also protecting that same organization from internal and external threats to infrastructure, data integrity, and security. The failure of IT to achieve both of these essential tasks will serve to perpetuate the need for "shadow IT."
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - July 22, 2014
Sophisticated attacks demand real-time risk management and continuous monitoring. Here's how federal agencies are meeting that challenge.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.