InformationWeek Stories by Alistair Croll

InformationWeek Stories by Alistair Crollhttp://www.informationweek.comInformationWeeken-usCopyright 2012, UBM LLC.2013-03-20T11:37:00ZHow Should We Measure Clouds?Too often, IT pros start at the bottom and work up. In the cloud, we need to look at the business model and set metrics from there.http://www.informationweek.com/cloud-computing/software/how-should-we-measure-clouds/240151231?cid=SBX_iwk_related_commentary_Infrastructure_as_a_Service_cloud_computing<div class="inlineStoryImage inlineStoryImageRight"><a href="http://www.informationweek.com/storage/data-protection/8-great-cloud-storage-services/240151180"><img src="http://twimgs.com/informationweek/galleries/automated/967/Cloud_Storage_Services_01_tn.jpg" alt="8 Great Cloud Storage Services" title="8 Great Cloud Storage Services" class="img175" /></a> <div class="storyImageTitle">8 Great Cloud Storage Services</div>(click image for larger view and for slideshow)</div>There isn't a simple answer to the above question. First of all, cloud computing is hidden behind a fog of abstraction. Whereas IT organizations could once instrument every element of an application, today's applications are like Descartes' brain in a jar -- we're never quite sure if they're real or virtual. Second, many cloud service providers' goals aren't aligned with those of their customers. Service providers want to maximize revenue and profit and want the freedom to do what they will with the underlying infrastructure. That's how they make the most of what they have and stay in business. Without that freedom, they lose economies of scale and skill. By contrast, customers want special treatment and instrumentation all the way down the stack. Third, people don't really understand metrics well. We still use averages even though they hide important fluctuations in service quality that can warn of problems before they become disasters. [ <a href="http://www.cloudconnectevent.com/santaclara/?_mc=MP_BTMEDIWKAXE">Cloud Connect</a> returns to Silicon Valley, April 2-5, 2013. Use Priority Code MPIWK by March 30 to save an extra $200 off the advance price of Conference Passes. Register for <a href="http://www.cloudconnectevent.com/santaclara/?_mc=MP_BTMEDIWKAXE">Cloud Connect</a> now. ] But there's an even bigger problem here. For half a century, IT has been about protecting precious resources. The reason you put up with carrying a stack of punched cards to the basement of the computing building at 3 a.m. was because mainframe resources were scarce and the humans abundant. No more. Each of us has three screens, one of which is seldom more than a meter from our bodies at any time. That means we're less concerned about the consumption of resources and more concerned about the completion of tasks. We shouldn't really care if the CPU is idle or maxed out, provided that users accomplish what they set out to do. Proponents of service-level agreements have long known this, but cloud monitoring, hiding behind the fog of virtualization, drives the point home. Application performance management (APM) and real user monitoring (RUM) have long been considered "advanced" forms of measurement. They go beyond up/down metrics or numbers related to utilization, and instead look at the success of the application from the user's point of view. (Disclosure: I founded Coradiant years ago, but I don't have any interest in the APM sector today.) APM and RUM have often languished somewhere between Web analytics (which show you what users did) and synthetic monitoring (which shows you whether the site is working). Today, however, the real question is: Could they do it, and well? <div style="margin:0; padding:0 0 10px 15px; width:244px; float:right;"><div style="margin:0; border-top:1px solid black; border-bottom:1px solid black; padding:6px;"><a href="http://www.informationweek.com/global-cio/"><img src="http://twimgs.com/infoweek/1217/217ID_GlobalCIO_75.jpg" width="75" height="75" border="0" align="right" alt="Global CIO" style="margin:0 0 6px 6px;"></a><div style="margin:0 0 6px 0; font-size:1.3em; font-weight:bold; color:#113e53;">Global CIOs: A Site Just For You</div>Visit <a href="http://www.informationweek.com/global-cio/">InformationWeek's Global CIO</a> -- our online community and information resource for CIOs operating in the global economy.</div> </div>   There's much evidence that slow applications undermine productivity, cost money and cut into revenues. Slow clouds need fixing. To do this, we need to go beyond APM and start with the business problem. Too often, IT professionals start at the bottom and work up. "Server 10 is down, which means the support site isn't working, which means the phone queue is too long, which impacts our customer satisfaction rating." They begin with the means and work back to the end. Instead, we need to step back and look at the business model. From there, we can derive the relevant metrics and what's considered an acceptable threshold. Then we can measure against those thresholds and report on violations. That's a much more palpable approach to measurement for executives. Starting at the model and working down suggests we say: "7% of visits need to result in an enrollment for us to meet our monthly target." From there, we can measure the steps of an enrollment and performance against the past or response targets. When we owned the infrastructure, this kind of approach was considered progressive. But the fog of cloud monitoring means it's often the only way we can measure. It lets us size cloud consumption, which in turn lets us define budgets -- since with the right architecture, you can have any performance you can pay for. And it leads to good metrics, since it's focused on rates and exceptions rather than averages. We'll be talking about how to measure cloud-based applications at the <a href="http://www.cloudconnectevent.com/santaclara/?_mc=ADCC100&gclid=CI7Vlc6-i7YCFcme4AodVAkAUg ">Cloud Connect event, April 2 to 5</a> in Santa Clara, Calif. In fact, we have a whole track of content dedicated to it, including sessions on WANs, application delivery networks, load-balancing and choosing the right metrics. Clouds are the IT of abundance, and they fundamentally change how we measure applications. Let's figure out how. <a href="http://www.cloudconnectevent.com/santaclara/?_mc=MP_BTMEDIWKAXE">Cloud Connect</a> returns to Silicon Valley, April 2-5, 2013, for four days of lectures, panels, tutorials and roundtable discussions on a comprehensive selection of cloud topics taught by leading industry experts. Join us in Silicon Valley to see new products, keep up-to-date on industry trends and create and strengthen professional relationships. Use Priority Code MPIWK by March 30 to save an extra $200 off the advance price of Conference Passes. Register for <a href="http://www.cloudconnectevent.com/santaclara/?_mc=MP_BTMEDIWKAXE">Cloud Connect</a> now. 2013-02-11T11:55:00ZLean IT: Think Big About AgilityCloud, DevOps, Agile and Lean Startup methodologies all aim to make companies more adaptive. But alone, no one of them can deliver on that promise.http://www.informationweek.com/cloud-computing/platform/lean-it-think-big-about-agility/240148276?cid=SBX_iwk_related_commentary_Infrastructure_as_a_Service_cloud_computingThere's been a lot of talk about agility lately. Everyone, it seems, wants their company to be a circus acrobat: nimble, capable of improbable feats of derring-do. Dozens of "big ideas" have put on the mantle of agility in the hopes of convincing the world they're the answer to the sluggishness of big organizations. The problem is that none of these ideas is a panacea; rather, each is a piece of a bigger puzzle, that of transforming an organization into an organism. Let's look at a few of these agile claimants first. <a href="http://en.wikipedia.org/wiki/Cloud_computing ">Cloud Computing</a> is an umbrella term for a variety of big changes in IT. It's the move from physical machines to virtual ones; the move from computers to computing; and the shift from on-premise to third-party pay-as-you go IT. Clouds are IT in an era of abundance, and used properly, they make huge rearchitecting of IT systems possible with just a few mouse clicks, because they remove the friction of change. <a href="http://en.wikipedia.org/wiki/Devops">DevOps</a> is a portmanteau of development -- the writing of software -- and operations -- the running of it. It's a recognition of the fact that modern developers aren't just coding the application, but also the infrastructure on which it runs, and increasingly the data on which it relies. A DevOps mentality means software that's able to adapt to the environment in which it operates, and that can scale and relocate in response to outages or increased demand. <a href="http://en.wikipedia.org/wiki/Agile_software_development">Agile Development</a> is, as its name implies, a more responsive way of building software. In contrast to the old "build it and they will come" model of waterfall development, which assumed that the specification was correct, Agile emphasizes short cycles of development, testing, validation, and adjustment, along with continuous deployment, in order to deal with shifting requirements. And <a href=" http://en.wikipedia.org/wiki/Lean_Startup ">Lean Startup</a> is a way of building new companies and new products through iterative learning. It focuses on identifying the riskiest part of a business, and doing just enough work to overcome that risk. Rather than selling what you can make, Lean says you should make what you can sell -- and you find out what you can sell through relentless, close customer development in search of the magical fit of the right product for the right market. [ Learn more about these and other hot topics at <a href="http://www.cloudconnectevent.com/?CID=MP_CCSV_IWK_Article_TL">Cloud Connect</a> Silicon Valley, April 2-5. There will be four days of lectures, panels, tutorials and roundtable discussions on a comprehensive selection of cloud topics taught by leading industry experts. Register for <a href="http://www.cloudconnectevent.com/?CID=MP_CCSV_IWK_Article_TL">Cloud Connect</a> now. ] None of these ideas is brand new. Agile development owes much to notions of incremental development in the 1950s; the first clouds were probably mainframes; and Lean Startup was inspired by concepts from Japanese manufacturing. But it's only in recent years that the widespread use and consumerization of technology that these things have truly gone mainstream. These and related issues will be discussed in a conference track that I am moderating on <a href="http://www.cloudconnectevent.com/santaclara/conference/futures-and-disruptions.php">"Futures and Disruptions"</a> at <a href="http://www.cloudconnectevent.com/santaclara/">Cloud Connect</a>, April 2-5, in Silicon Valley, Calif. From organization to organism Cloud, DevOps, Agile and Lean all want to make us more adaptive. But alone, no one of them can deliver on that promise. Cloud computing can reduce the coefficient of friction of changes dramatically -- but without a DevOps approach to adaptive infrastructure, it can't shine. Similarly, Lean Startup product managers can't achieve the tight iteration and continuous learning they need if the development team isn't using Agile coding. No, what companies want -- what startups want -- what everyone wants -- is to transform from an organization to an organism. An organism is a hierarchical assembly of systems working together as a single functional unit, often thought of as a self-organizing being. To react in a controlled, agile, measured way, organizations need to behave as organisms. And since IT is the central nervous system of the modern business, change starts here. This requires a holistic approach that encompasses all of these disciplines. You can't just have some of them and expect to reap the benefits of their entirety. Much of the disappointment, disillusionment, and backlash against these otherwise noble initiatives comes from not seeing the bigger picture. Part of a bigger whole Integrative philosopher and author <a href="http://en.wikipedia.org/wiki/Ken_Wilber ">Ken Wilbur</a> has spent his life trying to find commonalities in science, systems of belief, and so on. He talks a lot about "holons", a term coined by Arthur Koestler in 1967 in <a href="http://books.google.ca/books/about/The_ghost_in_the_machine.html?id=UAtbAAAAMAAJ&redir_esc=y">The Ghost In The Machine.</a> A holon is a thing that has both properties of self, and properties of membership. A cell in your body, for example, is part of a tissue. That tissue is part of an organ. That organ is part of you. You're an organism. As an organism, you function independently of your organs. You aren't consciously aware of muscles when you lift your arm; you don't think about nerve impulses when you notice something is hot; you don't raise your heartbeat when you need more oxygen. These things just happen. Each "whole" in your body is a thing unto itself, but sublimates itself into a greater whole.Humans are part of a greater whole, too. Jonathan Haidt explains in <a href="http://ideas.time.com/2012/03/27/have-we-evolved-to-be-religious/">The Righteous Mind</a> that sublimating ourselves into a greater whole has significant evolutionary advantage. "Ecstatic" experiences, such as a religious epiphany, or dancing at a rave, or even rooting for a sports team, provoke specific feelings in our brains that take us out of ourselves and make us feel part of something greater. We stop being self-organized, and start being part of the organism. We function as one, with a collective consciousness. It's this organism that we're after when we try to improve business. The circus acrobat's agility is their organism. It's their nerves, their muscles, their cardio-pulmonary system, working in concert, adapting instantaneously to changes, continuously learning and iterating. Organizations don't want a security department -- they want an immune system. They don't want a customer support desk -- we want a reflex action. Organizations regulate, delegate and proceduralize; organisms react, respond and learn. The reason we haven't reaped the full harvest of clouds, agile, Lean, and DevOps is that we're still looking at these things as organs, when instead we have to sublimate them into the organism. Like the wheels on a tank tread, each wheel can turn on its own -- but only when all the wheels work together does the traction of the vehicle improve dramatically. We need a name for this. I'm going to call it Lean IT. It's a re-engineering of the supply chain of businesses, from the conception of a product to its delivery. One might argue that this only applies to software products, but I'd disagree: -- What are Fedex, DHL, or UPS if not cloud logistics? -- What is mobile CRM if not realtime salesforce automation? -- What is 3-D printing if not the replacement of inventory with data? Why Lean is hard for fat companies Lean IT requires that we understand a broad range of disciplines -- not just Lean, Cloud, DevOps and Agile, but also tools like analytics (to measure and learn) and Continuous Deployment (to provision and deploy.) Lean IT is also a real challenge for traditional IT organizations, which abhor change and value stability. Wilbur has two important criteria for what makes a holon, which underscore why creating this kind of environment is challenging for such organizations. -- First, it must be whole unto itself, and simultaneously a part of a greater system, as the heart is to the body. Holons that cannot maintain their wholeness will break up into their constituent parts -- just as a company that can't maintain a holistic Lean IT strategy will retreat into age-old silos of development, QA, and operations. -- Second, Wilbur says that holons can't form a whole unless the other parts exist, and the conditions are right for them to thrive. Put another way, a heart won't survive unless it's part of an organism that can feed it, shelter it and so on. If you've got an agile initiative, but the organism lacks the other pieces, you won't achieve Lean IT. Favoring readiness over stability I'll end with another analogy. When you have a tall tower, it's stable. But it's not in its most stable configuration, which would be one of lying down, the point of lowest potential energy. Traditional IT wants the position of maximum stability (lying down), because that's predictable and reliable. A tower that's standing up seems like a liability, because it can be knocked over. But that tower has a tremendous amount of potential energy, easily converted to kinetic energy with only a slight push. It can react in a direction with significant force. Lean IT -- and the much-overused Agility for which it strives -- is a stance an organization needs to adopt. It's one of lower stability, and greater responsiveness. It's one where the organization is able to embrace new things, act in new directions, forget the past. It values the ability to change over the reliability of staying in one place. But to do this, it needs balance and poise. New organizations are naturally in this position. Without investments in the past, without baggage, they're free to choose and adjust. They don't have processes and procedures, and they don't have the liabilities that come from history. More than anything, Lean IT is a threat to fat organizations. Only by adopting lean approaches throughout the product, development, deployment and operational aspects of IT will we truly achieve the agility we're after. Only by creating an environment in which Cloud, Agile, Lean and DevOps can thrive will they avoid the traditional silos from which technologies suffer. And only then will those organizations become organisms capable of the adaptivity we want.2012-07-17T11:35:00ZThe Rise Of Shadow ITWhen the business wants change so much that it's willing to go rogue on IT, is it misinformed, or justified? Consider this advice, CIOs.http://www.informationweek.com/news/240003838?cid=SBX_iwk_related_commentary_Infrastructure_as_a_Service_cloud_computingPart One: IT Is from Mars, Business Is from Venus We've moved from a world of scarce IT--where organizations guard their technology--to one of abundance, where technology permeates every part of our daily lives. But the enterprise IT team is still acting like IT is a scarce, precious resource. They're charged with protecting and controlling its use, reducing risk; while the world of clouds, startups, and innovation focuses on its abundance, maximizing its rewards. Once, IT was protected. And with good reason: computers cost millions, broke easily, and represented a vast barrier to entry. Companies competed based on their ability to corral capital, infrastructure, and labor--to control the means of production. [ For more cloud computing analysis, see <a href="http://www.informationweek.com/news/cloud-computing/infrastructure/232301203?itc=edit_in_body_cross">Top 12 Cloud Trends Of 2012</a>. ] Today, however, industry after industry is being disrupted by startups that see these once-daunting barriers to entry as crumbling castle walls. The line of business wants to innovate, and central IT is facing an internal revolt: Shadow IT. We can track the rise of shadow IT to three things: the personal computer; data carriers; and software-as-a-service. When the first home spreadsheet came out, early adopters found ways to use it. But unlike central IT, which could be tightly controlled, clients had little to stop employees from going rogue. Visicalc was the first assault on the walls of central IT; the rise of the Web only made things worse. Despite every attempt to lock down desktops, employees found ways to use their own tools. A second flank of the assault happened when salespeople, eager for an edge on their quarterly targets, embraced two-way pagers. The Blackberry was a rogue device, but the political capital of its users made it impossible for IT to refuse them outright. But it was software-as-a-service that really hammered the nails into the coffin of centralization. Not only were SaaS sites pay-as-you-go, but they were trivially easy to deploy and configure. Departments started circumventing IT for everything from CRM to web analytics, performance monitoring, expense management, invoicing, and more. Every organization has an immune system. Enterprise IT has been particularly well entrenched, because to them, change is bad. Change is the leading cause of outages and downtime, so it is to be avoided. But to the line of business, change is good. Without change, things stagnate, and competitors get the upper hand. The tension between innovation and operation was almost unbearable. And the rise of cloud computing was the crack that broke open the dam.Part Two: Therapy Now I'm going to play therapist. Why does the line of business want change so much it's willing to create its own shadow IT organization? Is it because it hates enterprise IT? Does it think it can do a better job? Those would be pathological reasons. If we assume that it's rational, then what it should want is to raise profits and lower costs--to turn resources into value. That's what everyone should want. There are two possibilities. Either it's going rogue because it's misinformed, or it's going rogue because it's justified. Is it misinformed? There's no doubt that public clouds have privacy, governance, and lock-in concerns that the line of business might not be aware of. There's also little doubt that for a predictable, stable workload, the raw cost of on-demand computing is higher than owning your own machines because you're paying someone else's profit, and paying for variance--this is simply a function of Just-In-Time economics. Is it right? It takes far too long to spin up internal resources--often longer than the lifetime of the application--and today's climate of experimentation exacerbates the issue. But there's a much more fundamental reason this couple isn't talking: They're not using the same language. The line of business thinks top-down, and it wants to go down as little as possible. It starts by saying, "I want a website for this promotion," and goes only as deep as it needs to. On the other hand, the enterprise IT department thinks bottoms up, and it wants to go up as little as possible. It starts by saying, "You'll need three 4-core servers and 2TB of data, running Linux, for a month," and only goes up as high as it needs to. When enterprise IT says it's building a cloud, it's using a bottom-up definition: machines available when you need them, paid for based on the technical resources (RAM, compute cycles, storage, bytes sent) consumed. Think virtualization, plus. By contrast, when the line of business says it's using a cloud, it's describing a top-down definition: a functional piece of software that just works, paid for based on consumption, seats, or some other business metric. Think Salesforce, minus. And this, right here, is the reason they want a divorce. I call this the service gap.Part Three: What's a CIO to Do? So what's an enterprise IT professional to do? As it turns out, plenty. Good CIOs manage processes. But great CIOs ask the business to bring them problems, and then they try to solve those problems using technology. And in a connected, data-driven world, there's no shortage of problems to tackle. They just require different solutions. Take a look at a public cloud provider. They have dozens of services--from compute, to storage, to message busses, to human APIs, to mailing systems, to payment. Which restaurant would you rather eat at? Resisting the variety they crave is deadly. But building a set of services they can embrace is great. Because when you build services, you don't give them access to the underlying components. You keep control of those. And that's good for everything from governance, to licensing cost, to operational overhead. Here's a concrete example: Storage. Most developers say, "I want a database." What they really want is a service they can query; today, that request is usually translated by IT (with its device-centric hearing) into, "I want a database server." The architecture of a database depends a lot on how it's used. Cassandra is a kind of data store that supports fast injection of new data, for example. A traditional RDBMS is great for joining related tables together. Other architectures and tools work well for other applications--fast concurrent search results, large object storage, reliable, geographically redundant storage, and so on. If, rather than spending a week deliberating and issuing a server, the CIO's team stands up a service, several things happen. First, the application performance will be better, because it'll be tailored to the underlying infrastructure. Second, the licensing will be easier, because there'll be a single set of underlying tools and the variety happens at the API layer, not at the component layer. And third, governance and automation is easier to manage because of standardization. But this doesn't happen automatically. It takes a real effort of will for a traditional, bottoms-up, change-adverse IT team to see itself as the creator of services. In other words, you need to switch from a culture of scarcity (where IT is a precious resource to be protected) to a culture of abundance (where it's an open tool for innovation) by building a rich ecosystem of services atop infrastructure you control--in both public and private cloud environments. Rather than resisting change, IT executives need to embrace it—and appeal to basic human motivations with a little hustle, a bit of seduction, a dash of arm-twisting, and a sprinkling of raw terror. The rise of shadow IT is a topic we'll discuss during an upcoming <a href="http://event.on24.com/r.htm?e=492783&s=1&k=1EAA626B15A0BA8AB41DDEE4C4917AB5&partnerref=ac">Cloud Connect webinar</a> on July 26th. Alistair Croll, founder of analyst firm Bitcurrent, is conference chair of the Cloud Connect events. Cloud Connect is expanding to the Windy City. Join 1,200+ IT professionals at <a href="http://www.cloudconnectevent.com/chicago/?_mc=UFPQCH06">Cloud Connect Chicago</a>, where you will learn how to leverage new cloud technology solutions to increase productivity and improve your business agility. Join us in Chicago, Sept. 10-13. Register today! 2009-09-16T14:56:22ZFor CIOs, Clouds Are The Fourth ColumnClouds are transforming IT; that's not news. But regardless of your cloud computing agenda, clouds are already affecting your IT plans, because they give IT executives a cudgel with which to bludgeon traditional software and infrastructure providers.http://www.iweek-interim.com/news/229205025?cid=SBX_iwk_related_commentary_Infrastructure_as_a_Service_cloud_computingClouds are transforming IT; that's not news. But regardless of your cloud computing agenda, clouds are already affecting your IT plans, because they give IT executives a cudgel with which to bludgeon traditional software and infrastructure providers.Every IT decision of any real consequence starts with a shortlist of three competing offerings. One of them is usually the incumbent provider -- Cisco, IBM, EMC, Microsoft, and so on. Along with this incumbent are a couple of alternate providers. Sometimes these providers are simply "column fodder" designed to rein in the incumbent; but many IT companies have built healthy businesses by being the alternate. It's time for a fourth column: a cloud-based offering. That means every Request for Proposals that a company issues must have a cloud-based option, regardless of whether the company actually plans to adopt clouds. Here's why. Clouds set the upper bar. A cloud-based solution is often cheaper than hosted software. That's because it's not as customized, and can amortize costs across many buyers. As such, it's a good "high watermark" for any IT project. Clouds act as a sanity check on pricing. Clouds make the "basic feature" set apparent. Cloud-based offerings, particularly SaaS applications, are based on mass production. They offer 80% of the features one might want, for 20% of the cost. This is how cloud vendors achieve economies of scale -- they don't do a lot of customization, and where they do, it's usually a set of tools and scripting that let you tailor it yourself. This means the cloud offering shows you what the core features are. They make it clear what's an extra, and what's "table stakes." Then it's up to the vendors in the first three columns to prove that they're worth the premium pricing because of their special features. Clouds force true cost accounting. Enterprise software pricing is confusing -- sometimes intentionally so. It's often tacked on to the price of a CPU, or a computer. There are add-ons, installation fees, per-seat licenses, and more. It can be hard to compare offerings cleanly. Then there's the storage, networking, security, and processor power needed to run those offerings. A cloud, on the other hand, has a relatively simple pricing model. There aren't hidden costs. Despite Salesforce.com's famous "no software" logo, they're really all about no hardware. Because clouds are specific to users or usage, it's easier to charge costs back to individuals or departments. By including a cloud vendor in a competitive comparison, you force a true assessment of costs across all participants, and surface the hidden costs of software you run on your own. Cloud pricing is easy to get. Cloud providers have a very different sales strategy from enterprise software. They seldom have direct sales teams; instead, they rely on word of mouth and easy trials to get adopted. That means they don't guard their pricing jealously, since they have a one-size-fits-all mentality. Clouds are all about self-service IT access, and their sales and marketing processes are also more self-service. As a result, it's easier to get numbers and details for the fourth column: just visit their website. Make clouds the fourth column. Even if you believe you'll never use a cloud computing platform (you Luddite, you!) you need to treat a cloud offering as a fourth column when evaluating any IT solution. You'll be better armed, and more likely to discover hidden costs. Alistair Croll is principal analyst with Bitcurrent and conference chair of TechWeb's Cloud Connect 2010 conference.