In Search Of Open-Source Experts

As companies ramp up open-source deployments, expertise is in high demand and short supply. The costs of developing talent can add up.



Applied Industrial Technologies Inc., a distributor of fluid-power and engineered products, runs most of its infrastructure on Microsoft Windows. But two years ago Applied Industrial began running some Web, file, application, and directory servers on the open-source operating system Red Hat Linux. "Our technicians [were] saying, 'we need to have Linux servers in place--we could pull down costs,'" says IT director Bob Falkowski. "We changed out hardware, introduced new software components. But what we failed to realize is, when you do this type of process there's some added burden. You have to fall back on yourself as being the ultimate solution provider when things don't work."

Applied Industrial Technologies isn't the only company to be caught short by the effort and expertise required to support an increasing number of open-source projects. Companies often look at the bright side of deploying Linux and other open-source systems--the cost savings, the standardization, the freedom from vendor lock-in--but aren't well-enough prepared for the challenges that come with implementing or expanding the use of technology that's still in the early stages of development.

It was no surprise when Microsoft CEO Steve Ballmer issued a memo last month asserting that Windows computing environments offer, among other advantages, a lower cost of ownership than Linux and open-source systems. What may be surprising is the correlation between Ballmer's claims and many companies' real-world experiences. Not only did Ballmer point to independent analysts' reports to bolster his case, but there's growing evidence that many companies are running into unexpected costs, especially in connection with implementation and support, when trying out or switching to open-source systems. Even the costs of signing on with an IT vendor to support open-source deployments may cut into their estimated cost savings.

"It was a learning experience for us," admits Applied Industrial's Falkowski. The company now is "going through the painful process of developing the know-how," he says, with a big focus on retraining staff. Applied Industrial won't disclose how much it has invested in efforts to ramp up open-source expertise, but it has negotiated Linux training from IBM as part of a contract to replace its Tomcat open-source Java servlet and JavaServer Pages with IBM WebSphere. "We looked at the positive aspect too heavily and didn't look at the negative aspects enough."

One of the chief complaints of CIOs and CEOs is that they can't find enough qualified open-source programmers, says Faber Fedor, an open-source consultant with services firm Linux NJ.com Inc. "I don't think there are enough experienced people out there for the demand," he says. Yankee Group analyst Laura DiDio agrees. "There's a dearth of skilled Linux administrators, by comparison to the more-mature Windows, Unix, NetWare, and Macintosh environments," she says. And what happens when too much demand meets too little supply? "They can command a premium," DiDio says. "They get a 20% to 30% salary premium in the large metropolitan markets."

The shortage of open-source talent that Linux adoption is driving at the enterprise level will even out over time as colleges ramp up open-source programs and today's graduates become tomorrow's Linux experts, says Robert Jones, president of HotLinuxJobs, a division of Glacier Technology Services Inc. For now, there's a direct connection between how urgent a company's need for Linux talent is and how much it's willing to pay.

Some early adopters of open-source software in the data center recognized there would be no ready pool of talent to help with the effort but were lucky enough to have Unix pros on staff whose experience on that formerly open-system operating system translated to Linux with comparative ease.

"A Unix specialist and a Linux specialist are really cut from the same cloth," says Mike Prince, CIO at Burlington Coat Factory Warehouse Corp., which has been using Linux and the Apache open-source Web server since 1999. Two years ago, Burlington Coat rolled out point-of-sale systems running Red Hat and back-end data-center equipment running SuSE Linux. The company's large Oracle databases, now running on Linux, had to be switched over from Unix. The company had 15 seasoned Unix specialists on hand when it began the migration, which helped provide needed data-center experience.

It's a different story for companies that are primarily Microsoft shops. A total switch from Windows to Linux, or even a significant Linux deployment in the middle of a Windows environment, will be three to four times more expensive--and take three times as long to deploy--as an upgrade from one version of Windows to a newer release, according to a Yankee Group study Ballmer cited in his memo. He also cited a May Forrester Research study that said Linux training is on average 15% more expensive than Windows training. The author of that report, Forrester VP Julie Giera, says the caveat there is that her report's sample was very small, mainly because she couldn't find many companies who'd been running Linux more than a year and who closely tracked costs associated with the deployments.


Stephenson (left) and Norris have gotten open-source help for Columbia, Mo., Public Schools via Web queries and searches.

Stephenson (left) and Norris have gotten open-source help for Columbia, Mo., Public Schools via Web queries and searches.

Photo by Austin Walsh
Some companies moving forward with open-source projects are willing to take a chance on the next generation of programmers whose familiarity with open-source systems--thanks in large part to computer-science programs that have taken to Linux because of its low startup costs--may make up for their lack of professional experience. Gloria Stephenson, assistant director of network services at Columbia Public Schools in Missouri, took that chance four years ago, when she turned to Jeremy Norris, her young telecommunication and security specialist, for help replacing the school district's aging Cisco Systems firewall appliance. Just a year out of high school, Norris had been a student in a network-administration class Stephenson taught where he demonstrated proficiency working with open-source software, Stephenson says. He suggested replacing the Cisco appliance with IP Filter, a free open-source firewall program that runs on the FreeBSD operating system.

Even when the firewall implementation ran into problems--data packets started crashing as they passed through the filter--Norris was able to work around them until he could query the IP Filter mailing list for help. A day later, IP Filter administrator Darren Reed responded to Norris' query, stating that a fix was already in the works. "I had a feeling I could get it working," Norris says. "I've been on his mailing list for a while, and I'd seen other people's questions answered."



Columbia gets support for its Web server running on SuSE Linux from Novell, but the only way to troubleshoot other open-source programs is to do what Norris did: search the Web and post queries to the open-source community. Columbia's IT staff proactively engages the open-source community to get the latest patches and upgrades, but "this can be time consuming," Stephenson admits.

Most companies aren't going to rely on Web searches and hacker hangouts to recruit open-source talent. That's why business-class support may be best derived from vendors with open-source expertise, says Dan Frye, director of IBM's Linux Technology Center. "You can't put applications in an enterprise environment without the right service and support," Frye says. IBM employs about 1,000 IT professionals trained not only to use open-source software but also to support IBM's Linux customers and contribute code back to the open-source community. Hewlett-Packard, another Linux advocate, claims to employ more than 6,500 workers within HP services who have Linux expertise. Novell has about 700 support personnel who can provide customers with Linux systems support.

The cost of Linux support varies depending on what's required and the way support contracts are structured. Some users bundle Linux support within broader data-center contracts--with HP, IBM, or Novell that can climb well into six figures. HP says that 24-by-7 software support for Red Hat Enterprise Linux is priced at $1,332 per server per year, for example. Red Hat itself charges a $2,500 annual per-license fee for Red Hat Enterprise Linux AS on x86 servers that includes unlimited phone calls and Web access to service reps. The cost jumps to $18,000 for Red Hat on a mainframe.

Having access to support and services from such companies can ease the transition to open systems, vendors say. "Three years ago, you had to be a developer to deploy Linux," says Brian Stevens, VP of operating system development at Red Hat. "Today, you don't." But even with its in-house expertise, Burlington turns to IBM, Novell, or Red Hat for support with Linux-specific issues, CIO Prince says.

Financial-services provider KeyCorp has looked to HP and Red Hat to train its 1,400 IT employees on open-source technology rather than hire new programmers. About half of the financial-services company's IT operations use distributed applications written with IBM WebSphere tools and the Perl programming language as well as the Java 2 Enterprise Edition platform, Lotus Domino, and Microsoft .Net. KeyCorp still runs its core Cobol-based banking applications on mainframes and a DB2 relational database.


Successful Linux implementations drive demand for more cross-training of KeyCorp's Unix staff, says Bob Dutile, executive VP of Key Technology Services. Photo by Bob Stefko

Successful Linux implementations drive demand for more cross-training of KeyCorp's Unix staff, says Bob Dutile, executive VP of Key Technology Services.

Photo by Bob Stefko
KeyCorp spent $130,000 to get its Linux strategy off the ground during the fourth quarter of last year and the first of this year. Only about $10,000 to $15,000 was needed for initial training classes for staff Unix engineers. The rest went to Red Hat for helping the company do a total-cost-of-ownership study and develop a good open-source migration strategy. "Since then, this has very much been a bootstrap operation, with each successful five to 10 implementations driving demand for additional cross-training of our Unix staff and that additional cross-training enabling us to implement even more systems," says Bob Dutile, executive VP of Key Technology Services, KeyCorp's technology division. KeyCorp plans to migrate file, print, and document-processing apps this year to Red Hat Linux running on HP Intel-based ProLiant servers. Dutile expects that by the end of the year KeyCorp will have as many as 55 servers running Linux. "The specific training costs since the first quarter have been very low and have all been absorbed within the normal departmental operating budgets," he says.

While Dutile is reluctant to hire programmers right out of school--"I don't know that I've ever found a developer out of college who was ready for the market"--he does have a trick for finding good talent. He sometimes searches Google and SourceForge.net, a repos- itory for open-source code that's maintained by VA Software Corp., to see which programmers are working on projects that demonstrate skills valuable to KeyCorp. This lets Dutile take a look at a programmer's work. "If I was hiring a programmer from another bank, it's not likely that bank would let me go and evaluate the code that person worked on for them," Dutile says.

SourceForge has more than 900,000 registered users and includes a repository of more than 89,500 open-source programs under development. Although VA Software won't divulge the specific demographics of its user base, the company confirms that more than 40,000 companies, government agencies, and other organizations have two or more users registered with SourceForge. A percentage of those other registered users may be high-school kids or hobbyists, not potentially employable talent.

Open-source expertise is building. Young programmers such as Columbia Public Schools' Norris find it hard to envision a world without open-source software. He's considering going to school to study computer science. One of the main reasons? "It would let me contribute more back to the open-source community," he says.

And companies can contribute to the growing pool of open-source talent. Applied Industrial positioned its open-source training as a career-enhancing step. "We offered people here a way to reinvent themselves," says IT director Falkowski. But considering that Windows servers outnumber Linux servers in his company by a ratio of 4-to-1, it's understandable that not everyone was eager to leave a well-established environment for the less-established world of open-source. Most of those who are retraining, Falkowski says, have backgrounds in Unix and C or C++ programming skills.

The open-source world, like any emerging environment, is fraught with peril. There's the threat of lawsuits--spurious or not--by vendors claiming open-source software infringes on their intellectual property. But potential users of open-source systems also have to consider that there might not be enough expertise to support their projects adequately and that getting that expertise can be more expensive--and more of an effort--than they anticipate. Is open source worth it? Steve Ballmer has his opinion, but most companies are going to have to answer that question for themselves.

--With John Foley

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Email This  | 
Print  | 
RSS
More Insights
Copyright © 2021 UBM Electronics, A UBM company, All rights reserved. Privacy Policy | Terms of Service