Linux has grown rapidly over the last several years, moving from Web operations on the perimeter of the enterprise to workloads much closer to the heart of the business. In the process, it's gained a broader following of contributors and commercial users.
At the same time, the easy part of Linux's advance may be over. IDC analyst Al Gillen told about 300 attendees at the Linux Foundation Collaboration Summit in Austin, Texas, Tuesday that Linux has made many of its gains at the expense of legacy Unix systems. From here on, its growth may slow, as result of both server virtualization, which packs more applications on a single server, and head-to-head competition with Microsoft's Windows Server. "Never discount Microsoft," added Gillen.
Sun Microsystems stemmed Solaris users' erosion to Linux when it made its operating system open source code, said Gillen. "Solaris is not necessarily a threat, but it's going to be a competitor," said Gillen, in his keynote address.
Linux is more and more frequently acting as a database server, growth "that's heavily driven by Oracle," Gillen said. At the same time, Linux is assuming more business application workloads, including enterprise resource planning, customer-relationship management, and financial applications. "By 2011, the logistics and manufacturing applications alone will be a $1.2 billion market on Linux; human capital management will be a $2 billion market," he predicted.
Gillen has been tracking Linux since the first IDC Linux marketplace study in 1999. By 2011, the market for Linux systems -- server hardware plus operating systems -- will amount to $50 billion. Windows will still lead Linux by a wide margin. A way of comparing the two, he said, is Linux and Unix business applications together will amount to $96 billion in 2011 (with Linux applications representing one-third of the total); Windows Server applications, $110 billion.
But Gillen cited additional figures that show for every supported copy of Linux running in the enterprise, there is another copy running unsupported and unpaid for. The Linux ecosystem is twice as large as it appears in IDC revenue data because so many users feel they have the skills in-house to support their Linux system or are willing to rely on support from Linux community members.
Gillen's analysis was part of the second annual meeting on the status of Linux sponsored by the foundation. It is a nonprofit group headquartered in San Francisco, whose corporate sponsors include Oracle, HP, IBM, and its most recent recruit, Adobe Systems.
Part of the purpose of the summit is to let business users see and interact with Linux kernel developers, who are sometimes distant, god-like figures with absolute power over the operating system's core. One person glad to have the opportunity was Ed Reaves, a Nortel technology platform manager from Research Triangle Park, N.C.
Linux end users and server administrators are happy with 99.999% of uptime, Reaves noted, but Nortel and other telecommunications companies would like to move Linux reliability to 99.9999%, or one outage of about 30 seconds per year. Nortel's Linux developers produced a block of code that restarts Linux in 20 seconds in the event of a stalled operating system, called "the restart instead of halt patch," he noted.
But it doesn't appear to be moving into the kernel, to the dismay of Nortel executives concerned about avoiding outages. "How do you get a kernel patch released into the mainline?" Reaves asked, referring to the development process that steers additions to the kernel past reviewers and into a hierarchical code tree maintained by Linus Torvalds.
Kernel development is a many-segmented process, with key developers responsible for particular Linux subsystems. "The limiting resource is not development of code but review of code," responded Jonathan Corbet, a kernel developer and author of the Linux Foundation's "Linux Forecast" published on its Web site.
The smaller the patch, the more likely it is to find a speedy review, he added. The Nortel patch, however, was a sizable block of code requiring someone with knowledge of a particular part of the kernel.
James Bottomley, the kernel developer responsible for the SCSI subsystem, added, "It's necessary to find the right kind of review." The reviewer needs to have expertise in the area of the submitted code, time to review, and the respect and trust of the kernel maintainers in his subsystem area. The reviewing process is less recognized than the code writing, which has prompted kernel developers to include a "reviewed by" tag with patches to acknowledge the role of the reviewer.
"The bar is just high," conceded Arjen van der Ven, a kernel developer employed by Intel who works on Linux power management.
Another audience member cited the risk of finding 20 "flame mails telling you the inadequacy of your code" when you come to work the day after submitting a patch. Ted Ts'o, an unusually even tempered developer hired in December by the foundation for full-time kernel work, conceded that receiving intimidating e-mail was one risk of being a submitter. The kernel developers are trying not to feel so hard-pressed that they respond brusquely to contributed code, he said.
"I would rather that you be flamed by me than by Arjen over there," Ts'o added to a round of chuckles in the audience.
Another problem area between kernel developers and Linux users is troubleshooting bugs. The kernel developers want the broader community to report bugs and submit recommended fixes. "How many of you know how you report a bug?" Bottomley asked the audience, which included many Linux users. Only a few parties raised their hands.
Go to http://bugzilla.kernel.org, he instructed, and use the Web site's form.
Bug reporting is a priority of the kernel developers. Responding to prompting by lead kernel integrator Linus Torvalds 3 years ago, they began worrying less about catching each bug and more about incorporating changes in each iteration of the kernel. Let the larger community help detect and correct bugs, was the thinking at the time, several developers said.
The practice ran into a response last year from Dave Jones at Red Hat, who said he feared the number of bugs in the 2.6.21 kernel was going to lead to problems as it became part of Red Hat's community Linux, Fedora. "I was incredibly nervous" about shipping Fedora 7, he recounted yesterday, and in a blog on June 7 last year he acknowledged the reason he was nervous: "Looking at the first round of bugs that came in during the week after F7's release, it's pretty horrific."
Jones, Bottomley, and Corbet yesterday said the kernel developers are trying to catch more bugs in the code-building process prior to kernel releases. Kernel releases have slowed from a frenetic every two months to every 2.7 months, according to a recent report from the Linux Foundation by kernel developer Greg Kroah-Hartman and foundation marketing director Amanda McPherson.
"A lot of people don't report bugs [when they encounter them]. That's a problem," scolded Arjen.
Bottomley replied, "Finding bugs is a partnership. If you're an early tester and you find them, report them and we'll fix them."
Added Ts'o, "Even if there is only a one in a thousand chance they will hit, that's unacceptable to an enterprise." Linux kernel releases are subject to testing by Red Hat and Novell before going into an enterprise edition, with no or few changes to the kernel once an enterprise edition is launched.
Another area of ongoing concern is building drivers for peripherals that users want to work with Linux. Kroah-Hartman announced a year ago that he had 200 volunteers who would write drivers for manufacturers who cooperated with the effort by providing specifications and other support. This year there are 300 volunteers, but "there's not much for them to do."
Manufacturers sometimes publish specifications for their devices on which a drive might be based, but they don't tell Linux kernel programmers where they have deviated from their own specifications. Without manufacturer cooperation, reliable drivers can't be created, said Chris Wright, a kernel maintainer working on security. Manufacturers not used to the open nature of Linux kernel development are sometimes reluctant to divulge what they consider trade secrets.
The first day of the summit ended with energetic debate among mobile device makers who use Linux. Sean Moss-Pultz of OpenMoko, Derek Speed of Intel's Moblin project, Andrew Shikiar of the LiMo project, a consortium of Vodafone, Motorola, NTT DoCoMo, and other mobile vendors to develop a common, open mobile platform, David Schlesinger of Access, and Eric Chu of Google debated who was following standards and how mobile Linux devices should be developed.
"There was an amazing amount of contention. I love to see the passion," said Linux user Stefano De Panfilis, laboratory director at Engineering Informatica in Rome. Don't worry, said veteran Linux observer Michael Schultheiss, Unix systems specialist at Indiana University in Indianapolis and president of the local Linux user group. "The marketplace will sort it all out."
The gathering took place in The Commons meeting hall on the J. J. Pickle Research Campus of the University of Texas. IBM held its first company-wide strategic meeting on Linux in secret in the hall in 1999. Many of its Linux developers are based at IBM offices in Austin.