The lawsuit, filed in Delaware federal court, seeks a declaration that Red Hat is not violating SCO's intellectual property and an injunction that would bar SCO from making untrue claims that harm the Linux business.
"We're seeking a resolution ... to all the rhetoric as fast as possible," said Matthew Szulik, Red Hat's chief executive officer.
SCO, which owns key parts of the Unix operating system, claims its code has seeped into Linux. It is seeking $3 billion from International Business Machines Corp. for allegedly transferring Unix code to Linux. SCO has also sent warning letters to hundreds of other companies.
So far, Red Hat has not been sued. With Monday's lawsuit, however, it has taken the offensive against Lindon, Utah-based SCO.
SCO did not immediately return phone calls seeking comment.
A cloud of legal uncertainty has enveloped Linux since March, when SCO filed its breach-of-contract lawsuit against IBM. Though the suit itself relates to how IBM distributed its version of Unix, SCO has raised serious questions about Linux in its warning letters and in interviews.
The case also has cast a shadow over the LinuxWorld trade show, which is taking place this week in San Francisco, though legal issues aren't on the formal program.
SCO's claims raise questions about the essence of the open-source movement that's evolved on the notions that software code should be fully exposed and freely distributed rather than secret and proprietary, as Microsoft's programs chiefly are.
"I think that's why this is escalating into such an epic battle here," Darl McBride, SCO's chief executive, said last week. "We're not going to give in, and if they do give in, then they've got big problems. I guess that's what creates great big-time battles."
Linux is distributed under the GNU General Public License, which leaves users--not distributors--liable for any intellectual property issues that might arise. McBride often refers to this as the "hot potato" of Linux.
George Weise, a Gartner Inc. analyst, thinks SCO's claims should not be ignored, even if they seem baseless.
"To me that's akin to a home user getting letter from a legal firm with some claim against you and you throwing the letter in the trash," Weise said.
On Monday, Red Hat also announced that it was establishing a legal defense fund for companies and organizations that are developing Linux. Red Hat contributed $1 million to the fund.
But Red Hat is not offering help to customers. They're protected by the transparency of open-source code, Szulik said. SCO has yet to fully disclose the code that allegedly infringes on its intellectual property.
"The question is what are they being held liable to? That is the very essence of our complaint," Szulik said. "Today, it's innuendo and rumor. There is no basis that we have been able to determine for the claims that are being made."
Linux defenders also question SCO's past life as a Linux distributor. SCO pulled its Caldera distribution of Linux only after it started publicizing its infringement claims.