Luigi Auriemma, noted among other things for discovering a vulnerability in the Unreal Engine in 2004, on Thursday posted details about producing a buffer overflow error in QuickTime. Buffer overflows can often be exploited by attackers to compromise the affected system.
"The bug is a buffer-overflow and the return address can be fully overwritten so a malicious attacker could use it for executing malicious code on the victim," Auriemma said in an e-mail.
According to Auriemma, the vulnerability affects both Windows and Mac OS X versions of Apple's QuickTime software. But other researchers have been unable to successfully use the exploit on Mac OS X and have suggested that the flaw may lie in code specific to Windows.
In his description of the exploit, Auriemma explains that when QuickTime encounters a Real-Time Streaming Protocol (rtsp://) link and port 554 of the server is closed, the application will switch to the HTTP protocol on port 80. The server then sends a long HTTP error message, so long that it causes the buffer to overflow. This allows the attacker to take control the affected system.
Auriemma said that Apple has not been notified of the flaw in advance of its publication.
When Apple updated QuickTime to version 7.3.1 on Dec. 13, it fixed an RTSP buffer overflow bug (CVE-ID: CVE-2007-6166) related to the content-type/content-base header. The vulnerability Auriemma has identified relates to error message handling and remains unpatched.
Alfred Huger, VP of development at Symantec Security Response, said that the exploit appears to be valid. "The proof-of-concept code only managed to crash the product," he said. "But it's a safe assumption that if you can do that you may be able to execute remote code.
"It's very serious," Huger added, noting that it's one of a number of QuickTime vulnerabilities discovered in the past few months.
With the increasing popularity of Mac OS X on both computers and phones, several security researchers have observed that hackers are exploring vulnerabilities in Apple's products with more interest.
On Wednesday, US-CERT warned about a phony iPhone upgrade. And at least one recent malware program, Trojan.DNSChanger, has the potential to affect both Windows and Mac users.
On the Sunbelt Software blog on Monday, security researchers Patrick Jordan and Adam Thomas identified the latest in a series of sites trying to infect visitors with Trojan.DNSChanger by tricking them into installing a purported media codec to enable video viewing.
Huger said that hackers aren't specifically interested in Apple products. Rather, they look for holes in any widely distributed application, like QuickTime, or device to maximize malware distribution.
This article was edited on Jan. 11 to clarify that the vulnerability affects both Windows and Mac OS X versions of Apple's QuickTime software.