Computer Glitch Strands Thousands At LAX; What Went Wrong

The failure of computer networking hardware over the weekend at Los Angeles International Airport took U.S. Customs and Border Protection (CBP) computers offline, preventing about 17,000 travelers in the airport from being processed by immigration authorities.

The network outage delayed passengers at Tom Bradley International Terminal and on the tarmac for up to nine hours. The incident began about 2 p.m. on Saturday and was resolved by 11:45 p.m. By 3:30 a.m. Sunday, the last of the affected passengers exited the federal inspection area, according to CBP press officer Michael Flemming.

Flemming said there were also outages at LAX terminals 2, 4, 5, and 7, but those were resolved more quickly.

"The result was it wiped out our local area network, which in turn that wiped out access to our computer system which contained sensitive law enforcement data about international travelers," Flemming said. "We couldn't manually inspect the passengers without our data systems because that would have been a significant breach of security in terms of threats of terrorism."

Saturday, Aug. 11

When the local area network used by CBP computers failed on Saturday, CBP officials expected to resolve the problem quickly. That didn't happen, and CBP ended up calling Sprint Nextel, its service provider, to report a pair of failed routers about 1:30 p.m.

"All of our tests indicated that the routers were clean," said Sprint spokeswoman Kathleen Dunleavy.

With remote diagnostics showing no problems after three hours and mounting delays at the airport, Sprint sent technicians to the site. Around 8:00 p.m., it became clear that the problem was not with the network connection but with CBP's local area network. "We sent a technician out, and then a senior engineer got involved and helped customs identify a problem with their LAN," said Dunleavy, who stressed that the failed hardware was not Sprint equipment.

Los Angeles International Airport is currently undergoing renovation work. Flemming said that he could not confirm whether construction dust, vibration, and heat contributed to the CBP hardware failure but noted that "there has been an increase in those factors."

Sunday, Aug. 12

The schedule has been back to normal since Sunday, said Marshall Lowe, a spokesman for Los Angeles World Airports.

Late Sunday and early Monday, CBP computers experienced another systems outage, but this time the problem was resolved within 45 minutes, according to Flemming, who attributed it to a failed power supply.

Steve Lott, head of communications for the North America for the International Air Transport Association, expressed concern that the CBP didn't raise the level of urgency quickly enough.

"What really frustrates us is that CBP had no contingency plan," said Lott. "If this can happen at LAX, it raises questions about whether it could happen at other airports."

"We regret what happened but we could not risk letting in one person that could have been a terrorist," Flemming explained. "The conduct of the passengers that endured these delays was very much commendable."

Lott expressed concern that CBP might not have had adequate IT staff on site. "The government leans on the airlines to provide better customer service," he said. "We need to hold the government to same service standards as private industry."

Flemming could not immediately confirm how many IT personnel were on site at the time of the incident or provide further detail about the specifics of the CBP hardware failure. "Since the incident, we are making sure IT staff are there all the times instead of on-call," he said. "We're making changes to staffing, equipment, and procedure regarding this incident. It's just unacceptable to everyone to have a repeat of this problem."