Cloud // Infrastructure as a Service
05:31 PM
Connect Directly
Repost This

Obama's Developer Brain Trust: Inside The Big Battle

Tech leaders for the Obama for America team discuss their re-election project -- and a business-like culture clash between free-wheeling nerds and their goal-oriented bosses.

Kunesh said the sheer scale of what they were trying to do was daunting. When it came time to test how well an application would interact with voters who might respond to an email query, he was amazed to learn the sample would consist of 400,000 actual users. The whole list would have meant a mailing to 20 million.

As the team entered October, everyone agreed to a feature freeze and hardening of systems. On Oct. 21, Harper Reed, the technology chief who had hired the five, announced a dry run of key voter tracking and get-out-the vote election day applications and provided them with a script of how they would be tested. When the morning for the test arrived, Reed overrode the script and started firing unexpected challenges at them. At the same time, Amazon experienced a service outage in U.S. East, and instead of meeting Reed's challenge head on, the team had to divide in two, with one part dealing with the challenge posed by Amazon, the other dealing with Reed.

"Half responded to the real incidents (Amazon). Half responded to fake incidents," said Chris Gansen, technology lead for the Dashboard application. Reed posed dilemmas, such as the database server lost its cache memory function, then actually flushed out the database server cache. The team scrambled under his discerning eye to solve a "fake" problem that had suddenly become real.

At the end of the day, said Gansen, both teams were exhausted. They repeated the drills at the end of the month, then made additional preparations as Hurricane Sandy approached the East Coast in advance of the election. Believing Amazon's U.S. East data center complex was at risk, the team replicated all its systems to Amazon's U.S. West in northern California before Sandy struck.

The team had implemented a service-oriented architecture where many applications relied on services from a backend platform, called Narwhal. As word leaked out of the tech team's work, the technologists in the Mitt Romney camp announced they also had an Election Day system, called Orca. The Orca killer whale is the only known predator of narwhals.

At that point, the tech team was beginning to have faith in the reliability of the system they had built and it was not concerned by the reports. Its Narwhal platform could bring all the different activities that a volunteer or organizer had been involved in together to give a headquarters organizer a clearer idea of who they were dealing with and what they should do next.

As election day dawned, Narwhal worked as expected, while reports from the Romney camp indicated that Orca either failed to function or tied up communications lines by sending out 80-page PDFs to seniors who had volunteered to help turn out the vote. Nine days after the election, Ars Technica ran an insider piece on how the two respective systems performed, "Built to win: Deep inside Obama's campaign tech."

"Great relationships" developed across technology teams and in some cases, between the technologists and the politicos. "Towards the end, we got to where we needed to have people working cross-discipline on different functions," said Kunesh.

Reed "made us go for simplicity. The older users had iPads. The younger users had iPhones, and that was it," recalled Gansen. The technology team used HTML to straightforwardly gear its applications to mobile users needing simple communications and information fast.

"We've just established the new normal for what a political campaign will have to do," said Gansen. But he doubted the code created would survive as the backbone of a system for the next election. The next team should throw it out and start over, with conditions as it finds them at the time, he warned.

Several team members questioned why the Romney technology team didn't do better. "It's a little strange to me the party of business didn't do better… This isn't going to happen again," Gansen said.

Despite their reputation as the young guns of political technology, none of the five team members announced any clear plans of what he was going to do next.

"I have no idea," said the long-haired Kolak, perhaps the most noted member of the team as the mastermind of Narwhal. "We're in San Francisco. Is there any work out here?"

His audience of 300, heavily salted with young programmers from Google, Twitter, Zynga and Facebook, laughed appreciatively and broke into sustained applause.

2 of 2
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
Andrew Binstock
Andrew Binstock,
User Rank: Author
1/16/2013 | 2:24:06 AM
re: Obama's Developer Brain Trust: Inside The Big Battle
But enormously expensive. They said hosting on AWS was their biggest cost by far.
User Rank: Apprentice
1/15/2013 | 5:22:53 PM
re: Obama's Developer Brain Trust: Inside The Big Battle
That's a pretty cool cloud use right there.
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.