(10 of 14)
"We just kept saying, 'Let's pick ourselves up and fight,'" Park recalls. "And when the site came back, we pushed ahead nonstop ... We went from doing three or four releases"--upgrades or changes to the website--"in October to 25 in November."
"The team," says Zients, "ran two-minute drills to perfection. We had the best players on the field. Some plays didn't work. We talked about some of those. But there was never any finger pointing. People just hustled right back to the line, and we ran the next play."
Dickerson was so adamant about the need to forgo finger pointing and move on to the next play that during one stand-up in mid-November he demanded a round of applause for an engineer who called out from the back of the room that a brief outage had probably been the result of a mistake he had made.
Zients isn't a techie himself. He's a business executive, one of those people for whom control--achieved by lists, schedules, deadlines and incessant focus on his targeted data points--seems to be everything. He began an interview with me by reading from a script crowning the team's 10-week rescue mission as the White House's "Apollo 13 moment," as if he needed to hype this dramatic success story. And he bristled because a question threatened not to make "the best use of the time" he had allotted. So for him, this Apollo 13 moment must have been frustrating--because in situations like this the guy in the suit is never in control.
True, Zients had assembled a terrific team that had gelled perfectly. But his engineers could move only so fast. Though he had carte blanche to add resources, putting 10 people on a fix that would take one coder 10 days doesn't turn it into a one-day project. Coding doesn't work that way. "Jeff was a great leader, but there were limits," says Dickerson. "He would ask us every day if we were going to make the deadline ... He'd say how he had to report on how we were doing to the President. And I'd say till I was blue in the face, 'We're doing as much as we can as fast as we can, and we're going to do that no matter what the deadline is.'"
One crisis as the November deadline approached gave the team confidence that it could work through anything. Paul Smith, the campaign alumnus Burt had persuaded to join the team just as he was trying to raise money for a startup, had been working on a problem that had stumped everyone so far: the unique identifier that the website had to issue to anyone who was trying to enroll was taking too long to generate. By the afternoon of Nov. 6, the ID generator became so overloaded that the site was effectively down. "This kind of database problem is in basically everything I've ever worked on before," Smith says. "So I worked with the dev team to come up with a patch."
The patch worked in some ways, but the team learned a few days later that the identifications it was generating didn't have the right number of digits to match insurance companies' needs. So it had to be removed, and on Nov. 20 the old ID generator effectively shut the website down again. Smith and the team quickly designed a new patch, this time with the right number of digits, and executed what's called a "hot fix," meaning they put it onto the site almost instantaneously without testing. It worked.