The Lost Moderates: A TIME Roundtable

November's electoral earthquake felled some of Congress' sturdiest pillars, legislators who over the course of decades on the Hill reformed welfare, opened the Internet for commerce and balanced the federal budget. In some cases, the characteristic that made them successful is what made them vulnerable at the polls this year: their willingness to compromise with political opponents.

Four of these moderates sat down recently for a special roundtable conversation with TIME's Michael Grunwald. Democratic Congressmen Rick Boucher of Virginia served 28 years and wrote the 1991 bill opening the Internet to commerce; South Carolina Democrat John Spratt, also a 14-termer, led the push to pass the Balanced Budget Act of 1997; Democrat Paul Kanjorski of Pennsylvania served 26 years and was a key player in Congress' response to the financial meltdown in 2008 and the passage of this year's financial regulatory reform bill; and Republican Michael Castle of Delaware served 18 years in Congress and earned the federal government $8.6 billion through the mint's State quarters program.

The retiring Congressmen tap their experience to answer one final question: why they lost.

TIME: What went wrong? How come you're not coming back?

Mike Castle: I was a victim of the Tea Party. They came in and, frankly, did a good job politically. I didn't agree with a lot of what was said and to this day, I disagree with it. The candidate they nominated [Christine O'Donnell] could not hold up in a general election and got clobbered which is something the Tea Party needs to pay attention to. But they took me out in the primary and most people didn't think it could happen. I can't tell you how many letters, e-mails and personal conversations I've had with people saying, 'I had no idea you were in trouble or I would've been out there to vote for you in the primary.' And they just did not do that, unfortunately. And so we were unable to overcome the energy of the Tea Party movement.

TIME: The people have spoken?

Spratt: It was the economy and a short, one-word answer, that was the problem. I represent a part of 14 counties and in those counties, double-digit employment has reigned for over the last year and a half. And that would have made my reelection an upstream swim under any circumstance. But when you add the countercyclical economic policies we put into place that people didn't fully appreciate or agree with, that only added fuel to people's concerns. Namely the tarp money, the Recovery Act, the bailout of GM and Chryslerpeople didn't like that. It made people very wary of what we were doing.

Boucher: In my particular election about $2.5 million dollars, which goes a long way in a very inexpensive rural television market, was spent on television ads linking me to the Democratic leadership. And I've been able to previously win in a district that's Republican because of the focus on the local economic development that I've done. That has sustained me through this timebut in this particular year, the focus was quite different. It was fairly easy to link each of us to what were seen as unpopular measures. And the attack ads proved to be insurmountable and people overlooked my record of service The voters really did not know who was funding these ads. The organizations all have very nice names like Americans for Jobs Security, but their funding sources remain secret and under current law, they're not required to disclose them.

Castle: The media has gotten very involved in this. The media has taken very strong political positions maybe not in favor of a particular person running for office, but in terms of their ideological drifts, it makes a huge difference. [The media] tends to be either very conservative or very progressive. It has limited the ability of individuals like us to sit down and work out common solutions that are in the best interests of the country because our parties will not like that.

TIME: Is Congress capable of dealing with the big problems of the day?

Kanjorski: The thought that even as we speak today, probably in 50 congressional districts across this country, there are candidates beginning to form up committees to get ready to run two years from now is just frighteningwe are going to have wall-to-wall campaigns because of ambition. I'm not sure that ambition alone is a good justification for public office.

Boucher: It's just far more difficult for members of the House to reach across the aisle the way we used to. When many of us started, many large issues were resolved on a bipartisan basis. There were far more moderates both in the Republican ranks and the Democratic ranks in those days than there are today.

Spratt: We need to get back to the serious business of legislating. Because the campaign is always just over the horizon, too much of the debate on the floor today is to try to get your opponent or adversary on the House floor in a corner where you can say 'gotcha!' It's not to make a point, it's not to explain legislation, it's not to be pro or con, it's simply to say 'gotcha!'

TIME: Do any of you have regrets?

Spratt: I think we could've passed and packaged our major proposals in the last Congress in a more effective way. Healthcare for example: we should've broken it down in pieces and passed it step by step instead of trying to do a mammoth bill; same thing with the recovery act. $288 billion of the $787 billion in the Recovery Act was due to tax cuts, very few people were even aware of that. Castle: I would hope that the Republicans who are now in charge of the House would make legislation simpler.

Kanjorski: Twenty-six years ago, I think of myself as having been a sparrow. And then through the election process, I was given the opportunity to play an eagle. And I soared in the highest echelons of power in this country by the trust and faith of my constituents. And now I return to being a sparrow, and only in America can that happen.