For much of its 100-year history, spanning two world wars, the Cold War and the fight against terrorism, Britain's MI5 domestic intelligence agency has operated in total secrecy. The government acknowledged the existence of the service only in 1989 and publicly identified its leaders in 1992. Now, as part of efforts to make its operations more transparent, MI5 has given unprecedented access to its files to Britain's foremost intelligence scholar, Christopher Andrew, whose new book, The Defense of the Realm, is considered the most complete history of the agency ever published. TIME spoke with Andrew about the conspiracy theories he's debunked, former spies in the British government and his feelings about James Bond.
Were you given full access to MI5's files up to the present day?
I was given virtually unrestricted access. However, I did not ask for access
to 21st century files of cases that haven't yet come to court or that are
still under investigation.
What was the biggest surprise you uncovered?
One discovery made a real impression on me. The highest calling of any
intelligence agency is to tell truth to power. The first example of this I
came across is during the 1938 Munich crisis, one of the most ignoble
moments in British foreign policy. MI5 at that point actually understands
[Adolf] Hitler far better than the British government and then Prime
Minister Neville Chamberlain, in particular. He pays no attention to what
they tell him during the negotiations that lead to the Munich agreements. So
Vernon Kell [the head of MI5 at the time] decides the only way he can get
through to the Prime Minister is to tell him what Hitler is saying about
him, specifically that he calls him as an arsehole. He gives him the German
word that Hitler is said to use: Arschloch.
Do you think that transparency helps intelligence agencies in the long run?
Yes. The idea used to be that you don't want the public to know anything, so
you don't tell them anything. What changed a generation ago is that the
British people became less deferential, and if they're not given some idea
of what's going on, they fall for conspiracy theorists. The best-selling
book in the U.S. about British intelligence is, after all, Peter Wright's
Spycatcher. A couple of the stories that he put in there that are complete nonsense are still widely believed: that there was a plot against former Prime Minister Harold Wilson I now know there wasn't, though there was a file kept on him and that the head of MI5 for nine years, Sir Roger Hollis, was actually a Soviet agent. The files disprove this too. If you don't have information, you get disinformation.
Looking through 100 years of files, what do you see as MI5's biggest
success?
Definitely against Germany during the Second World War. Some of the best
German spies were turned into double agents, and that is the central part of
the most successful deception in the history of warfare, without which the
D-day landing couldn't have taken place. What I discovered in the files is
how it all began in a characteristically eccentric British way. The most
adventurous of the MI5 agents in the 1930s was an air ace from the First
World War named Christopher Draper. He's called the "Mad Major," because he
was absolutely obsessed with flying under London's bridges. He's invited
over to Germany. Hitler is very interested and spends over half an hour
talking to him at an air show. When he gets back [to London], he's asked to
become a spy for German intelligence, and he says fine, stopping only to ask
MI5 if that's O.K. on the way! By monitoring the contacts he has to make with
German intelligence, MI5 picks up a German agent they didn't know about and
begin to roll up the entire German network.
So MI5 has had its share of characters?
So far as I know, MI5 is the first government agency to actually identify a
sense of humor as a desirable trait in a recruit. When you look at the world
of espionage, it's the only profession in which a fictional character is at
least a hundred times better known than any real character. James Bond is
the only brand leader to have remained in the top spot over the years. An
intelligence officer once told me that in the depths of the Third World he
met a tribal leader whom he thought knew no English but who addressed him
with the famous phrase "Hello, Mr. Bond."
What about MI5's biggest failures?
Those are related to the biggest failures of Western intelligence generally.
When MI5 begins to get involved in Ireland in the early 1970s at that point
they knew less about Belfast than they did about Nairobi well, I haven't
come across a single file that relates intelligence during the Troubles that
begin in 1969 to intelligence between the Easter rising in 1916 and the
founding of an [Irish] Free State in 1922. Files from that previous period
show that intelligence was incredibly confused, and poorly coordinated with
local police. What happens in the 1970s? Exactly the same thing. It's a
nearly perfect example of the old adage that those who do not understand
past mistakes are doomed to repeat them.
Taking a long-term view might also have helped better predict Islamist terrorism in the 21st century. MI5 was slow to realize the potential threat, as it was busy dealing with IRA campaigns in the 1990s. It's not until 2003 that MI5 grasps that the radicalization of minority British Muslims is leading to terrorist attacks on British targets.
You discovered that three Labour Members of Parliament under Wilson were
considered Soviet agents. Did that come as a shock to you?
It did come as a bit of a surprise. What surprised me most, however, was the
degree to which getting too excited by the threat of communist subversion
was not usually done by MI5 but rather by government. Labour leadership sent
MI5 a list of names of 16 Labour MPs who they thought were more communist
than Labour. MI5 refused to get involved because it saw this as party
politics. However, the man at the top of that list [Will Owen] we now know had been a Czech agent for years!