3. Official history then and now

Precisely at the point at which Butterfield lost interest in the subject, official history in the United Kingdom came into its own. It is hard now to recollect that until the thirty-year rule came into effect in 1969, official narrative histories and collections of diplomatic documents were virtually the only available sources for the inner workings of government in the first half of the twentieth century. No wonder, therefore, that Namier and Taylor fell upon successive volumes of DBFP and DGFP so avidly: beyond memoirs, biographies and published diaries, they were all they had.¹ No wonder, too, that Taylor had to rely so heavily on the official histories of the two world wars when writing his English History 1914–1945, while challenging the basic premise of official history in terms almost identical to Butterfield’s:

The civil and military histories of the Second World War

Whatever their limitations, the official histories were indispensable. In terms of sheer output, Hancock’s civil and Butler’s military series represented a formidable achievement. In the late 1940s and 1950s alone, more than forty volumes were published, followed by a further twenty volumes in the following decade.³ While some were undistinguished, others, as Taylor tersely acknowledged, were works of lasting quality. They included Richard Titmuss’s Problems of Social Policy (1950), ‘a remarkable essay in creative understanding’; T.K. Derry’s The Campaign in Norway (1952), ‘remarkably frank’; and The Strategic Air Offensive against Germany (1961) by Temperley’s old sparring partner (and Butterfield’s bête-noire) Sir Charles Webster and Noble Frankland (who had served in Bomber Command during the war), ‘the most ruthlessly impartial of all official histories’.⁴

Yet Butterfield would have found much to reinforce his suspicions in the battles fought behind the scenes between authors and editors on the one hand and officialdom on the other. Even Hancock, who had prided himself on his ability to forge a ‘creative partnership’ between historians and officials, became disillusioned when draft copies of British War Economy, circulated in 1947, ‘evoked an instant torrent of official criticism and complaint’.⁵ Hancock had insisted that ‘the histories must be critical. To have told a “success story” – even when the success had been in the end resplendent – would have been futile and dangerous; the main processes of trial and error had to be revealed.’⁶ But such frankness was exactly what civil servants objected to. They did not wish policy to be shown as the product of ‘internecine warfare’ or ‘muddle, improvisation and fortuitous decisions’.⁷ Perhaps worn down by their objections, Hancock accepted most of the changes required.⁸ ‘The result’, Jose Harris writes, ‘was that the published edition of British War Economy was a much greyer, more discreet, more anonymous and less popularly accessible work than the original draft version.’⁹ The obstacles faced by Webster and Frankland were still greater, for their dissection of the strategy underpinning Bomber Command’s campaign of area bombing challenged not only the success of that campaign but the very doctrine upon which the Royal Air Force had been founded in 1918.¹⁰ In the face of opposition from wartime leaders as eminent as Viscount Portal and Sir Arthur Harris, it eventually required the intervention of the Cabinet Secretary, Sir Norman Brook, to secure approval for publication by the Prime Minister, Harold Macmillan.¹¹ After such ordeals it was unlikely that any official historian, even one whose work had achieved publication relatively unscathed, would have felt touched by the subtle ‘influence upon historians of admission to the charmed circle’.¹²

Documents on British Foreign Policy 1919–1939

The publication of Documents on British Foreign Policy 1919–1939 proceeded at a steady pace and with little further controversy. The third series, covering the 1938–9 period, was completed in nine volumes between 1949 and 1955. The fact that they were published so soon after the events they recorded seems remarkable by today’s standards (it is as if we were now publishing documents from the early years of the Cameron administration) and helped to compensate for the prolonged closure of the official archives. The first and second series, together with a new series IA (covering the period 1925–30), inevitably took much longer to complete. Woodward retired as editor in 1958, but DBFP continued under the editorship of Rohan Butler, Patrick Bury, Douglas Dakin, Margaret Lambert¹³ and Norton Medlicott, to be concluded in 1984 in a total of sixty-four volumes.¹⁴ However, the publication of Woodward’s narrative history of British Foreign Policy in the Second World War, completed in 1956, proved far more problematic and deeply frustrating for the ageing author. An abridged edition was published in 1962; the full five volumes did not appear until the early 1970s, by which time the documents to which they referred were already open at the Public Record Office.¹⁵

DBFP did not entirely escape criticism. Writing ten years after the end of the war, Desmond Williams returned to a familiar theme.¹⁶ He acknowledged that later volumes in the third series published some internal Foreign Office minutes but complained that they were mostly confined to the footnotes. This suggested that ‘the editors may have had a “guilty conscience”. Otherwise why should they have reversed their original decision, and then only do so in half-hearted fashion?’ He also complained about the privileged access granted to official historians – who must be ‘people in whom the Foreign Office has special confidence’ – but denied to others.¹⁷ Having got these gripes out of the way, Williams made some serious points. He compared DBFP unfavourably with DGFP because the latter (as he knew well) drew on a much wider range of official German sources, not least the Führer Chancellery, and thus gave a much clearer understanding of German policy. More importantly, and very much in the spirit of Butterfield’s ‘Foreign Office view of history’, he argued that there was ‘a fundamental defect in that the editors seem to have been working within a limited framework of ideas’.¹⁸ It was a ‘framework of the anti-appeasement viewpoint’ in which the documents selected by the editors seemed designed to ensure that the Foreign Office came out well.¹⁹ Because these documents were the only ones available to the reader, it was impossible to discover the full range of options and calculations in the minds of officials and ministers at the time:

Many of Williams’s points were repeated and amplified when Frank Spencer took stock of the state of play in 1962.²¹ He was in agreement over ‘the general superiority of the German over the British collection’, but did not find evidence of ‘tendentious selection’, blaming instead the sheer mass of material with which the editors had to contend, as well as the pressure of time:

Documents on German Foreign Policy 1918–1945

Spencer’s main criticism concerned the fate of Documents on German Foreign Policy 1918–1945. The series had been, he reported, ‘drastically curtailed’.²³ The American, British and French governments had decided not to publish the projected volumes on the Weimar period and to terminate the series not in May 1945 but in December 1941. This simplified a complicated story. By 1960 the captured documents had been returned to the care of the West German Foreign Office in Bonn and in 1961 the original tripartite editorial project had been extended to include German editors. Although the new quadripartite team would publish no further volumes in English translation, German-language volumes would cover the Weimar period in series A and B, while a new series E would cover the period from 1941 to the end of the war. These decisions represented the conclusion of a decade of wrangling between the federal German government and the governments of Britain, France and the United States. Much of the resistance to the return of the documents and the participation of German editors had been led by British historians.²⁴ In December 1953, the Foreign Office’s Historical Advisory Committee, comprising Namier, Medlicott, Woodward and Wheeler-Bennett, came down strongly in support of the British editors’ objections to such a move. Uppermost in their minds was not any suspicion of Nazi sympathies but the fear, once again, that the German Foreign Office might seek to influence access to the documents and their selection for publication, just as had happened with the Grosse Politik in the 1920s.²⁵ ‘Guarantees are waste paper,’ Namier wrote. ‘Once the documents are back in Germany, the German archivists will have the whip hand.’ ‘The present German government’, he went on, ‘is in a spiritual line of descent from the Weimar Republic; and it is its story that they will be keenest to obfuscate.’²⁶ As Donald Watt observed, the episode illustrated ‘the ferocious hostility towards Germany inherent in their generation of senior historians’: a hostility that proved strong enough to delay the return of the archives for a further seven years despite the Foreign Office’s desire to conciliate a government that was now one of Britain’s most important allies.²⁷ The series was completed in 1995 in seventy-five volumes: United States participation in the project ended in 1979 but the British editors, led by Ronald Wheatley from 1960 until his death in 1985, continued to meet their German and French counterparts for some years to come.²⁸ And until the beginning of the twenty-first century, the German section retained a presence in the Foreign Office in the person of one editorial assistant who had been there since the 1950s, along with a few surviving files of long-forgotten queries.

The peacetime official histories

By the mid-1960s all of the civil series of the Second World War official histories had been published, along with the bulk of the military series, and the future of the Cabinet Office Historical Section was in doubt. It was rescued, and the official history programme given a new lease of life, by the then Prime Minister, Harold Wilson. As leader of the opposition, Wilson had been interested in arguments that the fifty-year limit established by the 1958 Public Records Act should be reduced to forty or even thirty years. In a memorandum of July 1965, less than a year after assuming the premiership, Wilson came down strongly in favour of a thirty-year rule and added two further recommendations: first, that the range of official histories should be extended ‘to include selected periods or episodes of peace-time history’ under a new system of management by a bi-partisan committee of Privy Counsellors; second, that, either alternatively or in addition, selected documents on peacetime history from other government departments might be published ‘on the same lines as the Foreign Office series of documents on British Foreign Policy’.²⁹

In 1966 the Head of the Historical Section, W.I. McIndoe, confided to Rohan Butler that the introduction of a new peacetime series, in addition to a thirty-year rule, might have been an oversight, since the proposal ‘had originated as a sop to historians at a time when it was still contemplated that the closed period for public records might be longer than thirty years’.³⁰ But Wilson must have known what he was doing, and it is likely that he was influenced by the enormous success of a civil history published in 1964 that did not originate in the Cabinet Office, Margaret Gowing’s Britain and Atomic Energy, 1939–1945. Drawing on her years of experience of working with Keith Hancock and sponsored by an enlightened employer, the United Kingdom Atomic Energy Authority, Gowing’s book caught the spirit of the times and became a model to be emulated as the new programme of peacetime histories took shape.³¹

Following agreement with the leaders of the Liberal and Conservative opposition, Wilson’s proposals became law in August 1966. Rohan Butler, who had been appointed historical adviser to the Foreign Secretary by Lord Home in 1962, remained wary of any encroachment on the Foreign Office’s traditional freedom to publish documents on British foreign policy, or any suggestion that the new narrative histories might include episodes of post-war diplomatic history. Although he accepted ‘the possibility of a confidential study of British attempts to enter the Common Market’, the Suez crisis was another matter: Butler felt that the proposed committee of Privy Counsellors ‘would be likely to encourage much invidious log-rolling for particular projects (e.g. Suez) among historians, who would doubtless know who were the Privy Counsellors in question’. On the other hand, Butler accepted that other ‘external’ departments should be allowed to publish documents, and he welcomed the proposal of the Commonwealth Relations Office (which would merge with the Colonial Office in 1966 to form the Commonwealth Office) ‘to sponsor a documentary publication of the last phase of British rule in India, in deliberate preference to a narrative history’.³² This was to prove one of the most successful of the post-war official history projects. Launched in 1967 with Nicholas Mansergh as editor-in-chief, The Transfer of Power, 1942–7 was completed in twelve magisterial volumes between 1970 and 1983.³³

Official history had not yet finished with the war. In response to the growing body of unofficial history and memoirs (some highly contentious) devoted to special operations and European resistance in the Second World War, M.R.D. Foot’s SOE in France was published in 1966.³⁴ Following F.W. Winterbotham’s revelations in The Ultra Secret (1974), Sir Harry Hinsley’s British Intelligence in the Second World War appeared in five volumes between 1979 and 1990. The publication of the fifth volume, Strategic Deception, by Sir Michael Howard, had been prevented on security grounds by the then Prime Minister, Margaret Thatcher, and appeared only after she had left office.³⁵ The 1980s saw other missteps. Two further SOE histories were published by Oxford University Press, but the second of these, Charles Cruickshank’s SOE in Scandinavia (1986), incurred scathing judgements on its inaccuracies as well as on ‘the privatization of official history’.³⁶ It may have been for this reason that the SOE sub-series was halted for some years and that, when it was resumed, the task of writing SOE in the Low Countries (2001) was entrusted to the safe hands of M.R.D Foot, followed by the two volumes of Sir Brooks Richards’s Secret Flotillas (2004).

The first round of peacetime histories got off to a strong start. Beginning in 1975 with Sir Norman Chester’s Nationalisation of British Industry, it included a four-volume history of Environmental Planning, 1939–69 by J.B. Cullingworth and G. E. Cherry (1975–81), D.J. Morgan’s five-volume History of Colonial Development (1980) and the first volume of L.S Pressnell’s magisterial External Economic Policy since the War (1986).³⁷ But if the 1950s and 1960s had been the golden age of wartime official history, the first two decades of the twenty-first century saw a second round of peacetime histories reach a new peak in terms of both output and quality.³⁸ Publication had now passed from Her Majesty’s Stationery Office, via a brief interlude with Frank Cass, to Taylor & Francis under the Routledge imprint; and the contract (from 1999) was with a new body, the Whitehall History Publishing Group, comprising the Cabinet Office, the Foreign and Commonwealth Office and the Ministry of Defence. The purpose of the programme was defined in 2005, by what had now become the Histories, Openness and Records Unit of the Cabinet Office, as being ‘to provide authoritative histories in their own right; a reliable secondary source for historians until all the records are available in The National Archives; and a “fund of experience” for future government use’.³⁹ The Unit went on to describe how topics were chosen and commissioned:

What this meant in practice was indicated by the importance and interest of the subjects selected in 2000, when the OH committee and the group of Privy Counsellors last met, as well as the calibre of the historians the programme was able to attract: to give only a few examples, Sir Lawrence Freedman’s The Falklands Campaign (2005); Terry Gourvish’s The Channel Tunnel (2006); and Alan Milward’s history of Britain and European integration, The Rise and Fall of a National Strategy 1945–1963 (2002).⁴⁰

Beneath the surface, however, all was not well. In part this had to do with the impact of the austerity policies introduced after the economic crisis of 2008; but there were already signs of a slackening of official interest and support. It was significant, for instance, that when government departments responded to the Cabinet Secretary’s request for proposals for a new round of official histories, in October 2007, their suggestions were not followed up: the OH committee never met and the Privy Counsellors were never consulted.⁴¹ Clearly sensing something amiss, Lord Rodgers, the most active of the three Privy Counsellors, initiated the first of three House of Lords debates on the future of the programme in February 2008, with two others following in July 2013 and December 2015 (by which time both of his fellow Counsellors had died).⁴² All three debates elicited much support for official history on the part of Lord Rodgers’s fellow peers, including historians such as Lord Hennessy, Lord Bew and Lord Lexden. Moreover, Lord Rodgers’s first intervention had a positive outcome: the appointment in December 2008 of Sir Joe Pilling, a former Permanent Secretary of the Northern Ireland Department, to conduct an investigation into the official histories programme. Sir Joe produced his report in April 2009, with a supplementary report on the publishing side by Bill Hamilton, a well-known literary agent.⁴³

The Pilling report came down strongly in favour of maintaining the official history programme:

Sir Joe also recommended that it should continue to be managed by the Cabinet Office. He suggested, however, that the programme’s aims should be made more explicit; that its links with the academic world should be strengthened by the addition of an advisory council; that its name should be changed to the Public History Programme in order to enable it to reach a wider public; and that official historians should be encouraged to raise the profile of their research by publishing articles and delivering conference papers. Bill Hamilton, meanwhile, had some sharp comments about the high costs and low profile of the official history series, and the difficulty of persuading commercial publishers to take an interest in either the series as a whole or individual volumes:

By November 2009 the Cabinet Office had come up with draft responses to the report’s recommendations, and government departments had been asked for their views.⁴⁴ There is no sign that these responses were ever acted upon. The next we hear is a letter of August 2010 in which the Cabinet Secretary, Sir Gus O’Donnell, told the three Privy Counsellors: ‘Given the current challenging economic climate, I am sure that there is likely to be a hiatus in commissioning titles.’⁴⁵ ‘That is how it was,’ Lord Rodgers told his fellow peers five years later, in December 2015; ‘There was no further explanation and nothing further about how to implement the Pilling report.’ Six years beyond that debate, the hiatus continues and there is still no clarity about the fate of the Cabinet Office’s official histories programme. The most recent volumes in the series have maintained its reputation for high quality but, for the moment, they are likely to be the last.⁴⁶

The Cabinet Office histories faced other challenges. Their expensive and poorly marketed volumes had to compete with the popular and reasonably priced ‘authorized’ histories commissioned by the intelligence agencies – first Christopher Andrew’s history of MI5, Defence of the Realm (2009), then Keith Jeffery’s MI6 (2010) and, most recently, John Ferris’s Behind the Enigma: The Authorised History of GCHQ (2020).⁴⁷ Official historians also had to compete with the claims of academic historians to provide historical lessons as relevant to current problems as anything they could offer, and in forms much more readily accessible. History and Policy, founded in 2002, has produced more than 200 policy papers and has established strong links with departments such as the Treasury, the Home Office, the Department of Transport and the Cabinet Office itself, as have the Mile End Institute at Queen Mary University of London, and the Strand Group at King’s College London.⁴⁸ However, while they have done much to raise historical awareness across government, it is difficult to quantify their direct influence on decision making.⁴⁹ Theoretically, it must be easier for busy officials and ministers to read short policy papers online than heavy works of official history, but it remains unclear whether they do so.

Documents on British Foreign Policy Overseas

The malaise afflicting the Cabinet Office programme had wider causes. The privileged access to closed official files enjoyed by official historians had already been compromised by the thirty-year rule. The Freedom of Information Act of 2000, which came into effect on 1 January 2005, followed by the introduction of a new twenty-year rule in 2013, undermined any claim to exclusivity still further. These changes also affected the work of the Foreign Office historians, who had been engaged since the 1980s in a publishing project designed to record the history of British foreign policy since 1945, just as DBFP had done for the inter-war period. The idea of producing a successor to DBFP dated back to the mid-1960s, when completion of the inter-war series was in sight. In 1966 Rohan Butler reported that he was conducting a pilot survey and had already identified three potential problems: first, ‘the formidable increase in the bulk of the archives’; second, ‘special and possibly delicate considerations arising from the Cold War’; and, third, ‘ditto in respect of decolonization and its impact on British foreign policy’.⁵⁰ His proposed solution to the first problem was to produce smaller, more selective volumes than had been the practice with DBFP, but to supplement these with what he called ‘calendars’: lists that referred to a much larger body of unpublished documents which would be microfilmed and made publicly available:

This was the solution adopted for the new series, Documents on British Policy Overseas (DBPO), which was announced by the Foreign Secretary, Sir Alec Douglas-Home, in Parliament on 2 July 1973.⁵¹ It was nearly a decade since Butler had started his pilot survey, and it would be a further eleven years before the first volume appeared. The delay was a setback from which DBPO has never fully recovered. Nor did the first volume in series I (1945–50), The Conference at Potsdam, July–August 1945 (1984), weighing in at nearly 1300 pages, meet Butler’s original criterion, despite its use of calendars and microfiches. Published two years after Butler’s retirement and eight years after the post-1945 documents had begun to be released by the Public Record Office (PRO) under the thirty-year rule, it threatened rather to fulfil his prophecy ‘that if we are not careful any new documentary publication will run the danger of strangling itself with the editors progressing slower and slower in producing more and more volumes which fewer and fewer people can find time to read’.⁵² The first volume of series II (1950–5), covering the early years of European integration, was nearly as large.⁵³

The editorial team, now led by Margaret Pelly and Roger Bullen of LSE, reverted to Butler’s original idea of producing smaller volumes and achieved a respectable output over the next decade without, however, catching up with the documents released annually by the PRO. By now, DBPO was achieving a solid reputation among the academic community: Zara Steiner chose one of its volumes as her book of the year in the Financial Times, noting that it was a ‘superbly edited publication which amateur strategists will find more gripping than most studies of the British contribution to the cold war’.⁵⁴ This was a period in which the FCO historians became more outward-looking, organizing (in 1987) their first seminar for external academics and (in 1989) the first International Conference of Editors of Diplomatic Documents (ICEDD), beginning what was to become a biennial series.⁵⁵ But it was also a period marked by tragedy – the premature death of Roger Bullen in 1988, barely a year after his appointment as historical adviser to the Secretary of State, Sir Geoffrey Howe – and the end of the era of part-time external editors that had begun with Gooch and Temperley. DBPO now became a wholly in-house operation, led by Margaret Pelly, along with two former editorial assistants with DBFP, Heather Yasamee and Gill Bennett. On Pelly’s retirement in 1990, she was succeeded as head of Historical Branch by Yasamee, and as editor by Keith Hamilton from University College of Wales, Aberystwyth, the first external academic to be recruited as a full-time editor directly employed by the FCO.

The early 1990s were a period of increasingly open government: in 1992 the existence of the intelligence agencies was publicly avowed for the first time;⁵⁶ Foreign Secretary Douglas Hurd announced a re-review of all closed FCO records (leading, among other things, to the release of documents relating to Rudolf Hess and the return of the Krupp papers to Germany); and William Waldegrave launched an initiative inviting historians to propose blocks of papers that might be considered for release. At the same time, it was becoming evident that the timetable for publication of DBPO was falling further and further behind the PRO’s annual release programme. What was the point, some people at the FCO argued, in publishing records that were already publicly available at the PRO? It was in this atmosphere that the editors of DBPO took the decision to launch a new third series which would publish documents less than thirty years old: in other words, ones that had not been released to the Public Record Office and that no one else had yet had the opportunity to see. There would be no more calendars and no more microfiches. The new policy, which tacitly recognized that series I and II might never be completed, owed much to Keith Hamilton’s research in FCO files from the 1970s, in preparation for his in-house history of the Know How Fund,⁵⁷ which had convinced him that there was scope for a DBPO volume on the Helsinki Conference of 1975. It was carried forward by Gill Bennett, who had left the editorial team in 1991, but returned to the new position of Chief Historian in 1995. Her first volume in series III, Britain and the Soviet Union 1968–1972 (1997), exemplified the new approach, using the documents, interspersed with plenty of editorial comment, to tell a compelling and often dramatic story. The second volume, The Conference on Security and Cooperation in Europe 1972–1975, edited by Hamilton, was published at the same time. Like its successor, Détente in Europe 1972–1976 (2001), it was more traditional in approach, but both broke new ground in documenting the history of European détente in the 1970s.

After the turn of the century, however, the inexorable operation of the thirty-year rule, combined with the diversion of the editorial team to other tasks (itself a measure of their growing esteem within the FCO)⁵⁸ and the emergence of new technologies which seemed to threaten the viability of traditional print volumes, led to a further reassessment. In 2002 it was decided that in future DBPO would take the form of hybrid volumes, with introduction and critical apparatus in hard copy, accompanied by a CD-ROM containing facsimiles of the original documents. Ultimately only two volumes appeared in this format. The Year of Europe: America, Europe and the Energy Crisis 1972–1974 (2006) contained 568 documents on CD-ROM: almost as many as in the first volumes of the series, and in a far more compact form. But by the time Berlin in the Cold War 1948–1990 appeared in 2009, containing 509 documents (now on DVD), technology had moved on. The format had never proved attractive to readers and the ubiquity of the internet had made it as redundant as the microfiche.

One of my first decisions after succeeding Gill Bennett as Chief Historian in 2005 was to revert to hard copy. Since then, production of DBPO has averaged roughly one volume a year, still published by Routledge. Most have been drawn from what was once, but is no longer, the ‘closed’ period – in practice this has largely meant the Thatcher era – but we have also produced three further volumes in series I, covering such vital topics as the Marshall Plan and the North Atlantic Treaty, meaning that coverage of the early Cold War is now largely complete. On the other hand, there are still no volumes at all for the period between 1952 and 1968. Since 2008 the printed volumes have been augmented by a digital version, published by ProQuest, which includes all three series produced since 1924 – British Documents on the Origins of the War, Documents on British Foreign Policy and Documents on British Policy Overseas – and has all the advantages of instant searchability.⁵⁹ The snag, as always, is cost. The price of hardback volumes averages £100 each, although this is alleviated by cheaper paperback editions and e-books; the cost of subscriptions to ProQuest can be borne only by large institutional libraries. This contrasts with such freely available and easily searchable series as Foreign Relations of the United States and Diplomatic Documents of Switzerland.⁶⁰

Yet we continue to insist – rightly, in my view – that the practice of selecting and publishing diplomatic documents remains a valuable activity: one not rendered redundant even when the relevant archives are fully open to inspection, or when parts of their contents are randomly revealed to individual researchers by the Freedom of Information Act. It is not just a matter of providing a convenient insight into the documents held by the National Archives for those who live a long way from London, or a useful teaching tool for lecturers in international history, although DBPO does perform both of these functions. There is still a place for a coherently organized body of documents grouped around a clearly identified topic, and placed in context by a full scholarly apparatus. Not that DBPO would ever claim to have the last word on any subject: indeed, its editors can recall their own delight as young researchers when they found something in the archives that earlier editors of DBFP had missed – but something, on the other hand, they might never have discovered if DBFP had not provided the clue.

Moreover, I think we can claim that in many respects DBPO is better than DBFP – better, certainly, than the early volumes. There is, after all, a certain perfunctoriness in the brevity of Woodward’s and Butler’s prefaces and their willingness to skim the surface level of telegrams and despatches without delving deeper. Taylor and Butterfield were right to lament the lack of internal minutes and the absence of documents reflecting any perspectives other than those of the Foreign Office. DBPO reveals the FCO’s internal policy debates as well as its interactions with other parts of the government apparatus – above all, No. 10 Downing Street. If Margaret Thatcher’s indignant marginal comments and vigorous underlinings are indispensable to an understanding of British foreign policy in the 1980s, what are we to make of an account of Munich that tells us nothing directly about the views of Neville Chamberlain?⁶¹ We trawl more widely than our predecessors in other respects. As early as 1997, Gill Bennett’s volume on Britain and the Soviet Union printed a 1972 ‘Report by the Joint Intelligence Committee on the Soviet Threat’, and most of our subsequent volumes have contained some intelligence material, often from the files of the FCO’s Permanent Under-Secretary’s Department (PUSD), which handled liaison with the intelligence agencies. An illustration of how the intelligence dimension can transform our understanding of earlier periods of British foreign policy is provided by a recent publication on the Potsdam Conference of 1945 which combines a selection of documents from the first eight volumes of DBPO, series I, with a number of newly released documents relating to the Soviet defector Igor Gouzenko (codename ‘Corby’).⁶² The juxtaposition shows how Western governments engaged in the task of making the post-war world were unsettled by this first glimpse into a Soviet espionage network whose full extent was yet to be revealed.

Does official history have a future? There may be signs of a revival of interest at the Cabinet Office. The Cabinet Secretary appointed in September 2020, Simon Case, is a former student of Peter Hennessy, a powerful advocate of official history; some senior ministers are also said to be historically minded. It is also reassuring that Professor John Bew has been appointed the Prime Minister’s chief foreign policy adviser. Might government departments choose to commission their own histories, as the Ministry of Defence has recently done with its authorized history of British defence economic intelligence?⁶³ Each of the armed services, after all, retains its own historical branch with its own publishing tradition quite separate from that of the Cabinet Office. Could there be official histories of recent conflicts such as the first Gulf War or Afghanistan, or has the work of official history been superseded by that of official inquiries? After Chilcot, will we ever want an official history of the Iraq War? Will there be an official inquiry into the government’s handling of Brexit, or might the work be assigned to an official historian? Or perhaps official history in the form of books has no future at all, and government should instead have one or more ‘chief historians’ or ‘chief historical advisers’, as was suggested some years ago by Sir David Cannadine and more recently by Sir Anthony Seldon.⁶⁴

As for the publication of diplomatic documents, this may be an activity whose heyday was in the 1920s or the 1950s; but it is one in which both governments and academic institutions across the world still find it worthwhile to invest. The website of the ICEDD lists seventeen countries that publish series of diplomatic documents: they include some where the practice has lapsed, such as the Netherlands; others where it has lapsed and then been revived, as in Italy; and others of relatively recent origin which have already made substantial progress.⁶⁵ Documents on Irish Policy, launched in 1997, has published twelve volumes covering the period from 1919 to 1965.⁶⁶ Polish Diplomatic Documents, established as recently as 2005, has been even more productive, having published twenty-seven volumes covering both the period of the Second Republic (1918–45) and that of the communist era (1945–89).⁶⁷ At their biennial conferences, the editors still debate methodology. Some questions, such as whether volumes should be organized chronologically or by theme, date back to the beginnings of their craft in the 1920s.⁶⁸ Others are more fundamental. Will it be possible to edit and publish documents at all when paper archives no longer exist: when communications are entirely digital and governments have largely abandoned record-keeping as traditionally understood? Or, conversely, will it be possible to place a larger volume of records in the public sphere than ever before? Here the Turkish Foreign Ministry may hold the key to the future, having completed a project to digitize Turkey’s entire diplomatic record from the foundation of the Turkish Republic in 1919 to 2001, employing hundreds of staff working in purpose-built facilities.⁶⁹ And here, perhaps, we may come close to Butterfield’s ideal of ‘a world of independent historians, choosing their own subjects for research, and allowed by the government free access to the archives’.⁷⁰

_____________

¹ Admittedly Namier tended to prefer memoirs and diaries to diplomatic documents. But see his Diplomatic Prelude 1938–1939 (London, 1948) for a formidable early analysis based on the ‘coloured books’ published by the British, French, Polish and German governments following the outbreak of war; and Europe in Decay 1936–1940 (London, 1950), for his use of DBFP and DGFP, as well as Nazi-Soviet Relations, 1939–1941 and documents published by the Soviet Union. For Taylor, see the reviews collected in Struggles for Supremacy, ed. Wrigley, passim. The two series also account for the overwhelming majority of the references in Taylor’s The Origins of the Second World War (London, 1961).

² A.J.P. Taylor, English History 1914–1945 (Oxford, 1965), quoted from the revised paperback edition (Harmondsworth, 1970), p. 731.

³ For lists of the civil and military series, see Appendix II below. With a few exceptions published in the 1980s, all of the official narrative histories, as well as DBFP and DGFP, were published by Her Majesty’s Stationery Office, London, until the privatization of HMSO in 1996.

⁴ Taylor, English History, pp. 762–3.

⁵ Harris, ‘Thucydides amongst the mandarins’, p. 136.

⁶ W.K. Hancock and M.M. Gowing, British War Economy (London, 1949), p. xi. The complete volume is available at <http://www.ibiblio.org/hyperwar/UN/UK/UK-Civil-WarEcon/index.html>.

⁷ Quoted in Harris, ‘Thucydides amongst the mandarins’, p. 136.

⁸ In his memoirs, Country and Calling (London, 1954), pp. 203–4, Hancock gives a more positive account.

⁹ Harris, ‘Thucydides amongst the mandarins’, p. 139.

¹⁰ Frankland, History at War, pp. 80–113, gives an intense personal account, reinforced by archive sources; Seb Cox, ‘Setting the historical agenda: Webster and Frankland and the debate over the strategic bombing offensive against Germany, 1939–1945’, in The Last Word: Essays on Official History in the United States and the British Commonwealth, ed. Jeffrey Grey (Westport, CT, and London, 2003), pp. 147–73, provides important context and perspective.

¹¹ Ibid., pp. 158–9. Webster died seven weeks before the volumes were published, leaving Frankland alone to face the controversy that blew up in the press.

¹² Studies, p. 136; Human Relations, p. 198.

¹³ Later Margaret Pelly: no relation to the Hon. Margaret Lambert.

¹⁴ For a complete list see Appendix IV below.

¹⁵ Sir Llewellyn Woodward, British Foreign Policy in the Second World War (5 vols, London, 1970–6). For the full story see Beck, ‘Locked in a dusty cupboard’. Dying in 1971, Woodward survived to see only the first volume in print.

¹⁶ T. Desmond Williams, ‘The historiography of World War II’, in Historical Studies I, ed. Williams (London, 1958), pp. 33–49; reprinted in The Origins of the Second World War: Historical Interpretations, ed. Esmonde M. Robertson (London and Basingstoke, 1971), pp. 36–64. The first sentence of the article makes clear that it was written in 1955.

¹⁷ Ibid., p. 44.

¹⁸ Ibid., pp. 45–6.

¹⁹ Ibid., p. 46. This is, of course, the same argument as he had made about the selection of the German documents at the time he was working on them: see p. 10 above.

²⁰ Ibid., pp. 48–9. For an early analysis following the opening of the Foreign Office files, see Donald Lammers, ‘From Whitehall after Munich: The Foreign Office and the future course of British policy’, Historical Journal, xvi (1973), 831–56. In footnote 49 (pp. 853–4), Lammers writes: ‘A preliminary survey of some of the relevant papers does, nonetheless, suggest pretty strongly that it may be easy to over-state the case for the existence within the diplomatic service of a deep and coherent opposition to the main lines of government policy.’

²¹ Spencer, ‘The publication of British and German diplomatic documents’.

²² Ibid., pp. 256, 259.

²³ Ibid., p. 255.

²⁴ George O. Kent, ‘Editing diplomatic documents: A review of official US and German document series’, American Archivist, lvii (1994), 462–81; Zala, Geschichte, pp. 242–3; Eckert, Struggle for the Files, pp. 315–32.

²⁵ My former colleague Keith Hamilton (in an email of 16 April 2021) offers a valuable corrective to this view: ‘Might I just say a word in defence of the much-maligned editors of Die Grosse Politik? They were certainly not alone amongst editors and historians in adopting a distinctly patriotic stance in the aftermath of the Great War. Headlam-Morley, who was responsible for volume XI of British Documents on the Origins of the War (BD), never quite freed himself from his role as a wartime propagandist. Moreover, Gooch and Temperley could sometimes be very economical with the documents they published. The section in volume I of the series dealing with The Hague peace conference of 1899 includes no more than the final paragraph of a twenty-seven-page War Office memorandum, arguing against any limitation of armaments. Britain’s representatives at The Hague were able to rely on the Germans to make their case for them, and the editorial note at the beginning of the BD conference section reads: “The various questions discussed at the Conference … are discussed at full length in GP, XV, 197–346” (see BD, I, p. 223) – a wonderful opt out. In this instance British diplomacy and its editors owed much to those villains of the Wilhelmstrasse.’

²⁶ Quoted in D. Cameron Watt, ‘British historians, the war guilt issue, and post-war Germanophobia: A documentary note’, Historical Journal, xxxvi (1993), 179–85 (p. 181).

²⁷ Ibid., p. 179.

²⁸ Like the current series, Akten zur Auswärtigen Politik der Bundesrepublik Deutschland (AAPD), DGFP was produced under the auspices of the Institut für Zeitgeschichte, Munich, in collaboration with the German Foreign Office: see <https://www.ifz-muenchen.de/aktuelles/themen/akten-zur-auswaertigen-politik>. The entire series has been digitized by the Bayrische Staatsbibliothek: <https://digi20.digitale-sammlungen.de/de/fs1/start/static.html>. For an account of British participation in the last years of the project see Eleanor Breuning, ‘International cooperation’, in FCO Historians, Papers Presented at the Seminar for Editors of Diplomatic Documents, Held in the Foreign and Commonwealth Office on 9 November 1989 (Occasional Papers No. 2, London, 1989), pp. 39–43.

²⁹ Memorandum of 27 July 1965, printed in Keith Wilson, Forging the Collective Memory: Government and International Historians through Two World Wars (Providence, RI, and Oxford, 1996), pp. 289–93.

³⁰ Butler minute, ‘Official histories and the proposed thirty-year rule’, 26 May 1966, The National Archives (TNA), FO 370/2906.

³¹ Followed by her own peacetime history, co-authored with Lorna Arnold, Independence and Deterrence: Britain and Atomic Energy, 1945–52 (2 vols, London, 1974).

³² Butler minute, 26 May 1966 (note 30 above).

³³ It is worth mentioning in this connection the important British Documents on the End of Empire (BDEEP) series available at <https://bdeep.org>.

³⁴ Christopher J. Murphy, ‘The origins of SOE in France’, Historical Journal, xlvi (2003), 935–52.

³⁵ Charles Moore, Margaret Thatcher: The Authorized Biography, vol. 2, Everything She Wants (London, 2015), p. 137.

³⁶ Olav Riste, ‘Open season for rewriting history’, The Times, 10 June 1986, p. 12. The other volume, Charles Cruickshank’s SOE in the Far East (1983), did not attract the same level of criticism.

³⁷ There seems to be no publicly available complete list of the Cabinet Office peacetime histories, but see Appendix III for my attempt to provide one.

³⁸ Accurate information on the recent history of the Cabinet Office official histories programme is remarkably difficult to find. The Cabinet Office’s website (<https://www.gov.uk/government/organisations/cabinet-office>) makes no mention of official history at all. The best starting point is a research briefing, produced in support of a House of Lords short debate on the programme in December 2015: <https://lordslibrary.parliament.uk/research-briefings/lif-2015-0056>. This leads to a longer report by Nicola Newson, which can be downloaded. This report in turn contains a number of valuable links, including one (the only one I have been able to find) to Sir Joe Pilling’s important 2009 report on the official histories programme, <https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/62233/future-plans-government.pdf>. Perhaps the clearest description of how the system is supposed to work is contained in an archived article by the late Rodney Lowe (official historian of the Civil Service), ‘Official history’, <https://archives.history.ac.uk/makinghistory/resources/articles/official_history.html>.

³⁹ Typically, this is not a web page in current use, but one archived by The National Archives: <https://webarchive.nationalarchives.gov.uk/20080205143007/ http://www.cabinetoffice.gov.uk/publicationscheme/published_information/1/officialhistory.aspx>. The ‘fund of experience’ was a phrase dating back to the wartime Cabinet Secretary, Sir Edward Bridges, expressing an optimism that had largely been belied in practice: see Hancock, Country and Calling, pp. 196–7.

⁴⁰ See Appendix III and comment in note 37 above.

⁴¹ This is based on my own recollections and notes I made at the time.

⁴² House of Lords Hansard, vol. 698, 5 February 2008, cols 1013–25; vol. 747, 10 July 2013, cols 335–51; vol. 767, cols 189–200.

⁴³ <https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/62233/future-plans-government.pdf>.

⁴⁴ Personal recollection.

⁴⁵ Quoted by Lord Rodgers in the Lords debate of 10 December 2015, col. 190.

⁴⁶ For example, Matthew Jones, The Official History of the UK Strategic Nuclear Deterrent (2 vols, 2017); Stephen Wall, The Official History of Britain and the European Community, vol. 3 (2018). At a late stage in production, the Cabinet Office transferred these two works to the Ministry of Defence and the FCO respectively to oversee their completion. On 9 July 2019 Lord Rodgers asked a further three questions about the Cabinet Office official history programme and the FCO’s documentary series. In response to his question ‘whether Whitehall History Publishing still exists; if so, (1) what is its role, and (2) how is it managed and by whom; and if not, who is responsible for the publication of material it previously produced’, Lord Young of Cookham replied: ‘The Whitehall History Publishing, comprised of several historical branches of government departments and led by the Cabinet Office, produces material on historical matters and themes to meet individual departmental requirements. The next publication is due out in autumn 2019.’ In response to Lord Rodgers’s question when HMG ‘last reviewed the possible resumption of producing official histories’, Lord Young replied: ‘The Government’s official history series is intended to provide authoritative histories in their own right. Two volumes of Criminal Justice history were published earlier this year. The next publication is due out in autumn 2019.’ This did not, of course, address the question of whether any further official histories were in the pipeline. <https://members.parliament.uk/member/940/writtenquestions#expand-1137959>.

⁴⁷ All published by Bloomsbury, and all with contracts negotiated by Bill Hamilton on terms very favourable to the agencies concerned.

⁴⁸ <http://www.historyandpolicy.org>; <https://www.qmul.ac.uk/mei/about-us>; <https://thestrandgroup.kcl.ac.uk>.

⁴⁹ For a helpful discussion see Matthew Grant, ‘History and policy’, in A Practical Guide to Studying History: Skills and Approaches, ed. Tracey Loughran (London, 2017), pp. 233–47.

⁵⁰ Memorandum of 18 October 1966, ‘Historical presentation of recent British foreign policy’, TNA, FO 370/2906.

⁵¹ For details of internal FCO discussions, including the question of how to handle sensitive material, see FCO Historians, History at the Heart of Diplomacy, pp. 50–2, <https://issuu.com/fcohistorians/docs/history_at_the_heart_of_diplomacy-w>.

⁵² Butler memorandum of 18 October 1966, TNA, FO 370/2906.

⁵³ The Schuman Plan, the Council of Europe and Western European Integration 1950–1952, ed. Roger Bullen and M.E. Pelly (1986).

⁵⁴ Quoted in FCO Historians, History at the Heart of Diplomacy, pp. 106–8. Her choice was series II, vol. II, The London Conferences: Anglo-American Relations and Cold War Strategy January–June 1950 (1987).

⁵⁵ The papers given to the 1987 seminar are published at <https://issuu.com/fcohistorians/docs/hpop_1>; the proceedings of the 1989 conference are published in ‘International cooperation’, in FCO Historians, Papers Presented at the Seminar for Editors of Diplomatic Documents. For the ICEDD see <https://diplomatic-documents.org>.

⁵⁶ Strictly speaking, the existence of MI5 was avowed in 1989 with the Security Service Act, that of SIS and GCHQ in 1994 with the Intelligence Services Act. 1992 was the year in which the chiefs of the two main agencies were allowed to be named publicly.

⁵⁷ A much-revised and expanded version was later published as Transformational Diplomacy after the Cold War: Britain’s Know How Fund in Post-Communist Europe, 1989–2003 (London and New York, 2013).

⁵⁸ In 1996 the historians were commissioned by Foreign Secretary Malcolm Rifkind to investigate the British archives for information on the Nazi theft of gold and other Jewish property: their reports are available at <https://issuu.com/fcohistorians/docs/history_notes_cover_hphn_11> and <https://issuu.com/fcohistorians/docs/history_notes_cover_hphn_12>. In 1998 Gill Bennett was commissioned by Foreign Secretary Robin Cook to write a report on the origins of the Zinoviev Letter of 1924, a source of long-standing grievance on the part of the Labour Party. For the full story see Gill Bennett, The Zinoviev Letter: The Conspiracy that Never Dies (Oxford, 2018).

⁵⁹ <https://about.proquest.com/products-services/dbpo.html>.

⁶⁰ <https://history.state.gov/historicaldocuments; https://www.dodis.ch/en/home>.

⁶¹ It may be appropriate here to pay tribute to the remarkable collection of original documents made available on the website of the Margaret Thatcher Foundation: <https://www.margaretthatcher.org>.

⁶² Britain and the Making of the Post-War World: The Potsdam Conference and Beyond, ed. Gill Bennett and Richard Smith (2020), <https://issuu.com/fcohistorians/docs/britain_and_the_making_of_the_post-war_world_with_>; also available at <https://www.amazon.co.uk/Britain-making-Post-War-World-Conference/dp/B08CWCGSNR>.

⁶³ Peter Davies, British Defence Economic Intelligence: A Cold War in Whitehall 1929–90 (London, 2019).

⁶⁴ Sir David Cannadine spoke at a History and Policy meeting at the Cabinet War Rooms, London, on 5 December 2007; Anthony Seldon, ‘Why every government department needs a resident historian’, Prospect, 1 May 2020, <https://www.prospectmagazine.co.uk/politics/government-department-chief-historian-whitehall-number-10-coronavirus-covid-brexit>.

⁶⁵ <https://diplomatic-documents.org/editions>.

⁶⁶ <www.difp.ie>.

⁶⁷ <https://diplomatic-documents.org/information/?pdb=27>. Two volumes, covering the periods 1918–19 and 1938–9, have been published in English translation.

⁶⁸ In the bibliography of A.J.P. Taylor, The Struggle for Mastery in Europe 1848–1918 (Oxford, 1954), pp. 578–81, can be found typically trenchant comments on the pros and cons of chronology versus topic. By arranging their documents by subject, Taylor wrote, the editors of the Grosse Politik concealed ‘the connexion of one topic with another’. Gooch and Temperley then seemed to have adopted the German model ‘without reflection’ and ‘described it as “the British way” – a phrase often used in this country to cloak any irrational act’. Only the French got it right: ‘The documents are arranged in chronological order, while a table at the beginning of each volume sorts them into subjects. It is difficult to understand how any subsequent editor can have strayed from this arrangement.’ The Grosse Politik remains the model followed by the British editors, although the current German series, AAPD, has adopted the chronological approach. The Italians, however, have supplemented their original chronological series, I Documenti Diplomatici Italiani, with a new thematíc series, Documenti sulla Politica Internazionale dell’Italia: see <https://diplomatic-documents.org/information/?pdb=24>.

⁶⁹ As reported to the ICEDD conference in Washington in 2015, sixty-five million pages were planned to be digitized within three years by a 300-strong skilled work force operating in two shifts. They had hoped to achieve an output of 100,000 pages a day but were actually achieving 130,000. For an excellent explanatory video, see <http://diad.mfa.gov.tr/short-about-the-archives.en.mfa>.

⁷⁰ Butterfield to Blaxall, 9 May 1952 (Appendix I below).