A reliable and accessible evidence base is vital for all aspects of transparent and open government, particularly in an age of propagation of fake news and misinformation. Now more than ever, we need reliable facts about government decisions and actions for public accountability and economic and social development. Unless digital data and documents are created in the first instance, and then managed and protected as reliable evidence, they cannot serve the ends of transparency, openness and accountability, nor can they be used to track progress on goals for sustainable development.¹

This chapter primarily draws upon a programme of research² on transparency and information management conducted at the World Bank by the author and her colleagues from 2014 to 2016 to explore the issue of records and their relationship to 21st-century transparency. Although the research did not focus singularly on records and information management, it did encompass a number of findings on this theme. The main findings in relation to records and information management were:

Given these findings, the final section of the chapter proposes steps that can be taken to strengthen records and information management, as follows:

Current transparency initiatives are undermined by weak records and information management

Despite the fact that records – defined as ‘[i]‌nformation created, received and maintained as evidence and as an asset by an organization, or person, in pursuit of legal obligations or in the transaction of business’³ – provide an essential foundation for transparency in the 21st century, there is plenty of evidence to suggest that the state of records and information management in public agencies in countries around the world is problematic. For at least the last 15 years, there have been regular warnings about the impact of the loss of control of records relating to a range of government functions coming from the press, auditors, academic researchers and records professionals.⁴

Studies of barriers to effective implementation of right/access to information laws consistently emphasise that poor records and information management prevents governments from responding to requests or, if able to respond, slows down the timeliness of the responses.⁵ A recent survey of information commissioners points to an increasing number of denials of requests for information on the grounds that the information cannot be found or that it is too costly to produce.⁶ For instance, a 2016 shadow report from South Africa, compiled from statistics on requests for information (August 2015 to July 2016) under the Promotion of Access to Information Act (2000), confirms the trend: 46 per cent of requests (n=369) submitted to government were refused – that is, no information was provided. Of these, 58 per cent of the requests were ignored.⁷

Nor is the effect of weak records and information management limited to the effectiveness of implementation of right to information laws. According to World Bank Open Data Readiness Assessments, in many countries (for example, Kazakhstan, Serbia and Sierra Leone) proactive disclosure and open data initiatives are impeded by weak records management. In the United Kingdom, poor data quality was found to be hindering the government’s Open Data programme. The authors of the report conducted an analysis of 50 spending-related data releases by the Cabinet Office since May 2010. They found that the data were of such poor quality that using them would require advanced computer skills. Far from being a one-off problem, research suggests that this issue is ubiquitous and endemic. Some estimates indicate that as much as 80 per cent of the time and cost of an analytics project is attributable to the need to clean up ‘dirty data’.⁸ In addition to data quality issues, data provenance can be difficult to determine. Knowing where data originate and by what means they have been disclosed is key to being able to trust them. If end users do not trust data, they are unlikely to believe they can rely upon the information for accountability purposes.⁹

Weak records and information are hindering efforts to monitor progress on the Sustainable Development Goals (SDGs) as well. Administrative records are a primary source of government statistical information. When the systems that generate administrative records do not create trustworthy evidence of government actions, quality statistical data are not available for reporting purposes. As the report from the Workshop on Managing Digital Information as Evidence to Underpin Global Development Goals notes:

A study of disclosure information under right to information laws for the period 2011 to 2013 provides a summary of data about requests, complaints and appeals published by central reporting bodies in eight countries (Brazil, India, Jordan, Mexico, South Africa, Thailand, the United Kingdom and the United States). The study authors found that practices were far from standardised, and data were often unavailable or incomplete. They pointed out that, when good data are available:

Issues related to missing data, data quality and reliability of government administrative systems have made it difficult to use the reporting data to assess the effectiveness of right to information laws. Efforts to draw conclusions about the effectiveness of RTI law from alternate data sources, for instance reports of civil society groups or expert reviews, can lead to disputes between different groups about the accuracy and reliability of the facts.

Weakness in records and information management is a widespread and persistent problem

As noted earlier, weakness in records and information management is not a new phenomenon. Pino Akotia highlighted it explicitly in 1997 in relation to the management of public sector financial records in Ghana and the implications for government transparency and accountability. His study demonstrated how poor records management can undermine public sector reforms and financial controls.¹² His findings have been supported by those of the International Records Management Trust (IMRT), which has regularly conducted research into the relationship between good governance and record-keeping since 1989. For instance, an IRMT study in 2011 on the management of public sector financial records in sub-Saharan Africa found that:

A more recent study conducted by the World Bank on drivers of effectiveness in the implementation of right to information laws also found that weak records and information management can prevent effective operation of laws governing public access to information.¹⁴ A follow-up survey of six countries (Albania, Jordan, Scotland, South Africa, Thailand and Uganda) using the World Bank’s Right to Information Drivers of Effectiveness (RIDE) indicators found that records management scored poorly in all of the countries studied with the exception of Scotland, suggesting that this issue is an overlooked factor that undermines right to information implementation.¹⁵ Anecdotal cases continue to lend weight to the general findings, even in countries with fairly effective right to information systems. For example, in India in 2014, government officials in the Union Home Ministry were unable to respond to a request for information having destroyed the approximately 11,000 files that would have enabled them to respond. They were also unable to produce so much as a list of the files that had been destroyed.¹⁶

New digital forms of communication and conducting government business have exacerbated earlier weaknesses in records and information management

Digitisation and the introduction of e-government is often viewed as a panacea to government records and information management weaknesses. Even as technology makes it easier to initiate transparency reforms, the lack of attention to the quality and management of public sector data and documents can undermine the impact of right to information and open data initiatives. This trend can be observed across the board – virtually all countries are currently struggling with digital records management challenges to varying degrees.

For example, a survey on records management in Côte d’Ivoire, Ghana, Kenya, Liberia, Malawi, Sierra Leone, South Africa and Tanzania conducted by the World Bank in 2016 indicates that: 85 per cent of participating countries have digitised their public records, but only 16 per cent are storing digitised records and information in secure, professionally managed digital repositories that will ensure they will have access to good quality information through time.¹⁷ Seventy-one per cent of countries recognise that email is being used to conduct government business but, in an equal number of countries, public officials are using their personal email accounts and there are no policies in place to capture these types of records.¹⁸ Eighty-five per cent felt that they did not have sufficient policies and procedures in place to manage records in digital format in support of right to information and open government.¹⁹ And, more than half the countries reported that their staff had had no training in managing and preserving digital records and recognised an urgent need for technical assistance to provide such training.²⁰ They also pointed to the need to raise awareness across civil society organisations about records and information management challenges and their link to open government.

Weak control of digital records and information weakens transparency and public accountability mechanisms

Though much has been written about the positive power of technology and information to support greater transparency and accountability²¹ and, by extension, development, there have been unintended consequences and downside risks for transparency and accountability associated with the way recorded information is produced and managed in digitally enabled developing country public sector contexts. As a result of e-government initiatives and increasing digitalisation of government operations, public sector authorities have come to rely upon a growing array of communications technologies to create, exchange and store information – from traditional paper-based filing systems, to structured databases, ‘unstructured’ content management systems, social media platforms, web technologies, mobile platforms and cloud technology.²²

Managing the information created and contained in these systems has been another matter. Generally, there has been an inverse relationship between the age of the technology used to create, exchange and store information and the capability of public sector authorities to manage and preserve the information in a trustworthy and accessible form. Data and documents are often stored on personal drives, on personal accounts on commercially available email or social media platforms, on isolated computers, or on unmanaged network drives, making them difficult to access and unlikely to survive through time.

Nor are these issues unique to developing countries. In Canada, it is possible to point to examples of the impact of weak records and information management on transparency and public accountability. These include:

Persistent cultures of secrecy lead to oral government and avoidance of record-making and keeping

Many politicians and public officials do not truly embrace values of openness and transparency but continue to hold on to secrecy. Traditional cultures of secrecy can remain resistant to the introduction of right to information laws. Qualitative data from a recent study by the Africa Freedom of Information Centre show that in Kenya, Malawi and Uganda government/public officials continue to treat public information as ‘secret’ and therefore not accessible by anybody.²⁵ Journalists are considered by public officials to be the main beneficiaries of right to information legislation and not those they represent, namely the public, thus the need to enhance public officials’ knowledge of the right to information as being a right for all, and not only journalists.

When there are few staff incentives for openness and transparency (Table 11.1), this can leave an opening whereby public officials circumvent transparency and openness laws. This is most noticeable in the recent widespread use of private email servers and ‘out of band’ communication channels (for example, BlackBerry Messenger) by public officials. When not addressed, such practices may erupt into a government scandal that undermines public trust, as was the case in British Columbia in Canada’s triple-deleted emails scandal.²⁶

These cultures of secrecy can lead to bypassing of record controls and avoiding creating and keeping records altogether as a deliberate strategy to frustrate transparency laws. According to some, this ‘off the record’ practice of government is leading to more denial of access to information under right to information laws due to missing records.²⁷ For example, the Office of the Information Commissioner of Canada has seen an increase in the number of complaints received about missing records: of 51 per cent between 2011/2012 to 2012/2013 and 66 per cent from 2012/2013 to 2013/2014. This trend has led to a growing call to establish a legislative duty to document. Indeed, in a number of jurisdictions, such a duty already exists.

Good data are needed on records and information management implementation in support of transparency

Much of the evidence of negative side-effects of weak records and information management and, by extension, government transparency and accountability, derives from professional literature in the field of records and information management and archives administration, with some further evidence coming from transparency research. This body of literature has the advantage of being written by practitioners regarding what is happening on the ground within the public sector. However, it remains largely anecdotal in nature and geographically uneven in coverage. Consequently, it is very difficult to generalise from the findings: robust claims about effects and causality are difficult to make. To advance the discussion, good comparative data are needed about the state of records and information management in countries that could be used to analyse the relationship between the quality of recordkeeping and the ability to gather statistical data for reporting on SDGs.

Currently, no such up-to-date comparative data exist. In 1995, the International Council on Archives (ICA) undertook an ‘International Survey of Archival Development’, gathering statistical and qualitative information from national archives around the world.²⁸ This survey is now nearly 20 years out of date, and was completed before many countries began digitisation programmes or began to create and store records in digital form. There is therefore a need to establish a baseline picture of the current state of records and information management and archives administration as a first step towards identifying how the introduction of information and communication technology has changed the equation and the effect this has on public accountability and transparency.

The existence of good data and comparative indicators helps drive improvements and progress towards realisation of the SDGs. The Global Right to Information Rating developed jointly by the Centre for Law and Democracy and Access Info Europe, for example, has helped governments design new right to information laws, framed discussions between government and civil society about such laws, and in a number of cases (for instance, Morocco) has helped improve what was ultimately passed into law. There is an opportunity to achieve similar progress through development and application of records and information management indicators.

Recognising the need for a set of indicators to assess the state of records and information and to track progress, the programme of research on transparency and information management included the development of a tool for use in high-level assessments of the strength or weakness of systems of record, as defined in international standards and in relation to ICT implementation. The tool identifies essential factors that predict whether the records created and held in these systems will be available and trustworthy through time to support development goals, and it provides simple tests to determine whether or not these factors are in place. The tool is structured as follows:

The three main goals that good records and information management support are transparency, accountability and the rule of law. In turn, these goals deliver progress on other SDGs, such as reduced corruption or gender equality. Records are key enablers for achieving these goals, but only if they are available and have integrity. If records cannot be found, contain errors or have been manipulated, they cannot fulfil their purpose of providing an evidence base to measure economic and social progress. To test whether recordkeeping regimes are sufficiently robust, the project team devised a series of simple questions as follows:

Policy

Standards

Roles and responsibilities

Systems and practices

Can the five categories of information listed below be retrieved to provide evidence of actions and transactions one year old? Three years old? Five years old?

Capacity

Once the assessment is complete and areas of weakness have been identified, these can be addressed through the following strategies:

Policy

Standards

Roles and responsibilities

Systems and practices

Capacity

Steps that can be taken to strengthen records and information management

Strengthen laws and policies governing digital records management

Although a comprehensive and up-to-date dataset on the status of records and information management in governments worldwide does not yet exist, our programme of research indicates that in many countries, laws governing the management of records and information management need to be updated to improve the governance of digital records. Even countries with strong records management capacity struggle to keep up with the challenges to effective records management brought about by technological change. Efforts must be made, therefore, to clarify the status of new forms of documents, such as emails and social media, and to bring legal and regulatory frameworks for the management of records and right to information regulations into alignment with technological realities.

A recent example comes from the United States, where laws and policies were updated in 2016 to clarify guidance on how digital records, such as emails, must be handled in order to support the public’s right to information.²⁹ Scotland, too, has introduced recent reforms to its public records law requiring 250 public bodies to introduce records management plans approved by the national Keeper of the Records.³⁰ These reforms were introduced as a result of a public inquiry into problems with missing records.

Introduce independent records and information management oversight

Most public accountability mechanisms, including the right to information, operate best when there is some form of independent monitoring and oversight. In the case of right to information laws, this role is most often filled by an information commissioner who reports directly to the legislature. Given the importance of records in providing the evidence base for public accountability, it is surprising that in general, no such independent monitoring and oversight exists for the records and information management function. Instead, most public officials responsible for government recordkeeping report up the chain of command within the executive branch of government. This can weaken records and information management because records management staff may have a tendency to default to the values of secrecy in the same way that other public officials may do.³¹ Even when they are supportive of transparency and openness, they may have no recourse to draw attention to practices or conditions that undermine records and information management effectiveness.³² For this reason, there is a need to establish the same independent monitoring and oversight for records and information management as exists for other public accountability mechanisms within government.

In many jurisdictions, information commissioners have been taking up this role informally, but generally they have no power in relation to recordkeeping practices. Often, they can only make recommendations and level criticism when records are not created or effectively managed. An update to right to information laws, for example to grant information commissioners more power to investigate records and information management practices, to issue penalties and fines, to require remedial action and to provide records and information management training to information commissioners and their staff, would strengthen records creation and management as public accountability mechanisms.

Align incentives of public officials with RIM principles and transparency policies and laws

Transforming cultures of secrecy and oral government into open and transparent government, with effective records and information management practices, will require going beyond legislative and policy changes to address values and incentives structures. Values of openness and transparency need to be supported at the level of policy-makers. Senior executives and public officials at all levels need to be incentivised to shift their values into alignment with these values. This can be achieved through internal training, setting appropriate job demands, clear rules, clear lines of accountability, performance management systems and strong career prospects for officials responsible for records and information management and for right to information implementation on the front lines. For example, in British Columbia, following an incident where a public official was found to have destroyed documents to avoid making them available in response to a request under the province’s access to information law, the Chief Records Officer is leading a government-wide programme to provide access to information training and establish new lines of accountability, including setting access to information-related performance objectives for all staff.³³

Encourage collaboration

Staff working on right to information requests and those working on records and information management would benefit from working together closely, since records and information management staff are familiar with the nature and location of data and documents that may be requested under right to information laws. For example, the World Bank Group’s chief archivist has responsibility for implementing the access to information policy.³⁴

Collaboration among professional records managers and archivists and those engaged in right to information work will lead to more opportunities to work collaboratively within government as well. In that regard, it was heartening to see that the 2016 Open Government Partnership Summit included a session on records management and how to combat ‘off the record’ government. A blog on the session pointed to an observation arising from the discussions that, when it comes to open government reformers, ‘there are not always strong synergies between, on the one hand, Information Commissioners and right to know activists, and on the other, records-management professionals and public sector employee bodies who face records keeping challenges on a day-to-day basis’.³⁵ On the other hand, there was not one records manager or archivist on the panel of official speakers. Surely meaningful roundtable discussions about how to transform records and information management in support of transparency must begin by inviting records managers and archivists to sit at the table.

Conclusion

Although it is only one of many components of achieving effective implementation of right to information laws and open data, effective records and information management is critical. It has received little attention relative to its impact upon the effectiveness of transparency initiatives. The effectiveness of these initiatives, and by extension public accountability and trust, will depend upon making improvements to current practices of record creation, management and preservation in governments around the world.

Ultimately, this will require the ability to gather good data about the current state of the records and information management needed to benchmark initial conditions and make it possible to track progress. It will also rely upon updated laws and regulations that adequately address the new digital reality of government, the growing trend towards oral government/destruction of records and the need to align incentives in the public service to encourage behaviour that supports the goals of transparency and public accountability.

Finally, it will require independent monitoring and oversight of records and information management practices within government agencies, including more meaningful dialogue and closer ongoing collaboration between those responsible for transparency and those with responsibility for records and information management. With stronger records and information management across governments, the data needed to derive accurate, reliable and authentic statistical information relating to progress on the SDGs are achievable.