Following the PSNI and other recent data breaches, the ICO has issued guidance to public authorities. This guidance suggests a temporary stop on publishing Excel-style spreadsheets in response to FOI requests made via online platforms like WhatDoTheyKnow. The full advisory note is available online.
The advisory note emphasises that this is not a reason not to disclose requested information. Instead, the ICO says to release the information from original source spreadsheets as a CSV file – a simpler format than Excel Workbooks, with less potential for including hidden sheets or metadata that can lead to an accidental breach.
A focus on file formats is a blunt measure, and one that will need to be superseded by better procedures and technical processes.
We support authorities releasing data in the most appropriate format for the information being requested. This may sometimes mean an extract from a table, and sometimes a complete document. Excel spreadsheets are legitimate public documents, and information released in this format can be hugely valuable. It’s important to develop processes where they can be released safely.
Significant data breaches involving Excel files clearly show the risks when data management and release processes fail. These include not just breaches we see through WhatDoTheyKnow, but through disclosure logs and releases made directly to requesters. This is an opportunity for public authorities, the ICO and us at WhatDoTheyKnow to reflect on how we can best deliver the huge benefits of public transparency while safeguarding personal data.
Modern authorities need to be good at handling data. Data breaches happen at the intersection of technical and human processes. The FOI team can be the last link in the chain of a data breach when they release the information, but the root cause often goes back to wider organisational issues with the handling of sensitive data.
In the short run, the ICO has recommended training for staff involved with disclosing data. Many teams already have excellent processes and do excellent work, but all authorities should take this opportunity to consider their responsibility on the data they hold, and have appropriate processes in place.
Long term progress means developing good universal processes that keep data safe, regardless of the format of the data or how the data is released. All FOI releases should in principle be treated as if they are being released to the public, because the authority’s ability to stop a data breach ends when the information is released. Making FOI responses public produces huge efficiencies for the public sector, increasing transparency in practice, and multiplying the benefit to society of the information released.
Technology can also be part of the solution – we need to understand more about why existing technical ways of removing hidden information from Excel spreadsheets are not being used (as described in the ICO’s established guidance on disclosing information safely), and how new tools or guidance can make it easier to release data safely.
A core part of our work at WhatDoTheyKnow is dealing with the practical reality of promoting public transparency while protecting personal information. We take data breaches seriously and have processes in place for dealing with them as promptly as possible. We continue to plan and work to help reduce the occurrences and impact of personal data breaches through both our procedures and technical approach.
By monitoring how authorities respond to requests on WhatDoTheyKnow, we will seek to understand how this guidance is working in practice, and engage with the ICO and other organisations to promote effective long term approaches to this problem.
Notes on the content of the advisory
Below is our understanding of the advisory note by subject matter:
Freedom of Information requests
- Continue to comply with FOI responsibilities. This guidance is about releasing information in a way that reduces risk of accidental disclosure.
- Temporarily, do not release original source spreadsheets to online platforms like WhatDoTheyKnow. Instead – convert and release to CSV files.
- If that is not possible, then:
- Ask if the Excel sheet can be sent to a separate (non-public) address. Proceed with the original address if they ask for this.
- In all releases, go through processes to ensure there is no data breach in the material.
General data management
- Excel files are unsuitable working environments when they become very large (hundreds of thousands of rows). Authorities need to switch to appropriate data management systems that are more appropriate for managing larger amounts of data.
- Staff who use data software and are involved in disclosing information need continuous training.
- Understanding of pivot tables and their risks should be incorporated into data management.
The ICO plans to update their guidance on Disclosing Information Safely.
The checklist released accompanying the advisory has several useful steps on checking for hidden data in Excel sheets. However, on the ‘considered alternative ways to disclose’ step, refer back to the steps in the advisory note. Information converted to CSV can be released to WhatDoTheyKnow in compliance with the advisory note. The advisory note says that the source dataset should continue to be released to WhatDoTheyKnow if it cannot be converted, the requester does not want to use an alternative route, and the authority is confident it does not contain a data breach.