Following the PSNI and other recent data breaches, the ICO has issued guidance to public authorities. This guidance suggests a temporary stop on publishing Excel-style spreadsheets in response to FOI requests made via online platforms like WhatDoTheyKnow. The full advisory note is available online. 

The advisory note emphasises that this is not a reason not to disclose requested information. Instead, the ICO says to release the information from original source spreadsheets as a CSV file – a simpler format than Excel Workbooks, with less potential for including hidden sheets or metadata that can lead to an accidental breach.

–

A focus on file formats is a blunt measure, and one that will need to be superseded by better procedures and technical processes.

We support authorities releasing data in the most appropriate format for the information being requested. This may sometimes mean an extract from a table, and sometimes a complete document. Excel spreadsheets are legitimate public documents, and information released in this format can be hugely valuable. It’s important to develop processes where they can be released safely. 

Significant data breaches involving Excel files clearly show the risks when data management and release processes fail. These include not just breaches we see through WhatDoTheyKnow, but through disclosure logs and releases made directly to requesters. This is an opportunity for public authorities, the ICO and us at WhatDoTheyKnow to reflect on how we can best deliver the huge benefits of public transparency while safeguarding personal data. 

–

Modern authorities need to be good at handling data. Data breaches happen at the intersection of technical and human processes. The FOI team can be the last link in the chain of a data breach when they release the information, but the root cause often goes back to wider organisational issues with the handling of sensitive data.

In the short run, the ICO has recommended training for staff involved with disclosing data. Many teams already have excellent processes and do excellent work, but all authorities should take this opportunity to consider their responsibility on the data they hold, and have appropriate processes in place.

Long term progress means developing good universal processes that keep data safe, regardless of the format of the data or how the data is released. All FOI releases should in principle be treated as if they are being released to the public, because the authority’s ability to stop a data breach ends when the information is released. Making FOI responses public produces huge efficiencies for the public sector, increasing transparency in practice, and multiplying the benefit to society of the information released. 

Technology can also be part of the solution – we need to understand more about why existing technical ways of removing hidden information from Excel spreadsheets are not being used (as described in the ICO’s established guidance on disclosing information safely), and how new tools or guidance can make it easier to release data safely. 

–

A core part of our work at WhatDoTheyKnow is dealing with the practical reality of promoting public transparency while protecting personal information. We take data breaches seriously and have processes in place for dealing with them as promptly as possible. We continue to plan and work to help reduce the occurrences and impact of personal data breaches through both our procedures and technical approach. 

By monitoring how authorities respond to requests on WhatDoTheyKnow, we will seek to understand how this guidance is working in practice, and engage with the ICO and other organisations to promote effective long term approaches to this problem. 

---

Notes on the content of the advisory

Below is our understanding of the advisory note by subject matter:

Freedom of Information requests

• Continue to comply with FOI responsibilities. This guidance is about releasing information in a way that reduces risk of accidental disclosure. 
• Temporarily, do not release original source spreadsheets to online platforms like WhatDoTheyKnow. Instead – convert and release to CSV files.
• If that is not possible, then:
  • Ask if the Excel sheet can be sent to a separate (non-public) address. Proceed with the original address if they ask for this. 
  • In all releases, go through processes to ensure there is no data breach in the material. 

General data management

• Excel files are unsuitable working environments when they become very large (hundreds of thousands of rows). Authorities need to switch to appropriate data management systems that are more appropriate for managing larger amounts of data.  
• Staff who use data software and are involved in disclosing information need continuous training.  
• Understanding of pivot tables and their risks should be incorporated into data management.

The ICO plans to update their guidance on Disclosing Information Safely. 

The checklist released accompanying the advisory has several useful steps on checking for hidden data in Excel sheets. However, on the ‘considered alternative ways to disclose’ step, refer back to the steps in the advisory note. Information converted to CSV can be released to WhatDoTheyKnow in compliance with the advisory note. The advisory note says that the source dataset should continue to be released to WhatDoTheyKnow if it cannot be converted, the requester does not want to use an alternative route, and the authority is confident it does not contain a data breach.

If you’ve recently received a refusal to a Freedom of Information request you sent through WhatDoTheyKnow, you might have noticed our latest feature.

As mentioned in a previous blog post we’ve been working on functionality to help people challenge refusals, and the first iteration of this is up and running.

You might now see an automated notice above a refusal, identifying which exemption may have been applied. These are visible only to the person who made the request, and do not appear in every case; see more about that below.

Click ‘get help to challenge it’ and you’ll be presented with a series of questions about your request and the reply you received from the authority, to see whether there is any scope for a challenge.

Your answers to these questions will generate tailored advice on what to do next, also presenting you with editable fragments of text that you can use to get started with any challenge or request for clarification.

What’s the thinking?

WhatDoTheyKnow has always existed to make it easier for anyone to make an FOI request, without having to be an expert in the law.

We think we’ve got the initial part of that down pretty well (with the obvious caveat that there are always improvements to be made) — but up until now we haven’t directed a lot of attention to what happens after the authority responds.

Even for seasoned users of FOI, but especially for those new to the whole area, it is daunting to receive a refusal from an authority. By their nature, FOI responses contain some legalese, and this can be enough to make the best of us think, ‘Ah well, they probably know more about it than I do’ — and give up.

But another way of looking at it is that this legalese is there to help you. A body can’t just turn down your request because it doesn’t want to answer it: it has to say which exemption it is using under FOI law. If you know your way around that law, it is the key to understanding what to do next, and in fact, it’s this legal bumph that our code will be using to check what questions to ask you.

Before we introduced this intervention, a good number of users just gave up when they received a refusal from an authority — and that’s fair enough. We’d been directing users to a generic help page with details of what options are broadly available for challenging a refusal, but only the very determined would take it further.

And as we mentioned in our previous post, 22% of internal reviews (where you ask the request to be examined again, by a different member of staff at the authority) result in the full or partial release of information. It’s clear that bodies can, and do make mistakes sometimes.

Our new functionality helps you discern where that might have happened, and put together a decent challenge.

How it works

When an FOI request is refused, the authority have to give the reason, and this has to be one of a set number of ‘exemptions’ — clauses in the FOI Act that list the circumstances under which an authority is not obliged to disclose information.

Our code scans the response to identify which exemption/s have been applied. Remember, though, that this is just an automated best guess, so it’s still very much up to the user to check whether it’s correct!

At the next stage the user will be asked: “Did the authority mention any of these exemptions when refusing your request?” and there is the chance to remove exemptions if they’re wrong, and add any that haven’t been picked up by the code.

Then, informed by guidance from the Information Commissioner’s Office (ICO) for the identified exemption, we present a series of questions which should help you understand whether there are grounds for asking for an internal review.

Depending on your answers, you’ll see some advice. This might simply tell you that the exemption seems to have been applied correctly; it might advise you to ask the authority to clarify areas where their response is unclear, or it could point out where it appears that there has been an error on the part of the authority.

Note the various variations on ‘you have/may have grounds for an internal review’ — this is because each conclusion has been generated by a different response to one of the questions. In this example, the user has four different potential challenges to the refusal:

• The exemption seems to have been incorrectly applied
• The authority hasn’t provided some evidence that they should have
• The user has identified something about the contract that means the exemption shouldn’t have been applied to every part of it
• The authority hasn’t demonstrated that disclosure would be prejudicial to business interests

Whew! But if that’s a little overwhelming, there’s still some more support for the user.

Click ‘request an internal review’ and you’ll be shown some fragments of text you can copy and paste into your reply to help you start composing it. These are just prompts: they can be edited or overwritten to reflect your specific situation, and to allow you to express yourself in the language you’d ordinarily use.

Know your exemptions

Some exemptions are used far more commonly than others: for example we discovered when we began this work that the most often applied was the Section 12 exemption, ‘where cost of compliance exceeds appropriate limit’ — that is, responding in full would cost the authority too much in terms of expense or manhours.

Other common exemptions are Section 14 (turning down ‘vexatious’ or repeated requests for the same information); Section 40 (where the release would contain someone’s personal information); and Section 21 (where the information is already available elsewhere).

It’s worth knowing basic facts around exemptions: for example, if some of the information you’ve requested is covered by an exemption (say, a portion of it would contain personal information) the authority should still be releasing the rest. And if they tell you it’s available elsewhere, they should also do all they can to point you to the relevant place.

Some exemptions require a public interest test, where the authority must weigh up whether it is more beneficial for society as a whole to release the information than it is beneficial under the conditions of the exemption to refuse it. An example may be where the information relates to the UK’s defence capability, but would reveal something so important for the public to be aware of that any potential threat to national security is outweighed.

For this first stage of the new functionality, we covered the 13 most commonly used exemptions out of the full 27. Also, our tool might miss some exemptions. If that’s the case, or the exemption can’t be identified, you simply won’t see anything on the request.

Sharing knowledge

The ICO provides detailed guidance for authorities about each exemption, and this has also proved invaluable for us as we strive to point out where there may be room for requesting a review. As we’ve learned while creating this functionality, law is not precise: in many cases it is open to interpretation, and legal challenges at tribunal act as precedents, helping to consolidate its exact meaning. The ICO guidance is basically an attempt to unify such interpretation.

But the average person may not have the time to read up on the ins and outs of how this exemption or that should be applied, so we hope our sets of questions will give you a short cut towards understanding whether there’s a valid challenge to be made. Eventually, we hope it’s not too bold to suggest, the functionality may even increase the public’s understanding of FOI.

We are, of course, keen that this initiative doesn’t place an extra burden on authorities. The mechanism should work just as well in pointing out where the authority has acted correctly, and so discourage pointless challenges.

Where requests for internal reviews are made, the result should be that they are better informed, clearly structured and based on valid challenges. In time, this feature may even have the knock-on effect that authorities are incentivised to improve their initial responses, taking more care that exemptions are correctly applied.

We know this isn’t perfect yet, so if you’re from an authority and you want to share feedback, please do let us know. And if you’re a member of the public and you see anomalies in the wording or interface, please do also get in touch. You can contact us here.

—

Image: Andrew Fleming (CC by-nc/2.0)

While the UK begins the process of trying to return to some kind of normality after lockdown, full access to information must also be restored.

Back in April, we put out a blog post examining the state of Freedom of Information during the covid-19 crisis, looking at the UK and more broadly across the world. State-sanctioned delays were seen almost universally.

While we understood the difficulties faced by authorities redeploying staff members to the frontline, we said then that the right to information was perhaps more vital than ever. In times of national crisis, transparency is crucial both for retaining trust in our leaders and for keeping check on their activities.

WhatDoTheyKnow users have been asking pertinent questions about the pandemic, from requests for data on the number of cases in prisons and care homes, to the basis on which decisions about the national response strategy have been made. Potential students want to know about universities’ plans for the coming year; citizens are asking about measures put in place by their councils to encourage social distancing. And meanwhile, of course, requests for non-coronavirus-related topics are equally pressing: who’s keeping an eye on Brexit, or making sure the climate crisis doesn’t slip off the agenda for example?

The state of play

We’ve been linking to that initial post from the top of WhatDoTheyKnow, so that people making requests could get some background to the delays they might be experiencing.

But since then the global situation has moved on, and so have some aspects of FOI provision. At the time of writing:

• The Information Commissioner’s Office (ICO) is still stating that they “will not be penalising public authorities for prioritising other areas or adapting their usual approach during this extraordinary period.” Therefore, UK public authorities may still delay their requests without penalty. Read more on the ICO website.
• The Scottish Information Commissioner had previously introduced overseen a change that permitted [see below for clarification] a longer period in which authorities might respond to requests, but on 27 May a reversal came into effect and the period returned to its standard 20-day deadline. However, there is still an acknowledgement that the pandemic, and indeed their own previous relaxation of the required timescales, may have a knock on effect to requests made before that date. See full details here.

This does raise the question as to when the ICO foresees a return to business as usual. Of course, each authority will have its own experiences and challenges, with varying reasons for maintaining or removing an expectation of delayed responses. But they are guided by the regulator, and while the ICO continues to excuse lengthened response times, authorities may not hurry to do any different.

UPDATE: A representative from the Scottish Commissioner’s Office contacted us with the following clarification:

The changes in timescales under the FOI (Scotland) Act came about because the Scottish Parliament passed emergency legislation to change the timescales – they were not introduced by the Commissioner. Our position prior to the change in the law was set out in a statement we issued, and our comments, including concerns raised, on the legislation when it was introduced can be read here.

We’ve also sought to emphasise the importance of the duty to respond promptly, even during the period when the deadlines were extended, as set out in our guidance for requesting information during the pandemic. We think it’s important that requesters know their rights, and the right to a prompt response (not just one within 20/60 working days) is something that has remained consistent for FOI users throughout the pandemic.

Time to restore oversight

It’s unquestionably a time of great uncertainty for us all, with many returning to some semblance of normality while still unsure whether the much anticipated second peak is on the horizon. But given a national policy of this staged return, should the ICO not, like its Scottish counterpart, be encouraging authorities to do the same?

One compelling reason is hinted at by the Scottish Commissioner’s own caveat: that the longer the deadlines are allowed to extend, the more of a backlog will build up, causing further delays down the line.

We’d encourage authorities everywhere to re-examine any laxity they may have introduced at the start of lockdown, and to continue to do so regularly: is it still genuinely necessary now that staff may have been moved back from the covid-19 frontline?

And we’d urge them to treat the need for a timely, efficient FOI service as one of the top priorities during this uncertain period.

—

Image: Andrea Piacquadio

You might have seen it in the Daily Mirror: the full extent of the Department of Work and Pensions’ legal costs, incurred while fighting the obligation to name the companies who participated in the Workfare scheme.

Workfare is a government program which required the unemployed to work for one of the participating organisations, in exchange for no pay other than their existing benefits — working out lower than the minimum wage.

It’s a story in which our site WhatDoTheyKnow is strongly involved. The original request for the list of companies participating in the Workfare scheme was made on the site back in January 2012 by user Frank Zola.

That request was refused, noting that the information was “being withheld under Section 43 of the FOI Act which relates to the commercial interests of both the Department and those delivering services on our behalf”.

As any WhatDoTheyKnow user is given the means to do, Zola referred the request to the Information Commissioner. They ruled in favour of the release.

The government were unforthcoming, however, and the matter was taken to tribunal and through the court of appeal. Zola continued to pursue the case doggedly as the government repeatedly questioned the ruling that the information must be released into the public domain. Their defence was that the companies and charities listed as participating in the Workfare scheme might suffer negative effects to their reputation and commercial viability, given the strong swell of public opinion against the scheme.

In July 2016, four and a half years after the request had first been made, the full list was finally disclosed, and can be seen on WhatDoTheyKnow here.

But the story doesn’t end there. More than one person, including the Mirror’s own reporters, wondered just how much had been spent by defendants on both sides of the legal tussle. In August another user lodged this request with the DWP and discovered that their costs amounted to £92,250.

Meanwhile, a similar request to the ICO reveals that their costs in defending the case used a further £7,931 from the public purse.

We highlight this story partly because it shows the value of persistence. WhatDoTheyKnow is designed to help users to understand their rights. If your request is refused, it makes it clear that you have the right to request an internal review, making that route less intimidating to those who don’t know the ropes. If you go on to the appeals process, we hope that having all previous correspondence online helps with that. Other users can also offer help and support via the annotations system.

In this case though, we think many would have been deterred once the matter had been referred to the higher courts, and we congratulate everyone concerned for sticking to their guns and getting this information out into the public domain.

In a further twist, it’s perhaps worth relating that a few weeks ago, the supermarket Sainsbury’s contacted the WhatDoTheyKnow admin team and asked us to remove their name from the list of organisations who took part in Workfare, since “a small number of our stores did participate in the government’s Work Experience programme but this was not company policy”. We decided not to comply with this request.

—

Image: Andrew Writer (CC-by/2.0)