Alaveteli is our platform that anyone can use to run a Freedom of Information site in their own country or jurisdiction.

As the number of requests grows on an Alaveteli site, it can become increasingly difficult to find released information that you’re interested in.

You can search, but the more general the term, the more likely that you’ll pull in results where the term is mentioned incidentally rather than being directly related to the information released. Or you can browse by authority, but that’s more fiddly when you want the same information from a range of authorities.

There’s also the issue that people new to FOI might not have a clear idea of what to ask about. Freedom of Information is great because you can ask for anything, but as a newcomer that can feel overwhelming. You need some direction to know where to start.

Request categories allow us to curate related requests to bridge the gaps mentioned above.

Here’s an example of a category on WhatDoTheyKnow that compiles successful Freedom of Information releases related to the British Post Office scandal.

Categories can be created in the admin interface via the Requests > Categories menu item.

Request categories work in a similar way to the current public body categories – in fact, as part of this development we’ve revamped the underlying code so that it applies to both!

At their core, they’re composed of three things – the title, body (where we can add explanatory content), and requests grouped by a tag.

 

Notes can be added to call out key information, and categories can be added to headings to create a layer of hierarchy. As part of this development, we’ve also improved notes so that they can be more easily styled with some preset colours, and added rich text editing to improve the formatting of longer notes.

 

We’ve started building up some interesting categories of requests on WhatDoTheyKnow, but we’d love to hear which ones you’d like to see.

If you’re interested in how the development unfolded you can take a look at the related work on GitHub.

—

Banner image: Garmin B

 

CSV is a great format for releasing sets of structured data in response to Freedom of Information requests. Indeed, on WhatDoTheyKnow we’ve seen several thousands of CSVs released.

We’ve recently added the ability to explore CSV files via a Datasette instance. Here’s an example.

Opening the CSV in Datasette makes it easy to explore and analyse it in an interactive website.

If you’re not familiar, Datasette converts the CSV to an sqlite database, which means you can then query the data using SQL.

Alaveteli uses the publicly available lite.datasette.io instance by default, but you can host your own instance and configure it at theme level like we’ve done for WhatDoTheyKnow.

You can see the implementation details at mysociety/alaveteli/#7961.

—

Banner image: Joshua Fuller

Alaveteli is our platform that anyone can use to run a Freedom of Information site in their own country or jurisdiction.

We’ve added new functionality that allows Alaveteli sites to highlight blog posts on the homepage, so they’re more visible. In this way, Alaveteli sites can not only help users with the ‘how’ of making an FOI request, but also show, in a very tangible way, the ‘why’  — especially if the posts are highlighting impactful uses of the site.

Previously, Alaveteli had a basic way to pull in an RSS feed of blog posts. These were only available on the blog page (/blog), which often does a disservice to all the great work that gets written about.

We wanted to better signpost the blog from other pages in Alaveteli to celebrate great FOI use, and help users understand how a seemingly simple FOI can go on to have an outsized impact.

When the site runner configures a blog feed, posts are pulled into Alaveteli and cached in the database. This makes them available on the homepage. Here’s what that looks like on WhatDoTheyKnow:

They’re also visible in the admin interface via the Tools > Blog Posts menu item.

At present only the title, URL and publication date are cached in Alaveteli. These records are intended to be “pointers” to the canonical URL of the article hosted on the external blog service.

In the admin interface, blog posts can be tagged to indicate their subject matter.

As FOI requests and authorities can also be tagged, this allows the blog posts to be highlighted in the sidebar of appropriate pages where there’s a matching tag. So, if someone’s browsing requests or visiting an authority that deals with the climate emergency, for example, they’ll be shown relevant blog posts – hopefully making them more visible to people who have already displayed that they have an interest in the topic, and giving those people a bit more contextual knowledge.

In future we’d like to make these posts more visible, by importing header images and a short summary, and give the ability to display some posts when there isn’t a direct tag match.

You can read about the initial design and subsequent conversation and pull requests starting at mysociety/alaveteli#6589.

—

Banner image: Patrick Perkins

How do you bring about systemic change within structures that are embedded into the national culture? That’s a big question, but it’s one that users of our Freedom of Information site WhatDoTheyKnow are often tussling with.

One place to start is with data that helps you map the current state of affairs, and FOI can be the perfect medium for getting hold of that. When we spoke to Jack Russell from Parents for Inclusive Education (PfIE), a grassroots organisation of primary school parents in Northern Ireland, he explained the value of data very well: “it means you can start a conversation”.

So, what are PfIE trying to achieve?

“We came together because we want to see a more inclusive primary education for every child” – and they’re starting with religious education.

“We realised that, for many parents, there was a lack of clarity around how RE is delivered in Northern Ireland, and what rights parents have in this area.”

PfIE wanted to gather data on who comes into schools to deliver RE lessons, collective worship and assemblies. Their aim was to achieve an accurate, representative picture of practices across Northern Ireland, as opposed to their baseline assumptions which, as they admit, had up until then been based on anecdotal evidence.

From small beginnings

And so began a large-scale FOI project — although initially the team had much more modest plans: 

“At first, we were only going to contact our own schools to ask them who was given access and how this was communicated. 

“But then we realised that other parents might want to be informed about these practices at their schools — and they were entitled to answers too. So we decided to send a Freedom of Information request to every publicly funded primary school in Northern Ireland, apart from special schools: that was 772 in total.”

The organisation had some tech expertise amongst its members, and, as they explained, at first it seemed that WhatDoTheyKnow wouldn’t quite be suitable for their needs:

“One of our team — Laura — had successfully used WhatDoTheyKnow in the past to query hospitals about their waitlist times for outpatient appointments, so she suggested using it. But after some initial research, we decided not to, as we’d wanted to include attachments and links in our requests. 

“I’d written a script to batch send them all, but it turned out that these were heavily spam filtered by the schools’ email server, so we fell back on WhatDoTheyKnow.

“I’m really glad we did, as the fact that all correspondence will be public is a huge plus for us.”

Managing batch FOI requests

So, how did PfIE manage their 772 FOI requests? They signed up for our WhatDoTheyKnow Pro service, which is designed specifically to help keep track of large batches like this, and also allows users to keep their requests and responses private until they’re ready to release their findings.

“We focused our questions around two areas: first, access: which churches and religious organisations were being given access to schools, and how that access was managed via processes and/or controls; and secondly communication: whether and how parents were made aware of religious visitors; and were informed about the options to withdraw their children from religious practices.

“We asked 14 questions in total, some of which were yes/no or multiple choice, others which required free-form answers.”

FOI allows the request-maker to specify the format they’d like to receive their responses in, which can save a lot of data-cleansing further down the line. As Jack acknowledges,”we received submissions back from schools in varied formats, including Word and PDF attachments, and also as plain or rich text email replies.”

It was all useful, though. “The data we collected provides us with an objective, fully representative sample — we had a 99% reply rate — to gain an accurate understanding of RE practices in Northern Ireland primary schools. 

“We understand this response level to be unprecedented, according to academics we’ve spoken to who have conducted similar research. Our project is primarily focused on making data transparently available to parents, so from this perspective the 99% number is hugely encouraging. It also means that any aggregate conclusions we draw are as close to being unbiased as possible — we actually have a response rate that is higher than the NI Census 2021 (97%) which people were legally required to complete.”

Tenacious in the face of challenges

Getting to this gratifying result wasn’t all plain sailing, though. Jack explained the issues they encountered along the way:

“Some schools initially mistrusted the FOI request email that came through WhatDoTheyKnow, and didn’t know whether they had to reply. However, a couple of weeks after we sent the request out, the Northern Irish Education Authority issued guidance instructing schools to reply, providing an information document and template response.”

In any large batch of FOI requests there will be a variety of levels of response, and PfIE came across this too. 

“There were non-responses, partial responses and responses with an incorrect understanding of the question. Our first technique to remedy these was by following up via WhatDoTheyKnow, which provided alerts and tools which made this very easy to do — another reason I’m very glad we went with the platform!”

Fortunately, the FOI Act has a provision for dealing with non-responders: referring them to the Information Commissioner’s Office.

“For persistent non-repliers, we contacted the ICO, who very diligently helped us further encourage schools to respond.

“But several of the schools that responded late, following an ICO decision notice, sent their responses to our own email account, meaning that the responses didn’t appear on WhatDoTheyKnow. The team at WhatDoTheyKnow were very helpful in adding these: I sent through several batches of .eml files and they made sure they appeared within the conversation.”

On a mission

So how will PfIE be sharing their findings? They are launching a report today, On A Mission, with an event at Stormont. They’ve also created an online map to help people explore the data.

But they’re not stopping there: “After releasing the findings of our report, we plan to create resources and a set of best practices for schools to achieve a more inclusive RE experience for all students. We also plan to engage and empower parents, hopefully promoting a sense of transparency and open dialogue between the school and parental community.

“Beyond this, we have several other plans to empower parents, increase transparency and improve the education system in Northern Ireland”.

And that’s how you start to make change

PfIE have used the mechanism available to them to produce exactly the outcome they were after.

“The tools provided by mySociety, together with help from the ICO in chasing up the late responders, and the cooperation of the NI Education Authority in doing the same have been invaluable in achieving this level of response,” says Jack.

“We would definitely recommend WhatDoTheyKnow. The tools have been really useful in managing a large scale request, and the fact that all correspondence will be publicly searchable and visible is invaluable: it adds a great deal of credibility to our research by effectively underwriting our findings with an auditable trail of evidence. 

“And on top of this, the team have been super-helpful and a pleasure to work with! “

We’re glad to have been of service. Thanks very much to Jack for talking us through the project. If you’d like to know more, visit the PfIE website, where you can also sign up to their newsletter to be kept informed.

 

—

Image: Priscilla Du Preez

The ICO have today announced that they intend to fine the Police Service of Northern Ireland (PSNI) for their accidental release of staff’s personal information in August 2023. This data was released in response to a Freedom of Information request made using WhatDoTheyKnow.

mySociety is a charity; we run WhatDoTheyKnow as a vital tool to help anyone exercise their right to information held by public authorities. We understand the repercussions of a breach like this, which serves to demonstrate that public authorities must be good at dealing with personal information. We welcome the ICO’s emphasis on the importance of robust release processes to ensuring that information that is important to the public interest can be released safely. 

We take the responsibilities that come with operating a large platform extremely seriously, especially around the personal data breaches that can occur when authorities’ release processes fail. Following this breach, we’ve undertaken a significant programme of technical and process work to play our part in reducing the risks of this kind of incident.

We’ve developed a new piece of code which analyses spreadsheets as they come in as responses to FOI requests on WhatDoTheyKnow, and holds them for review if they are detected to contain hidden data. The deployment of this code has proven successful and we will be continuing to improve it. In its first three months, this spreadsheet analyser has screened 3,064 files and prevented the release of 21 spreadsheets that have been confirmed to contain data breaches, and 53 which were likely to contain data breaches (around 2% of the files screened in total).

In an ideal world, such measures would not be necessary; we continue to work with authorities making such releases to help them understand the reasons for data breaches, the potential severity of their impact, and how to avoid them.

This blog post was updated at 10:04 on 23 May to correct the figures around the number of spreadsheets screened.

 —

Image: Pietro Jeng

Responses obtained from a widespread FOI project can be difficult to analyse, until they are sorted into neat datasets. This allows you to make valid comparisons, pull out accurate statistics and ultimately ensure your findings are meaningful.

In our third seminar within the Using Freedom of Information for Campaigning and Advocacy series, we heard from two speakers. Maya Esslemont from After Exploitation explained how to prepare for an FOI project to ensure you get the best results possible (and what to do if you don’t); and Kay Achenbach from the Open Data Institute explained the problems with ‘messy’ data, and how to fix them.

You can watch the video here, or read the detailed report below.

Preparing for an FOI project

After Exploitation is a non-profit organisation using varied data sources, including FOI requests, to track the hidden outcomes of modern slavery in the UK.

Maya explained that they often stitch together data from different sources to uncover new insights on modern slavery. She began with a case study showing some recent work they had done, using WhatDoTheyKnow to help them understand the longer term outcomes after survivors report instances of trafficking. This stood as an excellent example of how much work needs to be done before sending your requests, if you are to be sure to get the results you need.

In this case, After Exploitation were keen to understand whether there is any truth in widely-held assumptions around why human trafficking cases are dropped before they are resolved: it’s often thought that there are factors such as the survivors themselves not engaging with the police, perhaps because of a nervousness around authorities.

But what are these assumptions based upon? Actual information was not publicly available, so we wouldn’t know if cases were being dropped because of low police resource, a lack of awareness or more nuanced factors. Until the data could be gathered and analysed, the perceptions would continue, perhaps erroneously.

Before starting, After Exploitation thought carefully about the audience for their findings and their ultimate aims: in this case the audience would be mostly the media, with the aim of correcting the record if the results flew in face of what was expected; but they knew that the data would also be of use to practitioners. For example, charities could use it to see which areas to target regionally for training and other types of intervention.

They placed FOI requests with police forces across the country, making sure to ask for data using the crime codes employed by the forces: were cases dropped because of ‘lack of evidence’; did they have a status of ‘reported’ but not gone on to exist as an official crime record?

The project had a good outcome: while some requests had to go to internal review, ultimately over 80% of the forces responded with quality data. The findings were worthwhile, too: general perceptions did indeed prove to be wrong and there was no indication that ‘no suspect identified’ was a result of the victim’s lack of involvement. The resulting story was able to challenge the general narrative.

So, how can After Exploitation’s learnings be applied to the work of other organisations or campaigns?

Maya says:

• Planning, rather than analysis, is the majority of the work;
• Identify the need and purpose before you even start to pick which authorities to send requests to;
• Be clear who the audience for your findings is;
• Consult with other stakeholders to make sure your parameters are really clear.

Planning

Before you even begin, make sure your project isn’t asking for data that has already been collected and is in the public domain — this might seem obvious but it’s easy to overlook. Check other people’s FOI requests (you can do this by searching on WhatDoTheyKnow); look for reports, research, inspectorate/watchdog outputs, and data released as part of parliamentary enquiries.

That said, even if you do find previous data, there is sometimes value in requesting more up to date or more detailed information with a new set of FOI requests. If you see a national report collating data from every council for example, you could do an FOI project asking every council for a more detailed breakdown of what is happening in their region.

But before sending a batch of requests to multiple authorities, ask yourself if there is a centralised source for your data. If so, then just one FOI request might be enough: for example, homelessness data is already collected by the Depts for Housing, Levelling Up and Communities, in which case one request to them would save time for both you, and more than 300 public authorities.

Another question to ask before starting off on your project is “what is the social need?”. Does this need justify the resource you will expend? Mass FOI projects can be a bit of a time commitment, but the utility might not just be for your organisation: perhaps you can also identify a social benefit if the data would be of use to other groups, academics or journalists.

Define your intended audience: will the data you gather be of interest to them? Do you have a sense of what they want? For example, MPs often like to see localised data that applies to their constituencies. Journalists like big numbers and case studies. If you think your findings are important but might have limited appeal, you could consider including an extra question to provide details that you don’t need for your own purposes, but which could provide a hook.

Next, will the data that you gather actually be suitable for the analysis you want to perform? To avoid time-consuming mistakes, make sure the data you’ll receive is broken down in the way that you need. As an example, suppose you wanted to ask local authorities for details of programmes offered to children in different age bands: you might receive data from one council who has offerings for children ‘under 18 months’ and another ‘under two years old’ — and where units differ, they are difficult to compare and contrast. Be really precise in your wording so there’s no mismatch, especially if your request is going to a lot of authorities.

Consider, too, whether you can you get enough responses to make your data meaningful: 2,000 people is the figure believed to be representative of the population as a whole. Decide how many responses you ideally need for your purposes — and, in a scenario where not all authorities respond, the minimum you can work with.

You might want to contact other groups or organisations who could be interested in the same data, and ask if there are details that would be useful to their work.

As suggested in Maya’s case study, try to use existing measurements where you can: if you shape your requests to the methodology the authorities themselves use to collect the information, such as KPIs or their own metrics of success, these will be much easier for them to supply.

If you’re not sure what these metrics are, you can sometimes access internal guidance by googling the name of the authority plus ‘guidance’. Alternatively, submit scoping requests to a handful of authorities to ask how they measure success, etc.

At this stage it’s also useful to decide what quality of data you will include or exclude. For example, if you ask about training materials and one authority says they offer training, but don’t include the actual materials, do you include it in your figures? The more authorities you ask, the more ambiguities like this you’ll normally encounter.

Think about where and how you will log the data as it comes in. Maya recommended WhatDoTheyKnow Projects as a good tool for extracting data. Whatever you use, you should consider accessibility: can your platform be accessed by everyone you’re working with, across different communities? Especially if you are working with volunteers, it’s important to remember that not everyone has a laptop.

Also consider the security of the platform: how much this matters will depend on how sensitive the data is, but recognise that Google sheets and many other platforms store the data in the cloud where it could be more vulnerable to abuse.

After Exploitation take great pains to ensure that their data is accurate. They recommend that each response is assessed by two different people, making sure that everyone knows the criteria so they’re applied consistently; and doing regular spot checks on a handful of cases to make sure they are all logged in the same way and there’s no duplicate logging.

This is time-intensive and arduous, but if you have other stakeholders they might be able to help with the data checking: for example, knowing that they would eventually place the story with the BBC, After Exploitation were happy to hand this task over to their inhouse data checkers.

What if things go wrong?

If you’ve done all the planning suggested above, it’s less likely that your project will go awry, but even if it does, Maya says that there’s always something you can do.

No or few responses: ask yourself whether you have the capacity to chase no/late replies, and if you still don’t get a response, to refer them to the ICO. If not, consider prioritising the bodies that are most relevant to your work, eg the biggest authorities or those in areas with the densest populations; but be prepared to defend accusations that not every authority had a fair hearing unless you do them all.

If you know your requests were well worded, but you’re not getting a lot of responses — perhaps because you’re dealing with a contentious issue, or simply because the authorities cash-strapped — you could shift to measuring the types of responses you get. If authorities aren’t able to answer the question, this can often be just as revealing.

Responses that don’t tell you what you set out to understand: Consider whether there are any alternative angles in the data you do have: are there any additional themes, particularly in any free text fields? Or try a new round of requests asking for more detailed information.

Responses don’t cover the whole country: If you can’t get data from everywhere, could you narrow down to just one area and still have useful findings? Even the most basic data can set the scene for other researchers or organisations to build on: you can put it out and outline the limitations.

Results

The impact of gathering data through FOI can be massively powerful, as After Exploitation’s work shows. They have revealed the wrongful detention of thousands of potential victims of human trafficking when the government were denying it could happen; opened the debate about locking up vulnerable people; and uncovered the flawed decision making in the Home Office on modern slavery cases. It was only through FOI requests that all this information came into the public domain and was picked up by mainstream media.

—

Combining different sources of data to create datasets

Kay Achenbach is a data trainer on the Open Data Institute’s learning team; the ODI works with government and companies to create a world where data is working for everyone.

Kay shared a case study from the medical field, in which an algorithm was being designed to quickly assess high numbers of chest x-rays. The aim was to automate the process so that people identified as needing intervention would be sent to specialists right away.

The developers wanted to make sure that different demographic groups weren’t being biased against, a common issue with algorithms built on existing data which can contain previously undetected biases.

The test material was a set of x-rays from a diverse population, that had already been examined by specialists. They ran them past the algorithm to see if the diagnoses produced were the same as those made by human doctors.

The doctors’ assessments came from three different datasets which, combined, comprised data from more than 700,000 real patients. As soon as you combine datasets from different sources, you are likely to come across discrepancies which can make analysis difficult.

In this case, one dataset had diagnoses of 14 different diseases, and another had 15 — and from these, only eight overlapped. The only aspect that could for sure be compared was the “no finding” label, applied when the patient is healthy. That limitation set what the algorithm was asked to do.

Other fields were problematic in various ways: only one of the three sources contained data on ethnicity; one source only contained data on the sickest patients; another was from a hospital that only takes patients with diseases that they are studying, meaning there were zero “no finding” labels. Two of the sources contained no socio-economic data. Sex was self-reported in two of the sources, but assigned by clinicians in the other, which could also affect outcomes.

The advice from all this is that you should look carefully at each dataset before you combine them, to see what the result of combining them would be. In short: does it reflect real life?

Ultimately the researchers found that the algorithm was reflecting existing biases: it was much more likely to under-diagnose patients from a minority group; more likely to make mistake with female patients, the under 20s, Black people, and those from low socio-economic groups. The bias was compounded for those in more than one of those groups.

Cleaning up datasets

Once you’ve obtained your datasets from different FOI requests, you’re highly likely to find mismatches in the data that can make comparisons difficult or even impossible — but cleaning up the data can help.

For example, in spreadsheets you might discover empty fields, text in a numbers column, rows shifted, dates written in a variety of formats, different wording for the same thing, columns without titles, typos and so on.

Kay introduced a tool from Google called Refine that will solve many of the issues of messy data, and  pointed out that the ODI has a free tutorial on how to use it, which you can find here.

In the second seminar of our Using Freedom of Information for Campaigning and Advocacy series, we learned how to use information from FOI requests to create stories and further your cause. 

First, we heard the experience of two different campaign groups — Privacy International and Climate Emergency UK — in getting their stories into the public eye; this was followed by tips from freelance journalist Rosie Taylor about pitching to newspapers.

You can watch the whole video over on YouTube, or read the summary below.

Privacy International

Ilia Siatitsa

Privacy international is a UK-based organisation, working with partners around the world to research and advocate against governmental and corporate abuses of data and technology. 

They’ve used FOI requests as a source of information that feeds into campaigns and advocacy for many years. Sometimes they use a preliminary round of FOI requests to help inform a subsequent, more focused one.

Their Neighbourhood Watched campaign, which investigated the use of new surveillance technologies by the UK police, is a good example (we’ve written about it before here). Privacy International submitted fact-finding FOI requests to many police forces across the UK, asking which technologies were being used at a local level for law enforcement.

The responses enabled them to identify several different types of tech, and that there was a massive regulatory gap around this area of law enforcement, with new, invasive technologies having been introduced before any guidance was put in place.

The information they obtained via FOI has inspired a number of different actions within a wider, multi-year campaign. Privacy International first rallied their supporters to write to their local Police and Crime Commissioner to ask for more information and better regulation. 

They later launched a similar campaign around police technologies being used at protests, producing a guide to inform people attending marches, so they knew what tech was being deployed by police, and how to mitigate some of the exposure.

They also made follow-up FOI requests around the specific technologies that their first round had identified. In this second round of FOI requests, Privacy International found that the responses were all coming back as refusals, using very similar or identical language and stating that the authorities could not confirm or deny that the information was held. 

Privacy International attempted to challenge these refusals via the ICO, but they were upheld; a subsequent appeal at the Information Rights Tribunal also upheld the decision and denied a request to appeal. Undaunted by this setback, Privacy International have moved back to advocacy, sending letters to police oversight bodies to point out that every other country that has introduced these technologies to their police forces has been more transparent about them. In 2020 they published a report criticising the way the police were using mobile phone extraction (where the contents of your phone are copied, no password required), calling for reform and safeguards.

So, while Privacy International haven’t yet won the battle, they continue to fight — and this is a good example of how FOI can form the basis of a multi-year campaign with many outputs, audiences and facets.

Here are Ilia’s top tips for submitting requests — also make sure you see our previous seminar, Getting the most from FOI, for lots more advice.

Questions from the audience:

Q: Can you make a rejection into the story?
A: You can, but it depends how you want to play it: you might decide that you don’t want the refusal decision to be out in public, setting a precedent for how authorities reply to responses. Privacy International are also trying a new approach, sending a different set of questions to see if that gets them better results. 

Q: One of your tips is “format matters”: any further advice here?
A: Authorities might try to give the least information possible, using the way you’ve formatted your question to minimise what they share, so look carefully at how you’ve worded your request before sending it, and consider how it might be responded to with this mindset. 

It can be very useful to use a yes/no question: this only takes the authority moments to answer. 

Or, rather than asking for stats, try asking for the documents that those stats can be found within. Responding to this type of request takes less time for the authority, but their response will contain more information. 

Authorities often come back and say that your request needs to be narrowed down, so that can be a strategy too: start with a broad request which you’ll be happy to whittle down, knowing that you actually want the narrower information.

Climate Emergency UK

Isaac Beevor 

Climate Emergency UK (CE UK) was founded around five years ago, with the aim of collating data and information on UK councils’ climate emergency declarations. Since then they’ve worked with mySociety to create CAPE, which collates all UK councils’ Climate Action Plans, and the Council Climate Action Scorecards, which first assessed all the plans, and subsequently councils’ actual climate action.

Isaac explained that in order to gather data for the latest iteration of the Scorecards, they’d sent around 4,000 FOI requests to UK local authorities: these were all asking for data which couldn’t be obtained by other means.

These requests, which were worded very specifically, allowed CE UK to compile data on: 

• Councils’ staffing levels for climate and implementing Biodiversity Net Gain (BNG);
• The average energy efficiency (EPC) ratings of council homes and the enforcement of the Minimum Energy Efficiency Standard (MEES);
• Whether councillors and management were receiving carbon literacy training;
• Whether the councils were lobbying their devolved national government, or the UK government, for further powers or funding.

As well as giving vital information that fed into the Scorecards project, the request about EPC ratings resulted in an exclusive [paywalled] on page two of the Financial Times.

Isaac shared how CE UK went about achieving this coverage, noting that any organisation could do the same: they are a small and relatively new charity, but followed some logical steps to pitch their story, and it paid off. 

First of all, they identified three potential stories, analysing the data they’d received and looked for trends within it to see what stood out the most. They wrote the headline for each, to make it easy for a journalist to imagine the piece and the way the data could be framed. 

CE UK also considered the stories’ relevance to what was in the news at the time. The cost of living crisis was very much in the zeitgeist, and that tied in well with their data around low energy efficiency standards in council housing.

They identified which newspaper they wanted to target, and found a suitable journalist to approach, and then simply emailed them with both the headlines and the detail to back them up. Isaac advises that it is reasonable to pitch a few potential stories at one time, especially if you have such rich data that you can pull several angles out of it. 

Finally, Isaac advises that having given your framing to the journalist, you must allow them the freedom to emphasise whichever parts of the story they want to, based on your clear explanation of the data and what it is saying.

Questions from the audience:

Q: Did CE UK use EIR (Environmental Information Regulations) requests? 

A: The requests were sent with a note that the authorities should feel free to treat them as either EIR or FOI requests. In these cases, the responses would be much the same so the distinction wasn’t a great concern for CE UK.

Q: How can one identify the right journalist to approach? 

A: CE UK were guided by where they wanted the story to go, based on the reputation of the paper. Ideally you can then identify a journalist who has an interest in your subject matter. Clearly they won’t know your data as well as you do, so make sure they understand the context — be really clear in explaining what your data is about. And it’s fine to pitch to more than one journalist: give them a deadline to respond by and if they don’t, move on to another.

Q: If the paper has a paywall how do you ensure as many people as possible see the story? 

A: As well as the FT exclusive, which gave that paper the ability to print first, CE UK later sent a press release round to more general and sector press. This was also picked up by many.

Rosie Taylor, freelance journalist

Rosie specialises in health and consumer affairs, writing news and features across all national press, and she often uses FOI in work. She also works with organisations to improve their media coverage. 

Rosie began by listing five key things to consider when pitching a story to the newspapers:

1. Relevance Your story needs to be relevant to that publication’s readers. All publications have slightly different audiences with unique interests and concerns.
2. Timeliness Can you hook into topics that are being talked a lot at the moment in the news? Make sure the journalist knows ‘why now?’.
3. Ease How easy are you making it for the editor to say yes? Overworked journalists don’t have time to build up a story, so ideally you should provide a complete package. If you’re giving them data, it’s all the better if you can give them the top line but also attach the datasets. Line up experts, provide case studies and pictures — it all really helps. Look at what a finished article looks like on the page: that is everything you’re going to need.
4. Targeting Make sure you’re sending your pitch to the right journalist in the right section of the right publication. Read the publication yourself and look at the stories; become familiar with which journalists are covering certain topics.
5. Timing Pitch plenty of time ahead of when you want the story to be published, to allow time for the journalist to write it.

When considering which news outlet you are targeting, you need to look at your ultimate aim: for example, the Financial Times is read by changemakers, so it fits the needs of many campaign or advocacy groups well. Perhaps you just want more people to know about your organisation, in which case a mass readership publication would suit you better.

We tend to think of each newspaper as a single entity but in fact they can contain different sections, each with their own editor and journalists, and slightly different  interests, audiences and timescales.

It pays to know which section you are targeting, and what you want it to look like on the page. Will the story be a few paragraphs or are you hoping for a double page spread?

You might pitch your story to local papers rather than a national. In fact, many of these are syndicated across the whole country, so you can still effectively attain national coverage that way.

If you are pitching to a daily newspaper with a Sunday edition: is it a seven-day operation, or are they two separate papers? For example, you shouldn’t pitch the Times and the Sunday Times simultaneously, as they run autonomously, while the Telegraph just runs seven days a week.

Similarly, some papers have a different team producing online content, like the Daily Mail newspaper and Mail Online.

Don’t feel that you have to write off a whole publication just because you’ve had a ‘no’ from one section – if the Sunday paper says no, you can still pitch the dailies; if the Health section says ‘no’, you can try another section.

There are two ways of pitching: ‘all round’, which goes to several papers at once, or as an exclusive.

All-rounds

If you are sending your story to multiple outlets at the same time, always put an embargo on the press release (a date and time after which it can be published). This ensures that you have control over the moment of release, and journalists welcome it as it gives them the time to write the story up.

Make your embargo clear: you can put it in big red capital letters, add it to the email title, et cetera. The general convention for print is an embargo of 00:01 (one minute past midnight) for the story to appear in the following day’s papers.

Online outlets really like embargos in the middle of the day (but that timing is a nightmare for print, so pick one). For broadcast, you can time the embargo to their news bulletins.

Make sure you’re available in the run-up to the embargo, including having your experts or case studies at hand, in case there are any extra questions. If you have embargoed the story for a Monday release, that means being available on the Sunday.

An all-round is always a gamble, because it can be scuppered by a bigger news story arising; with an exclusive you can discuss timing with your journalist and they might have the flexibility to put it out at a later date if that is still appropriate. 

Exclusives

With an exclusive, you can work with one publication and focus on getting quality coverage. You can still set an embargo if the timing is important to you; you can also do a joint exclusive for print and broadcast, so long as you are transparent with all parties.

As Isaac mentioned, if one paper declines your story, take it to another — you can pretend you’re still offering it to them first!

Be very clear that you’re offering your story as an exclusive. Explain why it is relevant to them, their readers, and is timely. You should do this further ahead of time than with an all-round, especially bearing in mind that you may have to pitch to more than one outlet; also, they might want to examine your data and go into the story more deeply.

As soon as your exclusive story has been published, you can send to all the other press and see if any of them pick it up — so an exclusive doesn’t tie your story to a single paper for good.

Timing

While Rosie says one shouldn’t be too hung up on timing — it is much more important to have a strong story — it does help to know the cycles to which newspapers work. 

Sunday papers have a day off on a Monday; pool ideas on Tuesdays and most of the content has been written by the Thursday. Pitch a few weeks ahead.

Daily papers work to rough weekly cycles. They have more space on Saturdays, when they like lighter stories with good human interest; while the Monday edition is smaller but also the most serious – a good time for dryer, data-driven stories.

On Sundays, daily papers tend to have a skeleton staff, so they might be grateful of a fully-worked story. Pitch on the Wednesday of the previous week, with an embargo for Monday morning, and your story will be worked on by the Sunday staff who will be glad to have something easy to include.

Supplements and weekly sections within daily papers all have their own cycles, so just pitch a couple of weeks ahead of when you need to run.

Questions from the audience

Q: Is it better to pitch to a freelancer like Rosie, or directly to a paper?

A: There are plusses to both, but Rosie says there are several benefits to pitching to a freelancer: they can pitch to multiple publications, know all the editors and know instinctively which would be the best fit. Plus they have an incentive to get your story published, because they are paid on publication.

On the other hand, staff journalists have more weight with the papers, so it’s easier for them to get stories in.

Q: Is it best to phone or email?

A: Don’t ever phone. The journalist will see your email – but they do get a lot, so you need to make sure it is eye-catching. If you are offering an exclusive, make it very clear that this is a personal email intended for its recipient, not a generic one.

Q: What sort of case studies could we be providing?

A: Even if your story is just based on data, there will still be a human impact in the story. For example, looking at the energy standards story, you could find someone who lives in an energy inefficient home or who hasn’t got money for their bills.

Q: How do big investigations get funded? 

A: Most are funded in-house, and developed internally. You might find yourself working with the newspaper’s own team. Complex stories take time, so you need a newspaper on board to pay for your time and any equipment you need. Sometimes, organisations like the Bureau of Investigative Journalism apply for grants to help them with in-depth stories.

We are currently running a series of free, online seminars on Using Freedom of Information for Campaigning and Advocacy. 

The aim is to upskill social change organisations, particularly those working with marginalised communities and with limited capacity. Attendees will come away from these sessions with the skills and understanding they need to support their campaign or advocacy work through FOI — and, by sharing the videos, we hope that the benefits will spread further, too.

The first seminar in the series was on Getting the Most From FOI. 

Jen, mySociety’s Projects and Partnerships Manager, gave practical advice on how to shape FOI requests to maximise the chances of a full response; what the outcomes of a request might be; and how to deal with each of those outcomes.

You can watch the video here. We’ll also summarise the advice below. 

 

Framing and wording FOI requests

The more thought you put into your request before submitting it, the better the outcomes are likely to be.

Plan for your desired results

Start by thinking about what you’re going to be using the information for. 

• What are you trying to do with it? 
• So what information do you need?
• How are you going to use it? 

For example, you might need the information to feed into some research, in which case you could request base level statistics. Or you might be looking for a big headline number to shape your request around, in which case you can make a single, very tightly defined request.

Asking the right place for the right information

Consider what information is actually recorded. You can only ask an authority for information they already hold — but that doesn’t necessarily just mean documents. Videos, photos, recordings, WhatsApp messages, etc all count as information, and can all be requested.

Once you’ve narrowed down what you need, identify which authority holds the information. It’s worth doing some research here, as it might not always be the one you first think of.

Keep your request well-defined 

Consider how you word your request. If an authority has to come back and ask for further clarification, this resets the clock on the 20 working days within which they have to respond  — and it won’t begin until they’ve received your clarification. So it’s good to try and pre-empt the problems that might cause delay or rejections of your FOI request.

Don’t be afraid to be very detailed: it’s better than missing something out. You can even include what you’re not interested in, to help narrow the request down.

What time period do you want information from? State this, because otherwise the authority might assume you mean for all time, in which case it could be rejected as being too big a task and therefore taking too much staff time to compile. 

Make sure you are extremely precise. For example, when you refer to “a year”, that might be interpreted as a calendar year, financial year, or school year, so specify which you mean.

If there’s anything you already know about the information — like how the types of record you want are generally named, or where they might be found — add those details to your request. You can even send an initial FOI request to ask how the information is held at that authority, which can inform your main request.

Make sure you’re asking for something the FOI officer can easily search for. As an example, asking for data about the ‘local area’ is too vague a term. So if you want information for a particular place, specify what you mean by providing the postcode, road names or the distance from a specific point.

Doing this sort of preparation work is definitely worthwhile, especially in fast moving campaigns, as a clarification will cost you another 20 working days — ie four whole weeks.

How to make a request on WhatDoTheyKnow

Jen made a request during the seminar which you can watch step by step from the timestamp 25:52 to 37:39.

Possible outcomes to FOI requests

Once you’ve made your request, as noted, the authority has 20 working days within which they must respond. 

If there is no response:

• Nudge the authority to remind them about your FOI request – you can do this through WhatDoTheyKnow just by adding to the thread on your request page.
• If after a few days there is still no reply, you can report the matter to the Information Commissioner’s Office (the ICO) – more on this shortly.
• If you decide you no longer need the information, and there’s no benefit to it being made public, you can (and should) withdraw your request.

If you get a response:

• Legally the authority must confirm or deny whether they hold the information (or tell you that they “neither confirm nor deny”).
• In the best case scenario, the information you’ve asked for is released with no other issues.
• As mentioned above, the authority might ask you for a clarification, because they need to understand your request better.
• They might say they’re performing a Public Interest test (more on this below). There’s no legal time limit within which this must happen, though guidance from the ICO suggests it should be completed within 20 days.
• Your request might be rejected. Again, more on this below.

The Public Interest test

The Public Interest test weighs the benefits to society of releasing the information against the arguments against releasing the information.

When an authority rejects a request, it has to be because of one of several set reasons (called ‘exemptions’). Some of these exemptions require them to hold the public interest test before they can be applied.

Other exemptions are absolute, that is they can be applied with no Public Interest test. The most common of these are:

• Section 21 — the information is already in the public domain
• Section 12 — the cost limit has been exceeded, ie it will take too much time or be too costly to fulfil your request.

Some authorities include details of how they’ve applied the Public Interest test, and how they reached their conclusion, as part of a reply.

Possible rejection responses

A rejection to an FOI request may take one of several forms.

• Information not held: the authority is saying that they don’t have the information you’re asking for. 
  • If you think that’s not right, resubmit your request after doing a bit more research. Include any evidence that supports your belief that they do have the information. It might simply be a matter of wording your request less ambiguously so that they know what you’re talking about.
  • You might ask for an internal review: this means requesting that the authority’s FOI team look at the decision making process applied to your request, and reconsider whether it was valid. This can be easily done via WhatDoTheyKnow: it guides you through the steps. And it’s worth doing: our research shows that, for example, 50% of refusals from local councils get overturned at the review stage. We recommend saying why you think an internal review should be performed (and in Scotland you must). 

• You can ask the FOI team to pass your request along to the right place, or tell you who might hold the information so you can send it there.

• The information is held, but your request has been rejected: If they are declining to provide the information you’ve asked for, the authority must explain which exemption — ie, authorised reason for rejection — it is applying (see below).
  • As above, if you don’t agree with their decision, you can ask for internal review, including your reasoning.

Possible outcomes of an internal review

After the internal review, there will be one of the following outcomes:

• The exemption is overturned, and the information you asked for is released
• The exemption is partially overturned, and some of the information is released
• The exemption is upheld, and no information is released

Possible reasons for rejection (exemptions)

Understanding all the various exemptions that can be applied to an FOI request requires time and effort — but if you receive a refusal and you’re not sure what the exemption means, you can always ask the WhatDoTheyKnow team for help.

Jen mentioned a couple of the most common exemptions we see being applied:

• Section 14: Vexatious or repeated requests. If you submit a lot of requests to the same authority within a short time frame, they might be seen as unreasonable (vexatious) — or they can be considered together by the authority, and then might hit the cost limit.
• Section 8: Asking you to provide more information about yourself: for example, if you’ve made a request under the name of your organisation, the authority might ask you to provide a person’s name instead. Bear in mind that you don’t have to use your full name: you can use an initial and last name, or just your title and surname, etc.

Prejudice test

Exemptions can be ‘absolute’ or ‘qualified’ exemptions. If they are absolute, then there’s no further action necessary from the authority. For some qualified exemptions, however, they must carry out a prejudice test.

This tests whether it can be demonstrated that there is a causal link between releasing the information you’re asking for, and harm arising.

Scotland

In Scotland, they have their own points of law around FOI, and they have their own Information Commissioner.

Scottish law around FOI is covered by WhatDoTheyKnow’s Help page here. 

If the authority still rejects your request after an internal review

If you have been through the process of having your request refused, requesting an internal review and still receiving a refusal, you may wish to continue to pursue the information — especially if it’s valuable to your work and you disagree with the grounds on which the rejection has been made; or you feel there’s a strong case for the information to be in public.

At this point you can take the matter to the ICO: fill in a form on their site or send an email to their review address (ICOCasework@ico.org.uk). Include your arguments as to why the information should be released.

Before you do so, it’s a good idea to read and follow the ICO guidance. There’s a help page on WhatDoTheyKnow as well.

The possible outcomes here are:

• The ICO rules in your favour and will tell the authority to release the information. If they don’t comply they may be in contempt of court.
• The ICO rules in favour of the authority’s non-release. This does not have to be the end of the matter; if really determined, you can go to tribunal and set out why you disagree with ICO’s decision. If you get to this stage, please do get in touch with the WhatDoTheyKnow team who can give help and advice.

—

Image: Desola Lanre-Ologun

…this is mySociety’s 20th anniversary Impact Report!

This has been a very special year – mySociety’s 20th anniversary.

So we haven’t just put together our usual review of the past twelve months: this Impact Report is a special edition, covering our entire history since 2003.

We look back at our beginnings as a small group of determined coders, and trace our history through the changes our services have made, here in the UK and across the world.

Discover daring acts of (data) piracy, and learn which vandalised phone box sparked the idea for FixMyStreet. Find out how our “cheap and cheerful open web technology” has been instrumental in helping citizens tackle vital issues, from the climate emergency to human trafficking.

It’s quite the read. Sit back, grab a mince pie if you have one to hand…and enjoy! Access the Anniversary Impact report here (web), or enjoy the print-faithful PDF version, or plain text and epub formats.

And if you’re interested in our activity on the SocietyWorks side, don’t miss their own, just as engrossing, annual report: you can read that here.

Rethinking our approach to marginalised communities

Read our new report
‘Using Access to Information to support social change’

Freedom of information is for everyone: that’s something we believe, and something we’re taking concrete steps to ensure.

As we celebrate the millionth public FOI request on WhatDoTheyKnow, it’s important to consider how we can ensure the next million can benefit a broader range of people to do more towards social justice.

Historically our userbase has skewed towards those who already hold privilege, with white, well-educated, affluent males most represented across all our UK services. This demographic has fluctuated a little over time, but not as considerably as we would like.

Thanks to a grant from the Joseph Rowntree Charitable Trust, we are taking proactive steps to address this imbalance, with the primary aim of supporting marginalised groups in using FOI as one of their tools for social change. 

Jen Bramley, our Partnerships Manager, is leading on a programme to firstly research into such groups’ needs and their perception of FOI; and then identify and deliver the training that will be most effective in giving them the hands-on skills required to include FOI into their campaigning toolkits.

The first part of that activity is completed. The research confirmed some perhaps predictable points around making the very concept of FOI clearer to these communities who may not have come across the term before; and ensuring that the language and interfaces on WhatDoTheyKnow are made more accessible.

But there were other learnings that we would have come to without speaking directly to our subjects. For example, we heard that some communities’ longstanding mistrust of authority extends to the idea of having any interaction with them, even within the rights conferred by the FOI Act; and that people in more deprived demographics are more likely to access the internet via mobile phone, making it much harder to access and understand dense documents that might have been released — and all the more so when they are in bulk.

Finally, there is a desire to see more positive accounts of people using FOI without the subject having to jump over several barriers to get the information they required. While we may see such stories as an inspiring narrative encouraging us not to give up, it’s also understandable that to people approaching FOI for the first time, such stories could seem offputting and unattainable.

We’ll be using everything we learned to inform future development on WhatDoTheyKnow. These improvements will be possible thanks to the time and experience generously given by our interviewees.

You can read the full report here.