Responses obtained from a widespread FOI project can be difficult to analyse, until they are sorted into neat datasets. This allows you to make valid comparisons, pull out accurate statistics and ultimately ensure your findings are meaningful.

In our third seminar within the Using Freedom of Information for Campaigning and Advocacy series, we heard from two speakers. Maya Esslemont from After Exploitation explained how to prepare for an FOI project to ensure you get the best results possible (and what to do if you don’t); and Kay Achenbach from the Open Data Institute explained the problems with ‘messy’ data, and how to fix them.

You can watch the video here, or read the detailed report below.

Preparing for an FOI project

After Exploitation is a non-profit organisation using varied data sources, including FOI requests, to track the hidden outcomes of modern slavery in the UK.

Maya explained that they often stitch together data from different sources to uncover new insights on modern slavery. She began with a case study showing some recent work they had done, using WhatDoTheyKnow to help them understand the longer term outcomes after survivors report instances of trafficking. This stood as an excellent example of how much work needs to be done before sending your requests, if you are to be sure to get the results you need.

In this case, After Exploitation were keen to understand whether there is any truth in widely-held assumptions around why human trafficking cases are dropped before they are resolved: it’s often thought that there are factors such as the survivors themselves not engaging with the police, perhaps because of a nervousness around authorities.

But what are these assumptions based upon? Actual information was not publicly available, so we wouldn’t know if cases were being dropped because of low police resource, a lack of awareness or more nuanced factors. Until the data could be gathered and analysed, the perceptions would continue, perhaps erroneously.

Before starting, After Exploitation thought carefully about the audience for their findings and their ultimate aims: in this case the audience would be mostly the media, with the aim of correcting the record if the results flew in face of what was expected; but they knew that the data would also be of use to practitioners. For example, charities could use it to see which areas to target regionally for training and other types of intervention.

They placed FOI requests with police forces across the country, making sure to ask for data using the crime codes employed by the forces: were cases dropped because of ‘lack of evidence’; did they have a status of ‘reported’ but not gone on to exist as an official crime record?

The project had a good outcome: while some requests had to go to internal review, ultimately over 80% of the forces responded with quality data. The findings were worthwhile, too: general perceptions did indeed prove to be wrong and there was no indication that ‘no suspect identified’ was a result of the victim’s lack of involvement. The resulting story was able to challenge the general narrative.

So, how can After Exploitation’s learnings be applied to the work of other organisations or campaigns?

Maya says:

• Planning, rather than analysis, is the majority of the work;
• Identify the need and purpose before you even start to pick which authorities to send requests to;
• Be clear who the audience for your findings is;
• Consult with other stakeholders to make sure your parameters are really clear.

Planning

Before you even begin, make sure your project isn’t asking for data that has already been collected and is in the public domain — this might seem obvious but it’s easy to overlook. Check other people’s FOI requests (you can do this by searching on WhatDoTheyKnow); look for reports, research, inspectorate/watchdog outputs, and data released as part of parliamentary enquiries.

That said, even if you do find previous data, there is sometimes value in requesting more up to date or more detailed information with a new set of FOI requests. If you see a national report collating data from every council for example, you could do an FOI project asking every council for a more detailed breakdown of what is happening in their region.

But before sending a batch of requests to multiple authorities, ask yourself if there is a centralised source for your data. If so, then just one FOI request might be enough: for example, homelessness data is already collected by the Depts for Housing, Levelling Up and Communities, in which case one request to them would save time for both you, and more than 300 public authorities.

Another question to ask before starting off on your project is “what is the social need?”. Does this need justify the resource you will expend? Mass FOI projects can be a bit of a time commitment, but the utility might not just be for your organisation: perhaps you can also identify a social benefit if the data would be of use to other groups, academics or journalists.

Define your intended audience: will the data you gather be of interest to them? Do you have a sense of what they want? For example, MPs often like to see localised data that applies to their constituencies. Journalists like big numbers and case studies. If you think your findings are important but might have limited appeal, you could consider including an extra question to provide details that you don’t need for your own purposes, but which could provide a hook.

Next, will the data that you gather actually be suitable for the analysis you want to perform? To avoid time-consuming mistakes, make sure the data you’ll receive is broken down in the way that you need. As an example, suppose you wanted to ask local authorities for details of programmes offered to children in different age bands: you might receive data from one council who has offerings for children ‘under 18 months’ and another ‘under two years old’ — and where units differ, they are difficult to compare and contrast. Be really precise in your wording so there’s no mismatch, especially if your request is going to a lot of authorities.

Consider, too, whether you can you get enough responses to make your data meaningful: 2,000 people is the figure believed to be representative of the population as a whole. Decide how many responses you ideally need for your purposes — and, in a scenario where not all authorities respond, the minimum you can work with.

You might want to contact other groups or organisations who could be interested in the same data, and ask if there are details that would be useful to their work.

As suggested in Maya’s case study, try to use existing measurements where you can: if you shape your requests to the methodology the authorities themselves use to collect the information, such as KPIs or their own metrics of success, these will be much easier for them to supply.

If you’re not sure what these metrics are, you can sometimes access internal guidance by googling the name of the authority plus ‘guidance’. Alternatively, submit scoping requests to a handful of authorities to ask how they measure success, etc.

At this stage it’s also useful to decide what quality of data you will include or exclude. For example, if you ask about training materials and one authority says they offer training, but don’t include the actual materials, do you include it in your figures? The more authorities you ask, the more ambiguities like this you’ll normally encounter.

Think about where and how you will log the data as it comes in. Maya recommended WhatDoTheyKnow Projects as a good tool for extracting data. Whatever you use, you should consider accessibility: can your platform be accessed by everyone you’re working with, across different communities? Especially if you are working with volunteers, it’s important to remember that not everyone has a laptop.

Also consider the security of the platform: how much this matters will depend on how sensitive the data is, but recognise that Google sheets and many other platforms store the data in the cloud where it could be more vulnerable to abuse.

After Exploitation take great pains to ensure that their data is accurate. They recommend that each response is assessed by two different people, making sure that everyone knows the criteria so they’re applied consistently; and doing regular spot checks on a handful of cases to make sure they are all logged in the same way and there’s no duplicate logging.

This is time-intensive and arduous, but if you have other stakeholders they might be able to help with the data checking: for example, knowing that they would eventually place the story with the BBC, After Exploitation were happy to hand this task over to their inhouse data checkers.

What if things go wrong?

If you’ve done all the planning suggested above, it’s less likely that your project will go awry, but even if it does, Maya says that there’s always something you can do.

No or few responses: ask yourself whether you have the capacity to chase no/late replies, and if you still don’t get a response, to refer them to the ICO. If not, consider prioritising the bodies that are most relevant to your work, eg the biggest authorities or those in areas with the densest populations; but be prepared to defend accusations that not every authority had a fair hearing unless you do them all.

If you know your requests were well worded, but you’re not getting a lot of responses — perhaps because you’re dealing with a contentious issue, or simply because the authorities cash-strapped — you could shift to measuring the types of responses you get. If authorities aren’t able to answer the question, this can often be just as revealing.

Responses that don’t tell you what you set out to understand: Consider whether there are any alternative angles in the data you do have: are there any additional themes, particularly in any free text fields? Or try a new round of requests asking for more detailed information.

Responses don’t cover the whole country: If you can’t get data from everywhere, could you narrow down to just one area and still have useful findings? Even the most basic data can set the scene for other researchers or organisations to build on: you can put it out and outline the limitations.

Results

The impact of gathering data through FOI can be massively powerful, as After Exploitation’s work shows. They have revealed the wrongful detention of thousands of potential victims of human trafficking when the government were denying it could happen; opened the debate about locking up vulnerable people; and uncovered the flawed decision making in the Home Office on modern slavery cases. It was only through FOI requests that all this information came into the public domain and was picked up by mainstream media.

—

Combining different sources of data to create datasets

Kay Achenbach is a data trainer on the Open Data Institute’s learning team; the ODI works with government and companies to create a world where data is working for everyone.

Kay shared a case study from the medical field, in which an algorithm was being designed to quickly assess high numbers of chest x-rays. The aim was to automate the process so that people identified as needing intervention would be sent to specialists right away.

The developers wanted to make sure that different demographic groups weren’t being biased against, a common issue with algorithms built on existing data which can contain previously undetected biases.

The test material was a set of x-rays from a diverse population, that had already been examined by specialists. They ran them past the algorithm to see if the diagnoses produced were the same as those made by human doctors.

The doctors’ assessments came from three different datasets which, combined, comprised data from more than 700,000 real patients. As soon as you combine datasets from different sources, you are likely to come across discrepancies which can make analysis difficult.

In this case, one dataset had diagnoses of 14 different diseases, and another had 15 — and from these, only eight overlapped. The only aspect that could for sure be compared was the “no finding” label, applied when the patient is healthy. That limitation set what the algorithm was asked to do.

Other fields were problematic in various ways: only one of the three sources contained data on ethnicity; one source only contained data on the sickest patients; another was from a hospital that only takes patients with diseases that they are studying, meaning there were zero “no finding” labels. Two of the sources contained no socio-economic data. Sex was self-reported in two of the sources, but assigned by clinicians in the other, which could also affect outcomes.

The advice from all this is that you should look carefully at each dataset before you combine them, to see what the result of combining them would be. In short: does it reflect real life?

Ultimately the researchers found that the algorithm was reflecting existing biases: it was much more likely to under-diagnose patients from a minority group; more likely to make mistake with female patients, the under 20s, Black people, and those from low socio-economic groups. The bias was compounded for those in more than one of those groups.

Cleaning up datasets

Once you’ve obtained your datasets from different FOI requests, you’re highly likely to find mismatches in the data that can make comparisons difficult or even impossible — but cleaning up the data can help.

For example, in spreadsheets you might discover empty fields, text in a numbers column, rows shifted, dates written in a variety of formats, different wording for the same thing, columns without titles, typos and so on.

Kay introduced a tool from Google called Refine that will solve many of the issues of messy data, and  pointed out that the ODI has a free tutorial on how to use it, which you can find here.

Several Olympics team members have trained and competed on the athletics track in Tooting Bec, London, and it’s a valued facility for local schools and athletics clubs alike. But by 2021, runners were finding that the surface had degraded so much that it was becoming dangerously hard to run on and training in spikes was no longer possible. After rainfall, the uneven surface would be covered in puddles: no longer was it fit for serious training or holding athletics competitions.

In fact, as campaigners pointed out, the track hadn’t been resurfaced for 36 years. Mandy Brown was one of those who campaigned for its refurbishment, and told us how Freedom of Information requests made through WhatDoTheyKnow helped provide the evidence needed to persuade Wandsworth Council to make an upgrade.  

As she explained, though, the battle wasn’t as simple as ‘one FOI request and done’. Putting in a request for what they’d thought of as a potential killer stat bore little fruit in the end — but as is so often the case, FOI supported the campaign in other, unforeseen ways.

What was that killer stat?

“Well, we wanted to know spend on the nearby Battersea Park track, which is in a more affluent ward, so we could compare it to spend on Tooting Bec track where the local population is less affluent and more diverse,” says Mandy, explaining the FOI request they submitted. 

The thinking was that they might be able to use any disparity in funding to point out the lack of fairness in funding decisions — but the truth turned out to be more complex. There might not have been money for the Tooting track, but there had been funding for a gym in the borough; and England Athletics had been joint funders on the Battersea refurbishment, so it’s not as if the council had shelled out the full amount.

It’s never a waste of time asking for information, though: bringing the facts into the open means that public debate can be based on truth and not assumptions. Plus, the mere fact that citizens have an eye on council expenditure can have an effect, as Mandy says:

“I do think repeated requests for information on the basis of the rejection of the initial application for funding played their part,” says Mandy, who submitted requests such as this one for minutes of the meeting where funding was discussed.

“Now, I believe that what really made the difference in the end was the PR campaign. This led to coverage in both local and national press. But in fact, that was boosted by information from someone else’s FOI request!

“We spotted from another request on WhatDoTheyKnow that pre-covid, more than 80,000 people used the track every year. This turned out to be an incredibly useful statistic to include in our press releases and interviews, and was picked up in every piece of coverage including Wandsworth Council’s own announcement when they finally agreed to fund the resurfacing. 

“So when all’s said and done, WhatDoTheyKnow played an important part in the overall campaign due to this publicly-available stat on track usage, and the general pressure applied by asking questions on the spend and decision-making process.” 

Thanks to Mandy for sharing this story: we believe that Freedom of Information is always useful in campaigning, for the reasons she mentions, and because public debate always benefits from an injection of factual information. 

And you never know: that FOI request you make today could be uncovering information that will inform someone else’s campaign in the future — a neat demonstration of why WhatDoTheyKnow publishes requests and responses in its permanent public archive.

 —

Image: Wandsworth Council (CC by-nc-nd/2.0)

Paul Bradshaw’s name is well known to those working around data and journalism in the UK. He has authored and contributed to several books on the topic, leads an MA in Data Journalism at Birmingham City University, and acts as a consultant in BBC England’s data unit.

In mySociety’s twentieth anniversary year, we’re looking to see where we’ve had impact, and in a recent conversation with Paul we were pleased when he noted that WhatDoTheyKnow was a something of a catalyst to his work around Freedom of Information for journalism.

In 2009, Paul secured funding from Channel 4 and Screen West Midlands to set up Help Me Investigate, a platform for collaborative journalism. As it happens, that year the same source of funding supported our time-mapping service Mapumental, and Will Perrin’s hyperlocal blog project Talk About Local. The three projects were often covered in the press as harbingers of a new, digital way of doing things.

The basic principle of Paul’s platform was that the internet permits collaboration between many people, each of whom can contribute a small piece towards the labour-intensive work of investigative journalism. It’s an approach we are all very familiar with these days: it is, of course, what we now call crowdsourcing — something mySociety has made use of in many of its own projects through the years, including our own WhatDoTheyKnow Projects.  

A user, curious to get to the bottom of something, would share a central question and list out the tasks that needed to be completed in order to answer it. And of course, as often as not, some of these tasks would be the placing of FOI requests through our site, WhatDoTheyKnow.

“When I launched Help Me Investigate”, says Paul, “WhatDoTheyKnow was a major tool in our toolkit, allowing us to easily share FOI requests that others could clone or learn from.” 

Even more than that, he reckons WhatDoTheyKnow was “probably responsible both for me getting started with FOI, and for teaching others to use the FOI Act.”

Since WhatDoTheyKnow’s beginnings, the aim has been to make FOI more accessible to everyone, so this was great to hear. We know that it’s a big leap to become ‘a person who submits FOI requests’, so what does that look like in practice?

“Firstly”, says Paul, “the site reduced the barriers considerably when it came to making an FOI request: knowing where to send that request is a big mental barrier when you lack confidence navigating faceless organisations; and having examples to look at also makes a big difference in being able to imagine what one looks like.”

Once someone has become adept with the Act, we can’t ask for much more than that they pass that knowledge onto others, creating a cascading effect of individuals who understand their rights and how to use them to uncover information. Paul is an example of exactly that:

“It made it possible for me to share that knowledge with others. I’ve used it with hundreds of journalism students to introduce them to FOI: ‘copy this request, find your organisation, paste, and send’ helps get them started, and empowers students who might be otherwise feeling disempowered.”

As proof of impact goes, ‘hundreds of journalism students learning how to use FOI’ certainly seems like a good one — it means that WhatDoTheyKnow has indirectly brought countless FOI-based stories to the public. 

Paul listed some of the FOI-based investigations undertaken by users of his site — now no longer live, but visible through the Internet Archive. These include the uncovering of a £2.2 million overspend on Birmingham City Council’s website; police claims of sabotage against Climate Camp protesters; and the varying availability of hormonal contraceptives across different postcodes.

It’s been fascinating to explore Help Me Investigate‘s archived pages, and a real reminder of what people can do when they come together. We are glad that WhatDoTheyKnow has played such a key part in that, and in the training of so many future journalists.

 

—

Image: Ashkan Forouzani

We recently ran a survey, asking the public to tell us how they’ve used our Freedom of Information site, WhatDoTheyKnow. 

To get people thinking, we put a number of prompts on social media: we asked journalists to tell us what stories they’ve uncovered through the site; and campaigners and activists to tell us how they’ve used FOI to underpin their efforts.

But WhatDoTheyKnow isn’t just useful for grand causes. Our final prompts (example below) asked: Did you use WhatDoTheyKnow just because you wanted to know something?

For all that FOI can be used to uncover vitally important information — information that changes hearts and minds, that even potentially brings down governments — there’s no doubt that it is also a useful tool for the person who simply starts wondering about something, and knows that the answer is held by our public authorities.

Such a person is Bristolian Steve Woods, who has been using WhatDoTheyKnow since its launch in 2008 — and continues to lodge a request whenever something piques his interest. From the cost of the Mayor of Bristol’s trip to a sustainable cities and towns conference in Geneva, to more details about an art piece named the Black Cloud Pavilion, and noble enquiries about the local council’s commitment to open standards, Steve has made 60 requests through WhatDoTheyKnow at the time of writing. 

We asked him to explain more about some of the more unusual ones.

The rubber bands request

“How many elastic bands – in terms of either numbers or weight – does the Royal Mail procure and/or consume per year?” was the main information that Steve asked for in this request.

He explains, “As a child in the 1950s and ’60s, I was taught not to litter, so I got really frustrated at seeing the large number of rubber bands dropped in the street by posties as they undid the bundles of letters on their rounds.”

Royal Mail obliged with a full response, and the answer may astound you: you can see the exact figures here.

Turned out that Steve wasn’t the only one interested in this figure. “The FoI request is cited on Wikipedia“, says Steve, “and resulted in a piece in the Daily Telegraph. My late brother-in-law used to take the paper and my nieces were delighted to see their uncle’s name in it.”

The Benin Bronze request

Earlier this year, Steve asked the council for updates on progress with repatriating an artefact acquired by dubious means and added to the city museum’s collection in 1935. 

“Some years ago, the local media reported that this object was to be repatriated to Nigeria”, begins Steve’s request, adding: “There have been no subsequent reports of its repatriation, so I am assuming this has still to happen.”

We asked Steve to tell us more. “Bristol City Museum & Art Gallery has a single bronze as looted from the Oba’s palace in 1897”, he says. “I learned that discussions about repatriation are still continuing.”

This request is a great example of how FOI can be used to obtain fuller background detail than is discoverable from newspaper coverage, especially once the media has moved on from a story. The response from the council provides these insights:

“Representatives from NCMM [National Commission for Museums and Monuments] were due to come to Bristol in April 2022 but in the end their busy itineraries couldn’t allow for that. They have become even busier in recent months due to the increased amount of global returns, and pledges to return, that have taken up their time. They have concentrated their efforts with larger institutions but we await direction from them on what they would like to happen next in the Bristol case.”

Meanwhile, Steve points out, “The museum has contributed to the Digital Benin project“.

The city services requests

Steve has also found FOI to be a useful tool for bringing improvements to his local area. “I have made various requests over the years about litter, fly-tipping and enforcement around environmental crimes. 

“In conjunction with working with local ward councillors and council officers, these have yielded positive results in the form of more enforcement officers – despite overall council job cuts – plus regular meetings at both ward and citywide level involving council officers, its waste management company, councillors, voluntary sector organisations and other interested parties.”

In 2009, Steve asked what proportion of his council tax was accounted for by weekly food waste collections, explaining his motivation thus: “My brown food waste recycling box was not emptied for one month while [waste management firm] SITA’s workforce was recently on strike. Via my council tax, I paid Bristol City Council for this service, which was not provided.”

The obliging response broke the figures down and came to the conclusion that each payer of council tax contributed 5p per week towards food waste collection.

The Colston request

The toppling of the Edward Colston statue and subsequent re-examination of Colston’s legacy in the city is something that is of interest to many Bristolians, and at the time of writing Steve has just received a response to his request asking what progress has been made by the council in renaming streets named after the slave trader.

“We require that all property owners on a given street provide their consent for its name to be changed, as such a move incurs administrative costs for those individuals/businesses to change legal documents etc. We have no plans to change this position,” says the council.

“We would therefore need to see overwhelming support from property owners on a given road in order to consider contacting all of them to confirm their unanimous consent to begin the process of changing their street’s name. This has not happened for any of the roads that your request mentioned.”

 —

Thanks to Steve for letting us know about his long and interesting use of FOI over the last decade and a half – we hope that he will continue to be curious about the events that unfolds around him, and will continue to use WhatDoTheyKnow to satisfy that curiosity.

When Joanne Bartley saw a bus go by with an advert for a local private primary school on its side, she knew there was something amiss. 

“It just happens that I’d recently made a Freedom of Information request through WhatDoTheyKnow, to check 11-plus pass rates at various primary schools,” she explained. “I was researching whether wealthier schools had more success than schools in poorer communities.”

Joanne lives in Kent, which unlike many counties, still assesses children with the 11-plus test. The bus advert claimed that 94% of the school’s pupils passed it, giving them access to grammar school.

“I had never seen a pass rate as high as 94%, not even in prep schools which can be mostly about 11-plus tuition,” says Joanne. 

“So I found the school’s true pass rate, which was much lower, and reported the school advert to the Advertising Standards agency. They upheld the complaint and the advert had to be withdrawn.”

What do small victories like this achieve? Joanne puts is succinctly:

“My FOI request meant that no more parents were conned into paying expensive school fees because of some dodgy statistic! Because most of the country got rid of grammar schools decades ago, the government doesn’t scrutinise the 11-plus test processes very much, and there’s little analysis of entry to grammar schools either.”

Joanne now works for a group called Comprehensive Future, which campaigns for fair school admission and an end to the 11-plus. FOI is a tool they frequently turn to.

“We use WhatDoTheyKnow an awful lot to look at social inequality in access to grammar schools. I use FOI to check how many disadvantaged pupils access the schools, to find out about appeals to these schools and more.”

The campaign’s findings frequently inform media in the education sector, national news stories, and even debates in Parliament. 

“FOI requests helped us see the numbers of disadvantaged pupils in the schools that were given £64 million by the government to expand to admit more children from poorer backgrounds. 

“We found that the policy had not worked as planned – perhaps this is one reason this fund has now been withdrawn. The story was covered in Schools Week and Comprehensive Future.

“And we had a story in the I newspaper, on the large percentages of privately educated pupils who attend grammar schools, based on this FOI request.

“We also looked at how few looked after children (children in care) attend grammar schools compared to other secondary schools and this was highlighted by a Conservative peer in a Lords debate.  

“Baroness Berridge said: At the census date last year, 68 of our grammar schools had no looked-after children at key stages 3 or 4. That is a product of not giving priority admissions and selecting on the basis of the entrance test only. If I think back to my school and remove all those children, it would have been a poorer education.

“She must have come across the figure about looked-after children on your website as we hadn’t given it to anyone at the time. So you see WhatDoTheyKnow is a useful resource for everyone!

“The FOI Act means that there’s transparency around school entry, and this is a very healthy thing. WhatDoTheyKnow is also super useful to see what other people are asking about schools and see their results.”

We were glad to hear this: Joanne’s examples make it clear that WhatDoTheyKnow helps campaigns get their causes into the national conversation. It’s free, and available to everyone, and as you can see from the examples given, it can be a very powerful tool.

Joanne agrees: “I just wanted to show the power of your site to make a difference to campaigns like ours. Freedom of Information creates openness around a problem in education that is not much talked about.

“I love WhatDoTheyKnow – it’s made a real difference to our campaign.” 

 

Have you used our Freedom of Information website, WhatDoTheyKnow, for something noteworthy? Or do you have fond memories of it that you’d like to share? Please tell us all about it here.

Perhaps you’re:

• A journalist who used FOI requests sent via the site (or the Pro service) to inform news stories
• A campaigner who changed hearts and minds with facts you uncovered through FOI
• An activist who requested information that was of use to yourself or a wider community
• A request-maker who used the site independently to discover information you wanted (or needed) to know
• A developer, volunteer or supporter who remembers the launch of the site or some other milestone in its history
• An organisation that’s used WhatDoTheyKnow’s capabilities within your own app or site

Or maybe none of these labels apply, but you’ve got something to tell us about your use, or memories, of WhatDoTheyKnow. See yesterday’s post for a look back on its 15 years so far!

Also, perhaps you have a vision for where we should go next: new features we could consider adding; partners we could work with; or ways in which we can advocate more effectively for transparency from our public authorities.

We’re all ears

It’s mySociety’s twentieth anniversary year, and like any organisation celebrating a significant milestone, we want to reflect on what’s been achieved. Just as importantly, we’re also taking the opportunity to set our course for the future.

Over the course of the year, we’ll be inviting your memories, thoughts, ideas and stories around everything we do — and we’re starting with WhatDoTheyKnow.

If you have something to share, please go ahead and fill in our form here; or if you want more detail of the sort of things we’re looking for, read on.

Looking back

WhatDoTheyKnow was launched in 2008, after a 2006 call-out to our supporters and followers to suggest what project mySociety should work on next. The chosen suggestion, originally conceived as the Freedom of Information Filer and Archive, was a response to the UK’s rights under its then-fairly-new Freedom of Information Act.

The vision has changed surprisingly little since then: as Tom Steinberg said at the time:

We think that the best way to build a top quality archive is to simultaneously build the best possible “File an FOI request” tool, and then publish both the requests and the responses made through it in the archive. From the private desire to easily file FOI requests we hope that we can generate the public benefit of an easy to use archive.

Were you following mySociety’s blog at the time of that call-out? Were you one of the first people to use WhatDoTheyKnow when it was launched? Have you been a volunteer at any time in the site’s history, and if so, has that affected any other areas of your life for the better?

Great uses of WhatDoTheyKnow

Over the years, we’ve heard about some truly gratifying and impactful uses of the service: we’ve written up many of these as case studies.

There’s the campaign to find out how much asbestos is in our schools, another to turn empty shops into premises for start-ups, and not to mention a dogged – and successful – attempt to get the West Ham stadium contract released.

More recently, you may have seen the account of one organisation’s effective campaign against surveillance cameras which seemingly send worrying data back to China.

We’re always hungry to hear stories like these, and even more so in this anniversary year. Please do fill in the form to let us know if you’ve changed the world, in big ways or small, by using WhatDoTheyKnow.

To the future

As part of our thinking around #Democracy2043, we’d love to hear which way you think transparency is headed in the UK, and what mySociety’s place is in that future.

This one requires a bit more head-scratching, but there are no wrong answers, and we’d love to hear thoughts about how we as an organisation need to adapt, or what wider society needs to put in place to allow for healthier, more transparent public institutions.

And so, if you have thoughts about any or all of these areas, please go and fill in our form now. Thanks!

2023 is a momentous year for us, marking not only mySociety’s 20th anniversary but also the 15th birthday of our flagship Freedom of Information (FOI) service, WhatDoTheyKnow.

For over a decade and a half, we’ve been working to empower journalists, activists and campaigners, researchers, and tens of thousands of curious citizens to access information from UK public authorities.

Fighting your corner

It’s hard to imagine what the UK’s FOI landscape would look like without WhatDoTheyKnow, but in the early days, we faced many important battles to establish the right to have requests responded to via our platform at all. 

And they’re not over: even today, we face fresh challenges, such as from public authorities who are putting barriers in the way of our users by refusing to answer valid requests unless these are submitted using a particular form. We are determined to continue to highlight poor practice and defend users’ information rights.

Half a million pieces of information

One of the advantages to using WhatDoTheyKnow is that it serves as a permanent archive of requests and responses. Any information that you get released using WhatDoTheyKnow is accessible to others to share and build on. From humble beginnings, there have now been over 500,000 requests that have resulted in the release of at least some information, turning this into a valuable resource.

Given the depth and breadth of the information on the site, it’s hard to pick a few examples to illustrate the impact of requests made through the service but here are some notable releases:

A 2013 request revealed the existence of the Home Office’s Interventions and Sanctions Directorate (ISD), which was responsible for overseeing the controversial hostile environment policy. Working with public and private sector partners, the ISD restricted access to benefits and services for irregular migrants, ensuring that sanctions were enforced. Four years later, the Windrush scandal exposed the devastating human consequences of this policy.

A request to the Independent Parliamentary Standards Authority (IPSA) uncovered a dispute with HMRC. This made front page headlines, after it was revealed that certain MPs had sought to utilise public funds to employ experts to complete their expense claims for them.

A 2021 request to the Science Museum revealed that the museum had signed a sponsorship agreement with Shell, where it gave an undertaking not to do or say anything that could damage the reputation of the oil company. The existence of this ‘gagging clause’ was reported by Channel 4 News and the Times among others.

Empowering others

Whilst it would be tempting to try to measure the platform’s success by the remarkable volume of information that has been released, or the myriad of news stories that have been written as a result, for me, WhatDoTheyKnow’s true strength lies in its ability to empower individuals. By simplifying and demystifying the requesting process, WhatDoTheyKnow has made it more accessible to individuals who might have otherwise never considered submitting a request for information.

The impact of WhatDoTheyKnow has stretched far beyond the United Kingdom. WhatDoTheyKnow gave rise to Alaveteli, the open-source FOI software that’s helping to open up governments across six continents. In addition to our core platform, we’ve also developed WhatDoTheyKnow Pro. Specifically tailored to journalists, this service enables users to keep their requests private while they work on their stories, before sharing the source data with the world.

None of this would have been possible without the dedication of our volunteer team, who have worked tirelessly behind the scenes to offer guidance and support to our users, as well as managing the day-to-day running of the site. We’re immensely grateful to them, and all of our donors and funders over the years, whose continued backing has ensured the ongoing success and growth of the service. 

We are excited about the next 15 years and we look forward to building on what we have already achieved to help more people to access more information more easily than ever before.

—

Image: Cottonbro Studio

Scrutiny of government ministers’ meetings with lobbyists has been boosted by the Open Access UK project from Transparency International UK, which integrates with our Freedom of Information service WhatDoTheyKnow. 

The fact that meetings between ministers, and people or organisations outside of Government have taken place is published, as required by the Ministerial Code, but typically, few details are proactively released. 

The Open Access UK service collates published declarations of meetings, and provides special links to WhatDoTheyKnow where users can find a pre-written Freedom of Information request for each meeting, asking for:

• The agenda
• The list of attendees, including details of any organisations they represented
• Copies of briefing notes and papers prepared in advance
• Notes or minutes recording  what was discussed
• Any correspondence associated with the attendees, including messages sent to follow up on the meeting

The service covers almost 90,000 ministerial meetings which have taken place since 2012, and it’s being actively maintained, with 1,500 more meetings added last month.  

To identify meetings of interest, searches can be carried out for the name of an organisation (such as a company, charity, union, or lobby group), minister, or by subject matter/policy area. Anyone interested in details of what happened at a meeting can request such information in public via WhatDoTheyKnow with a couple of clicks. You can also follow requests on any meeting, whether or not you are the person who submitted them.

It appears that meetings listed don’t only include physical ones, but online events and phone calls. Some detailed Government guidance on what should, and should not, be included in ministers’ transparency disclosures has been released via WhatDoTheyKnow (though the Government initially refused the request); more up-to date material also appears to be available, including a “pandemic-related update” which specifically covers remote meetings. 

The service does not currently cover meetings with ministers’ special advisers or civil servants where ministers are not present themselves, but Transparency International UK are inviting contact from anyone who would like to fund expanding the scope of the service to cover such meetings.  

 Responses to requests

The service has been running for some time, so everyone can see examples of how requests made through it have been responded to. 

There is a wide range of responses: in some cases the information sought has been substantively released promptly, while in other cases the responses have been less forthcoming.

As one would expect, the names of junior officials attending meetings, and involved in correspondence, are typically redacted. Often though, details of the substance of the matters discussed are also withheld. Exemptions commonly cited include those applying to “formulation of government policy” and “commercial interests”. Those exemptions are not absolute, but are subject to a public interest test: material should be released if the public interest in releasing it outweighs the interest in keeping it secret. 

Public bodies are permitted to delay a response while they consider whether the public interest lies in disclosure or not. We are concerned about the impact that such delays have on the speed of responses: we have noted examples of such delays both in responses to these requests and elsewhere on our service.  

We encourage requesters to ask for internal reviews if they are unhappy with the response to a request. 25% of internal reviews to UK central government departments result in the release of additional material, so asking for reconsideration is often worthwhile. We also provide advice on referring responses to the Information Commissioner, who is empowered to make decisions on whether information should be released or not. 

Ten things we’ve spotted in responses

(more…)

On 31 May 2022, Northern Trains Limited (Northern) wrote to us to demand that we stop publishing the salaries and job titles of the ten highest paid managers at the company. The Department for Transport had released this data in response to a request made via our Freedom of Information service, WhatDoTheyKnow. The request for removal was not only made on behalf of the company, but was also represented as being a request on behalf of the “director group”, which we have interpreted to mean those senior staff at the company whose salary data has been disclosed.

Having carefully considered our position we are continuing to publish this information.

Table: Salaries of the highest paid managers at Northern Rail Limited in £5k bands. 

Job title	Salary Banding (£)
Managing Director	245,001 – 250,000
Chief Operating Officer	210,001 – 215,000
Finance Director	165,001 – 170,000
Commercial and Customer director	150,001 – 155,000
Strategic Development director	145,001 – 150,000
Engineering Director	140,001 – 145,000
People Director	120,001 – 125,000
Regional Director	115,001 – 120,000
Programme Director	110,001 – 115,000

Source: DfT Freedom of Information release – released on at 30/05/2022

 

There is a strong public interest in favour of the release of information that helps people to understand  how resources are apportioned within an organisation. As we understand it, the Department for Transport has dealt with the FOI request in line with current best practice for transparency surrounding senior officials and high earners in the public sector, and has acted in accordance with current guidance from the Information Commissioner.

Northern Trains Limited, which operates under the ‘Northern’ brand, is wholly owned by the Department for Transport. The Government proactively publishes the exact salaries of the highest paid public sector employees as part of their regular proactive transparency releases. It would seem reasonable that Northern would also be expected to make similar information available about the salaries of its most senior staff, particularly when the salaries of senior officials at similar and related companies are already public. This includes those working for Northern’s parent company, DfT OLR Holdings Limited, Network Rail, and High Speed 2 Limited. Northern’s sister company LNER publishes information on the salaries of its directors in £5k bands on p46 of its latest annual accounts. In respect of DfT OLR Holdings Limited, the Government proactively publishes the salaries of their Chief Executive (£235,000-239,999), Group Finance Director (£220,000-£224,999) and Chair (£150,000-£154,999).

Northern routinely publishes exact salary information for junior roles on their careers website, and the material released by the Department for Transport is very similar to this. Recently, Northern has advertised that they will pay a full-time train cleaner based at Wigan £18,500 and a grade B maintenance worker based at Newton Heath £33,035 a year. We believe that Northern should have no objections to us publishing that their Managing Director receives a salary of between £245,001 and £250,000 a year.

We don’t know why the Northern Managing Director’s salary has been omitted from the data proactively published by the Cabinet Office. Perhaps they’ve been confused by the complexity of corporate structures involved, and have not looked beyond companies directly wholly owned by the Government when seeking to identify highly paid and senior public servants who should be included. We asked the Cabinet Office to comment and they shirked responsibility for the data they publish saying: 

“Although Cabinet Office compile and publish the £150k list on GOV.UK, other departments provide us with the list of salaries to be included. DfT will have sent us their senior salaries list covering its departments, agencies and non departmental public bodies. You would be best to direct your query to them, and they should be able to advise why this salary fell out of scope.”

We contacted the Department for Transport for comment but as of the time of writing we had not received a substantive response.   

We don’t know if there is an issue with the criteria for proactive publication of salaries by the Cabinet Office or if the Department for Transport have not followed the existing criteria. 

We strongly believe in preserving and promoting transparency and openness, and the accountability of those in positions of power and in maintaining a public archive of Freedom of Information requests and responses. We carefully consider all requests to remove material from our website. We balance the interests of individuals and organisations asking us to take material down with the interests in favour of continued publication.

Northern’s attempt to keep the salaries of its senior executives secret came while the threat of strike action on the railways over pay was growing. On 7 June 2022, the RMT announced 3 days of national strike action in what it called “the biggest dispute on the network since 1989.” Northern is expected to be one of the companies whose services are affected. When assessing whether to keep publishing the information, we considered the journalistic value of the data released. We expect the senior staff salaries, and the attempt to keep those salaries hidden from the public, may well be considered especially newsworthy during this period. The material that was released will help to inform the ongoing debate around pay levels in this sector.

We list DfT OLR Holdings Limited, and the three rail companies it owns on behalf of the British public, on WhatDoTheyKnow so anyone can make FOI requests to them in public. All the bodies are subject to Freedom of Information law:

• https://www.whatdotheyknow.com/body/dohl
• https://www.whatdotheyknow.com/body/northern
• https://www.whatdotheyknow.com/body/southeastern
• https://www.whatdotheyknow.com/body/lner

We thank Northern Rail for drawing our attention to this release of their senior management salary data, which might otherwise have gone largely unnoticed.

For more information on how we deal with takedown requests like this, and our legal basis for processing personal information see: https://www.whatdotheyknow.com/help/privacy#legal_basis

—

Image: Ben Garratt (Unsplash licence)

We’ve recently been considering whether we should add individual courts to WhatDoTheyKnow.com, so that users could make FOI requests to them in public. Doing so would certainly align with our wider mission of making it easy to access information from public bodies; but there are also some clear reasons against their inclusion. 

In this post we’ll examine both sides of the issue. But first, some context.

At the moment, FOI requests for information held by courts can be made via the listing on WhatDoTheyKnow for the courts service, HMCTS. Individual courts are generally not considered to be authorities in their own right, so this would mean adding bodies that are not strictly subject to FOI themselves — which is not a new concept for us: we will often list parts of public bodies separately if we think this will help our users.

Transparency is particularly important when it comes to courts, as they exercise the power of the state and their decisions can have huge impacts on individuals, organisations, the environment and society. 

In favour of listing individual courts

Further to our general principle that it is good to give access to governmental bodies serving the public, there are some more nuanced reasons to include courts in our listings:

• Requests often end up there anyway. On receipt of a request better answered by a local or individual court, HMCTS will often forward it to them, or advise the request-maker to contact the court directly themselves. The FOI process may be quicker and more efficient for all parties if requests are just sent directly to the court in question.
• It would serve an educational purpose Listing courts individually would promote the fact that FOI requests can be made for information held by courts.
• Information can be obtained from courts via FOI. Statistics, information on spending, details of room usage etc. could all be requested from courts, and we would expect such requests to be successful. Section 32 of the FOI Act exempts court records, meaning they’ll just refuse an FOI request for these, but you should be able to access other information that they hold.
• Separate requests may not trigger the cost limit Under Section 12 of the Act, authorities can refuse FOI requests if it will take them more than a certain number of working hours to provide the information. Requests made to a series of individual courts may not be aggregated for the purposes of considering the cost limits, and more information may be obtained via a series of requests made to individual courts than would be obtained via a request made to the court service centrally.

Against listing individual courts

There is really just one substantial reason against listing courts, but it is important and we give significant weight to it:

• Courts may release sensitive information When authorities respond to a request made through WhatDoTheyKnow, the information they release is published on the website. But there are rights other than FOI that give access to information from courts, eg section 5.8 of the Criminal Procedure Rules and Part 5 of The Civil Procedure Rules 1998. Court officers may consider that, due to these provisions, they are required to release information which it would be irresponsible, and sometimes illegal, to publish in response to requests made through WhatDoTheyKnow.

In conclusion

Having worked our way through these pros and cons, we conclude that listing individual courts on WhatDoTheyKnow is currently high risk, and probably not the best way to pursue greater transparency from the court system.

As in other areas, rather than improving the way requests for information are handled, proactive publication of material such as information on cases before courts, and their outcomes, would be preferable. Information which it is not appropriate to publish should be separated from other material by the courts service.   

Another approach is to make FOI requests to bodies such as the police, for material they have presented to courts, and such requests may well be successful. 

It is worth noting that there are currently three courts listed on WhatDoTheyKnow:

• Supreme Court of the United Kingdom
• The High Court of the Justiciary, which is the supreme criminal court of Scotland.
• The Judicial Committee of the Privy Council

Due to the nature of the work that these courts undertake, we believe they are lower risk listings than others. In the case of the Supreme Court they do even have their own FOI contact point and publication scheme, so should be used to responding responsibly and appropriately to FOI requests.

If you want to join the discussion of the question examined in this post, you can do so on the public “ticket” for this issue on GitHub, or you can contact the WhatDoTheyKnow team.

—

Image: Tingey law firm