Paul Bradshaw’s name is well known to those working around data and journalism in the UK. He has authored and contributed to several books on the topic, leads an MA in Data Journalism at Birmingham City University, and acts as a consultant in BBC England’s data unit.

In mySociety’s twentieth anniversary year, we’re looking to see where we’ve had impact, and in a recent conversation with Paul we were pleased when he noted that WhatDoTheyKnow was a something of a catalyst to his work around Freedom of Information for journalism.

In 2009, Paul secured funding from Channel 4 and Screen West Midlands to set up Help Me Investigate, a platform for collaborative journalism. As it happens, that year the same source of funding supported our time-mapping service Mapumental, and Will Perrin’s hyperlocal blog project Talk About Local. The three projects were often covered in the press as harbingers of a new, digital way of doing things.

The basic principle of Paul’s platform was that the internet permits collaboration between many people, each of whom can contribute a small piece towards the labour-intensive work of investigative journalism. It’s an approach we are all very familiar with these days: it is, of course, what we now call crowdsourcing — something mySociety has made use of in many of its own projects through the years, including our own WhatDoTheyKnow Projects.  

A user, curious to get to the bottom of something, would share a central question and list out the tasks that needed to be completed in order to answer it. And of course, as often as not, some of these tasks would be the placing of FOI requests through our site, WhatDoTheyKnow.

“When I launched Help Me Investigate”, says Paul, “WhatDoTheyKnow was a major tool in our toolkit, allowing us to easily share FOI requests that others could clone or learn from.” 

Even more than that, he reckons WhatDoTheyKnow was “probably responsible both for me getting started with FOI, and for teaching others to use the FOI Act.”

Since WhatDoTheyKnow’s beginnings, the aim has been to make FOI more accessible to everyone, so this was great to hear. We know that it’s a big leap to become ‘a person who submits FOI requests’, so what does that look like in practice?

“Firstly”, says Paul, “the site reduced the barriers considerably when it came to making an FOI request: knowing where to send that request is a big mental barrier when you lack confidence navigating faceless organisations; and having examples to look at also makes a big difference in being able to imagine what one looks like.”

Once someone has become adept with the Act, we can’t ask for much more than that they pass that knowledge onto others, creating a cascading effect of individuals who understand their rights and how to use them to uncover information. Paul is an example of exactly that:

“It made it possible for me to share that knowledge with others. I’ve used it with hundreds of journalism students to introduce them to FOI: ‘copy this request, find your organisation, paste, and send’ helps get them started, and empowers students who might be otherwise feeling disempowered.”

As proof of impact goes, ‘hundreds of journalism students learning how to use FOI’ certainly seems like a good one — it means that WhatDoTheyKnow has indirectly brought countless FOI-based stories to the public. 

Paul listed some of the FOI-based investigations undertaken by users of his site — now no longer live, but visible through the Internet Archive. These include the uncovering of a £2.2 million overspend on Birmingham City Council’s website; police claims of sabotage against Climate Camp protesters; and the varying availability of hormonal contraceptives across different postcodes.

It’s been fascinating to explore Help Me Investigate‘s archived pages, and a real reminder of what people can do when they come together. We are glad that WhatDoTheyKnow has played such a key part in that, and in the training of so many future journalists.

—

Image: Ashkan Forouzani

With the aim of making large scale Freedom of Information investigations easier for community newsrooms and campaigning organisations, we’ve spent the first half of 2020 developing collaboration tools for WhatDoTheyKnow to speed up and bring others into the FOI management process.

In an initial pilot, 17 contributors saved a journalist 6.5 hours by taking on half of the work of managing responses to requests.

We’re actively looking to partner with membership-driven news organisations or impactful campaign groups to run further pilot projects to help refine the features. If that’s you, please get in touch.

FOI can be hard without dedicated tools

We know FOI can be hard work, especially when you make large batch requests that return a huge amount of data.

While our Pro tools make life easier, much of the work simply involves triaging whether you got a response or just an automated acknowledgement, and whether the authority actually released the information you requested.

After that, you then need to sift through various different formats of data, different understandings of the questions, and follow up with clarifications.

All this comes before you can start analysing the data to build up a narrative for a story.

A compelling membership proposition

News organisations are increasingly looking for sustainability by offering memberships – where you pay a monthly fee to support the organisation – instead of relying on advertising revenue to support themselves.

Memberships are still a relatively unproven and unexplored area, and organisations are still in the process of discovery over what makes someone want to pay for their news output. Is it just being able to read the stories, or do people want more involvement?

There’s evidence to suggest that members do want to get more involved.

Crowdsourcing some of the work of the FOI process from the membership presents an opportunity to help take some of the load off journalists, while also bringing members into the reporting process so that they value the final output more.

Many hands make light work

With this new functionality, once you’ve made your requests – either individually or as part of a batch – they can be added to a Project. Contributors can then be invited to the project where they are briefed on what the project is about and the tasks they can help with.

Helping to classifying responses

When you’re making FOI requests, each response to each request needs to be read to establish whether the authority has provided the information asked for – a process that is difficult to automate, given the huge variety of language that can be deployed by authorities. With large batch requests this can be a time-consuming process.

Projects creates a pool of responses that need classifying that contributors can work through to take some of the onus off the project owner.

Contributors read the original FOI request and latest response, and then classify its current status appropriately. This doesn’t take much specialist understanding of FOI, so it’s a really easy way to get lots of people to help out.

Helping to extract data

In larger FOI investigations requesters are usually looking to build up a dataset so that they can compare responses from different authorities.

This usually involves lots of spreadsheets, copy & paste, and hours of hard work.

Projects provides dedicated tools to help build this dataset by creating a pool of requests that contributors can extract data points from using structured forms.

Allowing contributors to help build up a dataset that will be used for real-life reporting and research helps them feel more directly involved and connected to the organisation, hopefully adding value to the membership proposition.

Project owners are then able to download the crowdsourced dataset to investigate, using their analysis tools of choice.

What we learned from our pilot

In our pilot project contributors took on 50% of the classification tasks, accounting for 57% of the 14.8 hours overall spent classifying, saving the journalist around 6.5 hours of the administrative work required before she could start reviewing the data releases. This is a clear indication that crowdsourcing key parts of the FOI investigation process can save a significant amount of time.

The journalist we worked with was enthusiastic about using the Projects interface again in the future, even if she wouldn’t be inviting external contributors. She expressed that it would be ideal to collaborate with interns to help sift through classifications and responses.

With an 82% conversion rate from joining to taking action and nearly 40% of contributors returning for more than one session there’s clearly an appetite from contributors to get involved and help out. The contributors we interviewed understood that by helping with menial tasks, they were allowing the journalist more time to focus on work which required specialist expertise.

A potential for global benefit

Through the Nesta Future News Fund we worked with openDemocracy to design and develop WhatDoTheyKnow Projects to support this collaboration, and ran a pilot collaborative project made up from a batch of over 800 FOI requests.

Projects is of course built into Alaveteli – the platform that powers WhatDoTheyKnow and many other FOI sites around the world, so it’s not just going to be of use in the UK, but for every jurisdiction where an Alaveteli site is utilising the Pro add-on.

—

Image: Duy Pham