With the aim of making large scale Freedom of Information investigations easier for community newsrooms and campaigning organisations, we’ve spent the first half of 2020 developing collaboration tools for WhatDoTheyKnow to speed up and bring others into the FOI management process.
In an initial pilot, 17 contributors saved a journalist 6.5 hours by taking on half of the work of managing responses to requests.
We’re actively looking to partner with membership-driven news organisations or impactful campaign groups to run further pilot projects to help refine the features. If that’s you, please get in touch.
FOI can be hard without dedicated tools
We know FOI can be hard work, especially when you make large batch requests that return a huge amount of data.
While our Pro tools make life easier, much of the work simply involves triaging whether you got a response or just an automated acknowledgement, and whether the authority actually released the information you requested.
After that, you then need to sift through various different formats of data, different understandings of the questions, and follow up with clarifications.
All this comes before you can start analysing the data to build up a narrative for a story.
A compelling membership proposition
News organisations are increasingly looking for sustainability by offering memberships – where you pay a monthly fee to support the organisation – instead of relying on advertising revenue to support themselves.
Memberships are still a relatively unproven and unexplored area, and organisations are still in the process of discovery over what makes someone want to pay for their news output. Is it just being able to read the stories, or do people want more involvement?
There’s evidence to suggest that members do want to get more involved.
Crowdsourcing some of the work of the FOI process from the membership presents an opportunity to help take some of the load off journalists, while also bringing members into the reporting process so that they value the final output more.
Many hands make light work
With this new functionality, once you’ve made your requests – either individually or as part of a batch – they can be added to a Project. Contributors can then be invited to the project where they are briefed on what the project is about and the tasks they can help with.
Helping to classifying responses
When you’re making FOI requests, each response to each request needs to be read to establish whether the authority has provided the information asked for – a process that is difficult to automate, given the huge variety of language that can be deployed by authorities. With large batch requests this can be a time-consuming process.
Projects creates a pool of responses that need classifying that contributors can work through to take some of the onus off the project owner.
Contributors read the original FOI request and latest response, and then classify its current status appropriately. This doesn’t take much specialist understanding of FOI, so it’s a really easy way to get lots of people to help out.
Helping to extract data
In larger FOI investigations requesters are usually looking to build up a dataset so that they can compare responses from different authorities.
This usually involves lots of spreadsheets, copy & paste, and hours of hard work.
Projects provides dedicated tools to help build this dataset by creating a pool of requests that contributors can extract data points from using structured forms.
Allowing contributors to help build up a dataset that will be used for real-life reporting and research helps them feel more directly involved and connected to the organisation, hopefully adding value to the membership proposition.
Project owners are then able to download the crowdsourced dataset to investigate, using their analysis tools of choice.
What we learned from our pilot
In our pilot project contributors took on 50% of the classification tasks, accounting for 57% of the 14.8 hours overall spent classifying, saving the journalist around 6.5 hours of the administrative work required before she could start reviewing the data releases. This is a clear indication that crowdsourcing key parts of the FOI investigation process can save a significant amount of time.
The journalist we worked with was enthusiastic about using the Projects interface again in the future, even if she wouldn’t be inviting external contributors. She expressed that it would be ideal to collaborate with interns to help sift through classifications and responses.
With an 82% conversion rate from joining to taking action and nearly 40% of contributors returning for more than one session there’s clearly an appetite from contributors to get involved and help out. The contributors we interviewed understood that by helping with menial tasks, they were allowing the journalist more time to focus on work which required specialist expertise.
A potential for global benefit
Through the Nesta Future News Fund we worked with openDemocracy to design and develop WhatDoTheyKnow Projects to support this collaboration, and ran a pilot collaborative project made up from a batch of over 800 FOI requests.
Projects is of course built into Alaveteli – the platform that powers WhatDoTheyKnow and many other FOI sites around the world, so it’s not just going to be of use in the UK, but for every jurisdiction where an Alaveteli site is utilising the Pro add-on.
Image: Duy Pham
We’re in the process of conducting a discovery and prototyping exercise to understand how Hackney Council currently respond to FOI requests — and also Subject Access Requests (SAR) — ahead of the new General Data Protection Regulation (GDPR), coming into force in May 2018.
The aim is to explore how we can help members of the public find the information they are looking for when attempting to submit an FOI or SAR request and subsequently, when a request is complete, making it easier to publish the non-personal responses to requests through a searchable public disclosure log.
Information should be free
When someone makes a Freedom of Information request to a public body, we like to think that the information provided, often at a not insignificant cost, should be available freely to everyone, in public.
That’s the basis of our Alaveteli software which runs in at least 28 countries, and WhatDoTheyKnow in the UK which has grown to become a vast database approaching half a million FOI requests and responses over the last 10 years, from almost 19,000 public bodies.
From our own research we know that at least 15% of all FOI requests made in the UK pass through WhatDoTheyKnow, and that rises to over 30% of all requests to some central government departments. That still leaves somewhere over 70% of all requests that feasibly could be made available to the public.
What usually happens instead is that these individual requests remain hidden away in private mailboxes and probably won’t ever see the light of day.
Our FOI strategy
In response to this we’re on a mission to expand the share of FOI requests that are catalogued and released in public through WhatDoTheyKnow. This requires us to have an understanding of the nature and source of these other requests.
Broadly speaking around one third of these remaining requests are made by commercial businesses seeking contractual information from councils, NHS trusts and the like. About a fifth are from journalists researching for stories, another slice come from students or academics, and the remainder from individuals who are often making just a single request.
Our overall strategy is pretty simple: expand the scope of WhatDoTheyKnow where we can to capture more requests directly, and create new services to cater to specific user needs.
This thinking is what led to our service for journalists and campaigners, WhatDoTheyKnow Pro. (So far we’ve been reluctant to directly cater to FOI requests made by commercial businesses, although on balance it would be better if these requests did eventually make it into the public domain.)
Working with Hackney
Through WhatDoTheyKnow we’ve been pretty firmly focused on helping citizens make good FOI requests. Some readers may remember our previous forays into this area, with WhatDoTheyKnow for Councils (since retired). We found issues with that: specifically, the assumption of immediate publication in particular placed us in position as both poacher and gamekeeper, creating a conflict of interest we weren’t comfortable with.
However when we consider the full lifecycle of creating a response to an FOI request we still believe that we can use our experience of FOI to help public bodies support better drafting of initial requests and aid the management of responding to these requests.
Which brings us back to our new collaboration with Hackney.
More specifically, we are working with Hackney to explore how we can:
- help users better submit clear and valid requests
- integrate this request form with other sources of information (including with a disclosure log) to try and help users find what they need more quickly and conveniently
- integrate with case management services so that queries are answered quickly and information published openly wherever possible
- use information from previous requests to speed the allocation of a particular request to a specific council service
- support compliance with current legislation, and pre-empt forthcoming changes (GDPR).
We’re just at the beginning of this process but we’ll be blogging and sharing more updates over the next couple of months. We will also be speaking to other public sector bodies, councils in particular, about how they manage the process of responding to FOI requests, the challenges they face, and the opportunities this offers them to proactively release more publicly available data.
Hackney are a great partner to work with. As you might be already aware they have been very active in adopting user-centred, agile methods to develop new services, they are comfortable and vocal talking about their work in public (check out the HackIT blog here) and they are especially focused on bringing their staff along with them as they evolve their approach.
We’ve got a lot to learn from them, and hopefully they can benefit from some of our experience representing the needs of citizens.
If you’re responsible for managing FOI requests or data protection in your own public sector body and you’d like to follow this project in more detail — or if you’d like to participate in some of the discovery work — then please get in touch at firstname.lastname@example.org.
Update: See part 2 of this blog post here.