With the aim of making large scale Freedom of Information investigations easier for community newsrooms and campaigning organisations, we’ve spent the first half of 2020 developing collaboration tools for WhatDoTheyKnow to speed up and bring others into the FOI management process.
In an initial pilot, 17 contributors saved a journalist 6.5 hours by taking on half of the work of managing responses to requests.
We’re actively looking to partner with membership-driven news organisations or impactful campaign groups to run further pilot projects to help refine the features. If that’s you, please get in touch.
FOI can be hard without dedicated tools
We know FOI can be hard work, especially when you make large batch requests that return a huge amount of data.
While our Pro tools make life easier, much of the work simply involves triaging whether you got a response or just an automated acknowledgement, and whether the authority actually released the information you requested.
After that, you then need to sift through various different formats of data, different understandings of the questions, and follow up with clarifications.
All this comes before you can start analysing the data to build up a narrative for a story.
A compelling membership proposition
News organisations are increasingly looking for sustainability by offering memberships – where you pay a monthly fee to support the organisation – instead of relying on advertising revenue to support themselves.
Memberships are still a relatively unproven and unexplored area, and organisations are still in the process of discovery over what makes someone want to pay for their news output. Is it just being able to read the stories, or do people want more involvement?
There’s evidence to suggest that members do want to get more involved.
Crowdsourcing some of the work of the FOI process from the membership presents an opportunity to help take some of the load off journalists, while also bringing members into the reporting process so that they value the final output more.
Many hands make light work
With this new functionality, once you’ve made your requests – either individually or as part of a batch – they can be added to a Project. Contributors can then be invited to the project where they are briefed on what the project is about and the tasks they can help with.
Helping to classifying responses
When you’re making FOI requests, each response to each request needs to be read to establish whether the authority has provided the information asked for – a process that is difficult to automate, given the huge variety of language that can be deployed by authorities. With large batch requests this can be a time-consuming process.
Projects creates a pool of responses that need classifying that contributors can work through to take some of the onus off the project owner.
Contributors read the original FOI request and latest response, and then classify its current status appropriately. This doesn’t take much specialist understanding of FOI, so it’s a really easy way to get lots of people to help out.
Helping to extract data
In larger FOI investigations requesters are usually looking to build up a dataset so that they can compare responses from different authorities.
This usually involves lots of spreadsheets, copy & paste, and hours of hard work.
Projects provides dedicated tools to help build this dataset by creating a pool of requests that contributors can extract data points from using structured forms.
Allowing contributors to help build up a dataset that will be used for real-life reporting and research helps them feel more directly involved and connected to the organisation, hopefully adding value to the membership proposition.
Project owners are then able to download the crowdsourced dataset to investigate, using their analysis tools of choice.
What we learned from our pilot
In our pilot project contributors took on 50% of the classification tasks, accounting for 57% of the 14.8 hours overall spent classifying, saving the journalist around 6.5 hours of the administrative work required before she could start reviewing the data releases. This is a clear indication that crowdsourcing key parts of the FOI investigation process can save a significant amount of time.
The journalist we worked with was enthusiastic about using the Projects interface again in the future, even if she wouldn’t be inviting external contributors. She expressed that it would be ideal to collaborate with interns to help sift through classifications and responses.
With an 82% conversion rate from joining to taking action and nearly 40% of contributors returning for more than one session there’s clearly an appetite from contributors to get involved and help out. The contributors we interviewed understood that by helping with menial tasks, they were allowing the journalist more time to focus on work which required specialist expertise.
A potential for global benefit
Through the Nesta Future News Fund we worked with openDemocracy to design and develop WhatDoTheyKnow Projects to support this collaboration, and ran a pilot collaborative project made up from a batch of over 800 FOI requests.
Projects is of course built into Alaveteli – the platform that powers WhatDoTheyKnow and many other FOI sites around the world, so it’s not just going to be of use in the UK, but for every jurisdiction where an Alaveteli site is utilising the Pro add-on.
Image: Duy Pham
Over the last few months, we’ve been working with Hackney Council to design and make a Freedom of Information management system, imaginatively named FOI For Councils — and last time we left you with our pre-development thoughts. Well, now it’s up and running.
With this project, we had two main aims:
- first, to make the process really slick and easy to use for citizens;
- second, to reduce the quantity of FOI requests submitted, relieving some pressure on the Information Officers at the receiving end.
The solution we came up with achieves both those aims, and there’s one feature in particular that we’re super-excited about.
Case Management Integration
One of the development decisions taken early on was for the system to be a very lightweight layer, largely powered by the new Infreemation case management system that Hackney were in the process of commissioning.
Infreemation is targeted primarily at Information Officers, so there was no use in reinventing the wheel and building a heavy backend for our own FOI for Councils software.
Instead we built the FOI request process, using our experience in designing for citizens, and submitted the data directly to Infreemation using their API. This means that every request goes straight in to the case management system used by Information Officers, with no need for double entry; a set-up we’re very familiar with from our work integrating council systems with FixMyStreet.
Information Officers respond to the FOI request through Infreemation, and when they publish the response to Infreemation’s disclosure log, FOI for Councils can pull that response into its innovative suggestions engine, which we’ll discuss shortly.
All this means that Information Officers get to use the tools that are designed directly with them in mind, but citizens get the best experience possible for the process at hand, rather than trying to battle the typical generic forms offered by one-size-fits-all solutions.
On the user side of things we managed to reduce the process to a maximum of 6 screens for the entire process.
Throughout the whole user journey we ask for only three details: name; email address and then the actual request for information.
Each screen provides contextual help along the way, maximising the chances that the FOI request will be well-formed by the time it gets submitted.
Making the process intuitive for the people using it is a key factor in building citizens’ trust in an authority. Too often we see complex forms with terrible usability that almost seem designed to put people off exercising their rights.
So far, so good. But for us, the most interesting piece of the process is the suggestions step.
Before the citizen submits their request to the authority, we scan the text for keywords to see if anything matches the pool of already-published information.
If we find any matches, we show the top three to the citizen to hopefully answer their question before they submit it to the authority. This helps the citizen avoid a 20-day wait for information that they might be able to access immediately. If the suggestions don’t answer their question, the citizen can easily continue with their request.
Suggestions also benefit the authority, by reducing workload when requests can be answered by existing public information.
We’ve tried to make this suggestions step as unobtrusive as possible, while still adding value for the citizen and the authority.
The suggestions system is driven by two sources:
- Manually curated links to existing information
- The published answers to previous FOI requests
The curated links can be added to the suggestions pool by Information Officers where they spot patterns in the information most commonly requested, or perhaps in response to current events.
The intelligent part of the system though, is the automated suggestions.
As FOI for Councils integrates with the Infreemation case management system, we can feed the suggestion pool with the anonymised responses to previous requests where the authority has published them to the disclosure log.
By doing this the authority is making each FOI response work a little harder for them. Over time this automatic suggestion pool should help to reduce duplicate FOI requests.
FOI for Councils also analyses the number of times each suggestion is shown, clicked, and even whether the suggestion has prevented any additional FOI requests being made.
This allows Information Officers to see which information is being asked for, but where existing resources aren’t providing the information necessary to the citizen.
We’ll be keeping a keen eye on how this works out for Hackney, and we’ll be sure to report back with any insights.
As you’ll know if you read our first blog post from this project, we did originally envision a platform that would process Subject Access Requests as well as FOI. In the end this proved beyond the resources we had available for this phase of work.
For us, this has been a really instructive piece of work in showing how authorities can commission process-specific services that connect together to give everyone a better user experience.
If you’re responsible for managing FOI requests or data protection in your own public sector body and you’d like to talk about project in more detail, please get in touch at email@example.com.
We’ve just released Alaveteli 0.26! Here are some of the highlights.
Request page design update
After some research in to where people enter the site we decided to revamp the request pages to give a better first impression.
We’ve used the “action bar” pattern from the authority pages to move the request actions to a neater drop-down menu. We’ve also promoted the “follow” button to help other types of users interact with the site.
Since lots of users are entering an Alaveteli on the request pages, it might not be obvious that they too can ask for information. We’ve now made an obvious link to the new request flow from the sidebar of the request pages to emphasise this.
The correspondence bubbles have had a bit of a makeover too. Its now a lot more obvious how to link to a particular piece of correspondence, and we’ve tidied the header so that its a little clearer who’s saying what.
The listing of similar requests in the request page sidebar has been improved after observing they were useful to users.
Also in design-world we’ve added the more modern request status icons, made the search interfaces more consistent and helped prevent blank searches on the “Find an authority” page.
Admin UI Improvements
As an Alaveteli grows it can get trickier to keep an eye on everything that’s happening on the site.
We’ve now added a new comments list so that admins can catch offensive or spam comments sooner.
For the same reasons, we’ve added sorting to the users list and made banned users more obvious.
The CSV import page layout and inline documentation has also been updated.
The new statistics page adds contributor leaderboards to help admins identify users as potential volunteers, as well as a graph showing when site admins hide things to improve the transparency of the site.
Extra search powers
Conversion tracking improvements
The full list of highlights and upgrade notes for this release is in the changelog.
Thanks again to everyone who’s contributed!
We’ve just released Alaveteli 0.24! Here are some of the highlights.
We’ve added better management for censor rules in the admin interface. Previously, only request and user censor rules could be managed; now you can manage rules for authorities and global rules that get applied to everything.
We’ve added support deleting incoming messages in bulk on a request page. This is useful if you’re experiencing spam to the holding pen. You can zap them all in a couple of clicks.
Facebook is a big driver of traffic to Alavetelis. We’ve added support for a specific opengraph image for pages when shared on Facebook. You’ll need to add a custom version for your site in your theme. Take a look at the upgrade notes for more info.
Long Term Support
Alaveteli 0.24 adds support for the most current versions of Debian and Ruby. Jessie has security support until May 2018 and Long Term Support until May 2020.
The full list of highlights and upgrade notes for this release is in the changelog.
Thanks again to everyone who’s contributed!
We’ve just released Alaveteli 0.22!
Luke Bacon improved the design and accessibility of the search form.
Code quality came top of the list at AlaveteliCon 2015. This release includes contributions from James McKinney, Henare Degan, Caleb Tutty, Petter Reinholdtsen and Gorm Eriksen – all helping to clean up code, make Alaveteli work even better and make it easier to translate.
WhatDoTheyKnow’s Public Authority pages were suffering, so we took a dive in to the code around this area. Improving it had a huge impact on the page we were looking at and should have benefits across the application.
Maintenance & Security
When enabled, each request has a “Create a widget for this request” action available in the sidebar.
Any visitor can copy the iframe embed code to paste on their own website.
Waving Goodbye to the Past
You can see the full list of highlights and upgrade notes in the changelog.
Thanks again to everyone who’s contributed.