FOI Fest skills session: WhatDoTheyKnow and Alaveteli
mySociety’s own Julia Cushion and Gareth Rees talk us through the basic principles of our FOI website WhatDoTheyKnow, and the open codebase that allows others to run similar websites around the world, Alaveteli – all in ten minutes.
If you would prefer to watch this session as a video, you can do so here.
Sign up for our newsletter or drop us a line.
Please fill in our short survey to help us understand more about why and how you listen!
Transcript
0:00
[Gavin Freeguard:] Welcome to FOI Fest 2026!
0:04
[Julia Cushion:] We are from mySociety, makers of WhatDoTheyKnow, but actually wanted to start us today thinking about the code that is behind WhatDoTheyKnow, which we call Alaveteli, and reminding everybody that that is an open source codebase, which means that anybody around the world can run their own Freedom of Information website, and lots of people do.
0:23
We have around 25 to 30 Freedom of Information websites around the world, and it’s an open source platform that has helped citizens make over 1.5 million Freedom of Information requests. And it was really nice to hear earlier, two people talking about making requests around the world, and yeah, our software, websites, mean that not only can you make those requests, but obviously, crucially, you can view other people’s as well.
0:49
One nice little feature that you might not have seen, we’ve recently launched this new page that we’re calling the Showcase, which is a kind of Pinterest-style scroll where you can look through all sorts of exciting examples of uses of FOI around the world, and trying to get across that principle that there is loads and loads of exciting, interesting, positive work going on through FOI, and so, yeah, you’ll get these slides afterwards, and can have a look through that as well.
1:15
Again, I did just mention this, but just to run you through the basics of Alaveteli, it is, at heart, a database of authorities who are subject to FOI, and includes the correct contact details for those authorities.
1:27
We obviously try really, really hard with our amazing team of volunteers to keep those contact details up to date, and, we try to make a nice, simple, non-threatening interface for people to help them write their request.
1:42
When you’re looking at public body websites, it might seem kind of daunting, or there’s a lot to process, and so we try to keep that more simple and give a bit of basic advice, and then, once the request is sent, the correspondence is handled through the platform, and the response is made public.
2:01
So WhatDoTheyKnow is now an archive of millions and millions, or over a million requests. It will also keep the disclosure and email log, so we can prove that their correspondence was sent, rather than public authorities claiming that they didn’t receive them. So we can show that that is the case.
2:18
And then the key benefit of this means that so many requests are published, and that you can go and through and find out what people asked for, what they got back, whether it was successful, or what rejections they received, and that is our Alaveteli codebase, as I said, it’s used in 25 different places, but I’ll hand over to Gareth to talk more specifically about WhatDoTheyKnow.
2:42
[Gareth Rees:] You’ll all probably be familiar with our UK site, WhatDoTheyKnow, which is run by a small team in mySociety and a group of really dedicated volunteers.
2:52
It was launched in 2008, a couple of years after the FOI Act, and has grown to over 1.25 million public requests on the platform, and we estimate it makes up around 15% of requests made in the UK, as well as helping with the mechanics of just sending requests.
3:08
We really try to use WhatDoTheyKnow to help improve the wider FOI system for everyone. So, for example, we write guides to help people understand FOI and how to write effective requests and understand the technical jargon around refusals, so yeah, we’re always trying to help requesters make better requests, and always interested to hear how we can make the requesting experience better for public authorities as well.
3:32
So do feel that we do encourage you to reach out and talk to us. What you might be a little less familiar with is WhatDoTheyKnow Pro. So back in 2016 we started designing additional sets of tools aimed at journalists, campaigners, researchers, or anyone who uses FOI more than just occasionally.
3:51
And some of the key challenges we were trying to solve with Pro are being able to do FOI investigations in private, making it easier to get comparable information from several authorities without having to wade through loads of different websites, find contact details, and use a random array of forms, and then make it easier to manage these requests without an overflowing inbox and a spreadsheet from hell.
4:21
Pro has some really great features on top of the basic account that helps tackle some of these challenges of using FOI more professionally. So, I’ll hand off to Julia to talk through some of the key highlights.
4:33
[Julia:] Cool. Thank you so much. So, yeah, the first one is that you can make private requests now through WhatDoTheyKnow Pro. So that means that you can embargo your request.
4:42
The default is three months, but you can go up to a year, and then if it is running out, you can then refresh that and continue to keep it embargoed, or you can, of course, publish it, which we do encourage, because we are trying to create, as we say, this public archive of requests.
4:58
And then batch requests. Yes, another one which I think is really, really helpful, and we’ve heard a lot today also about the value of creating your own dataset by asking the same question to multiple authorities, so that is one FOI request to multiple authorities, and it’s about building those datasets that have never been curated in one place before.
5:18
The features within WhatDoTheyKnow Pro allow you to do this with a click of a button, which we think is really powerful. Obviously, we do, you know, give advice to make sure that this isn’t used vexatiously, but I think it has created some really brilliant results that I will talk through in a little bit here.
5:37
And yeah, one more thing, just to chat through quickly, is this enhanced dashboard. We’ve heard a lot about the power of a super successful spreadsheet to manage your FOI requests, and that’s the sort of thing we are trying to look at here.
5:50
So it’s kind of a to-do list, and yeah, it tells you where your requests are at, and then it will also tell you within your batches what’s going on, you can classify them as whether they’ve been successful or not, and it gives you searching and filtering, so you can see where your requests are at, and that at-a-glance view of how much information you’ve got, where your total set of information is.
6:18
But one of our newest features of Pro, I’m going to hand back to Gareth. to walk through.
6:22
[Gareth:] So yeah, we shared earlier about this standardisation problem. So, even if you make a batch request, you know, even if you ask the group of authorities for the same data, it’s rare for it all to come back in the same way, you know that you might track it over like different financial year, calendar year, etc.
6:41
Or respond with these figures in really different formats, so managing the batch is quite difficult in its own right. And then you’ve got got a load of work to make that data actually usable, so something we’ve been prototyping for a while and released fully at the end of last year, and now built into Pro, is our collaboration tool around FOI investigations.
7:06
So the idea of Projects is, you take batch requests and you can bring in people, so either members of your team or people in your community, or volunteers like Climate Emergency use, to help with some key parts of that process.
7:18
So when you set up a project around a group of requests, you can create a set of structured questions for the information you’re looking to pull out from the requests, and then you get a homepage which collects all those requests together with progress for the key tests that these contributors can help with.
7:38
So test one is classifying requests, so, you know, did they release the information? Did they refuse? Am I still waiting?
7:44
And then the second stage is extraction, so pulling out that data from the request into a more structured data format, so then you can just download that as a CSV to work with, and you know, actually use that data and compare in a more consistent way,
8:03
[Julia:] Yeah. Absolutely, and it’d be nice to tell you some success stories from WhatDoTheyKnow and WhatDoTheyKnow Pro, and it’s worth noting that we have this part of the website, FOI in Action, which allows you to add a citation to your request or somebody else’s request, and we now have them on a huge variety of topics, as you can see here, all sorts of important ones, and it’s really important us to show how, as we were saying earlier, FOI works in practice, how people are making change using this.
8:30
So please do, if you’re ever using WhatDoTheyKnow, or can link in a request to a story, that’s really helpful for us and for people learning the value of FOI, and there’s a few different examples here.
8:43
This top right screenshot is from Climate Emergency UK, from the Scorecards that you heard Isaac talk about earlier, but there’s all sorts of different ones here, and we do try to track them, and we just think that there’s so much power in this idea of getting raw material but bringing the end together, and going from really what we would call fragmented data to something really powerful.
9:06
And Isaac was talking about the impact of the Scorecards earlier, so that’s another exciting project that we contribute to, and just to say, as well as making WhatDoTheyKnow and Alaveteli, and keeping those sites running, we do what we call some upstream work to try and make sure that FOI is working for everybody.
9:22
And so obviously you know Alex has been talking today, who’s written a lot of these great publications, but we do a lot of work to make sure that we can monitor how things are going.
9:33
Alex recently gave evidence in the Scottish Parliament for the new FOI Bill, which was exciting, and we’re always trying to make sure that we are reacting to how things are working in in practice, and taking the sort of stories that we see through the platform and making them known.
9:51
An example of this is, we’re not just trying to run a transactional service, but we call out bad practice when we see it, and so we have seen a pattern of some authorities refusing requests through email, and as they’re trying to encourage requesters to use specific portals that are less accessible, and we call that out.
10:11
And so, yeah, there’s an example here where we highlighted some of that, and you know, something we’re really proud of is our relationship with the ICO, where we can take these things and try to get guidance, change, and advice, so that the system works really well for everybody. I’ll hand back to Gareth.
10:31
[Gareth: ] So, a little bit about what’s next for WhatDoTheyKnow over the next year. We’re really focused on trying to get more out of the information that’s on WhatDoTheyKnow, so as we heard earlier, each request is viewed about 160 times at minimum, so you know we know there’s so much value already there on WhatDoTheyKnow.
10:51
So what we want to do is improve search, so make it easier to actually find information that is relevant to your interest and your investigations.
11:00
We’re also looking at improving the statistics that FOI can generate around the FOI process, so things like response times, exemption usage, and you know, overall performance on a quite granular level.
11:19
We’re also exploring topic extraction, so being able to group similar requests around a topic, so for example, you know, green infrastructure, or being able to identify that requests about bike lanes are similar to requests about, you know, bus routes, like greener transport, and then matching that up with the statistics as well, so we’ll be able to see, like, you know, whether those requests are more successful or not, for example.
11:56
So, yeah, we’d love to hear from you about what would be specifically helpful in these areas, because yeah, really want to make the information released from what they know as impactful as it can be.
12:09
[Julia:] Cool, and I think we are actually out of time. Yeah, please do join our mailing list, and yeah, email Gareth or I directly. We’re really keen to hear people’s feedback. We’re also on BlueSky and those sorts of places, so yeah. Thank you so much.
12:22
[Myf Nixon:] Hello to our most completist listeners. If you’re still listening, thank you so much for sticking with this right to the end. I’m Myf from mySociety, and it’s my job to put these podcasts together.
12:36
And you know what, it would be really helpful for us to know a bit more about how and why you are listening to them, so if you have a moment, please do take a look at the show notes, where you’ll find a link to a very short survey. It’d help us so much if you could fill that in. Thank you.