Back in March, we flagged up the ‘batch request’ feature we’d been working on for the WhatDoTheyKnow Pro service. Batch requests are now switched on for every WhatDoTheyKnow Pro subscriber, by default.
Batch enables users to send the same Freedom of Information request to several bodies at once, and we spent a substantial amount of time building and testing it because we wanted to be confident that the feature wouldn’t be abused — or if it was, that we could catch irregular behaviour.
Part of that testing has involved making the feature available to a limited number of WhatDoTheyKnow Pro subscribers, and loosely monitoring how it was used. We’re glad to say that during this four-month period, the activity was all acceptable.
However, we also realised that we should tighten up our terms and conditions to reflect our expectations around usage of Batch, and add some advice to our Help pages about making responsible and effective requests, both of which we’ve now done. We’ve also added some automatic notifications that will alert the team when multiple batch requests are made, so that we can check that everything is in order.
If you think Batch might be useful in your own work or campaigning, and you’d like to find out more about WhatDoTheyKnow Pro, you can do that here.
Image: Ankush Minda
Back in February, we postponed celebrations for the tenth anniversary of our Freedom of Information site WhatDoTheyKnow, because of extreme weather conditions. Gales and snow had shut down public transport; guests from further afield were unsure they’d make it to our London venue.
Little did we know that our rescheduled event would face its own exceptional circumstances. Not only did we find ourselves at the other end of the thermometer, with the hottest temperatures of the year thus far, but we were also competing with England playing a World Cup match.
All this being so, we were glad to see so many people turn out to help us celebrate — though it was pointed out that the Venn diagram between FOI enthusiasts and football fans might have a fairly small overlap. We’ll get our Research department on to that, at some point.
We’d decked the room with some rather unique — but meaningful — decorations: a selection of information uncovered by WhatDoTheyKnow’s users over the past decade (see photo, above), and screenshots of the many FOI sites running on our Alaveteli software around the world.
Talking of Alaveteli sites, we were delighted to welcome among our guests Andreas Pavlou who previously worked with AccessInfo, the organisation who run Europe FOI site AskTheEU, and Claude Archer from Anticor, who run Belgium’s Transparencia.be.
Claude actually drove, without incident, all the way from Brussels — only to scrape against the kerb right outside Newspeak House and get a flat tyre. But mySociety is not just a collection of weedy developers, you know. Well, ok, fair enough, until recently we were just that — but since Georgie joined our ranks a few weeks ago, it turns out that we now have a highly practical colleague who can change a wheel. And that’s just what she did.
That drama aside, the party went smoothly.
There were cakes, of course.
Then some mingling. It was great to meet many WhatDoTheyKnow users, and especially those who employ the site for their campaigns.
And on to the presentations. WhatDoTheyKnow’s Richard Taylor spoke about what it is like to be a volunteer on the site, and the kind of tasks they deal with in keeping the service available for everyone, not to mention free from litigation. You can read his talk here.
We interviewed Francis Irving, who was one of two people to suggest that mySociety build an FOI site when we had an open call for ideas — and who then went on to help build it. Much as we enjoy mySociety’s current status as an established organisation, Francis’ descriptions of our early days and ‘punk’ attitude were rather beguiling.
Finally, investigative journalist Jenna Corderoy shared her experiences of being one of the first people to try WhatDoTheyKnow Pro, our toolkit for FOI professionals and activists. In a stroke of incredible timing, she mentioned a story which she’d been working on, saying that she knew it would break soon, but it might be weeks or even a year before it did.
We woke up the next morning to hear that this very story was the BBC’s main headline for the day. Watch this space, because we’ll be asking Jenna to fill us in with some more background, and we’ll be sure to share it all here on the blog.
Oh, and in case you’re wondering… we did eventually switch the big screen over to the football, and all those Civic Tech geeks did actually get caught up in watching the penalty shoot-out decider.
I guess the Venn diagram stretched a little bit that night.
Thank you so much to everyone who came along: we hope you had as much fun as we did.
When we started building WhatDoTheyKnow Pro, our toolkit for professional users of FOI, we knew that there was one feature which would be a game-changer for such users: the ability to send a request to multiple authorities at once.
In this blog post, we examine what we wanted the tool to do, how we are guarding against abuse, and finally we’ll give a step by step walkthrough of the interface.
Investigative news stories or in-depth pieces of research often require information from a multitude of different sources. By gathering statistics or information from multiple authorities, journalists, activists and researchers can build up a previously-unseen picture, for example of how widespread a particular problem is, or where there are inequities in medical provision across the country.
It’s something that many professional users of FOI are doing already, usually with the aid of their own homemade spreadsheets on which they keep track of requests made, dates by which replies should be expected, which bodies have responded, which need chasing, and of course the information held in the responses themselves.
The standard WhatDoTheyKnow website already provides several helpful features that you just don’t get with a DIY system: it has all the right email addresses for authorities, for example; it guides you through the FOI process; and it will send you an email reminder when the deadline for response arrives — even taking bank holidays into account.
But we knew that in order for our batch request feature to woo people away from their spreadsheets, it needed to do more than those homebuilt systems, some of which have been refined over several years and work well, even if a bit clunkily, for their owners.
Power and responsibility
One important consideration was uppermost in our minds when it came to batch requests: it costs authorities time and money to respond to each request, and of course that multiplies with batch requests. We are keen to promote responsible use of FOI, so we want to fold appropriate safeguards and guidance into whatever system we build.
As mentioned, with WhatDoTheyKnow Pro we’re focusing on features that are genuinely useful for professional users of FOI, but we also want to help those users make better, more focused requests — ones that are more likely to get useful responses and see the light of day as news stories. So it was important that, in making it simple to send multiple requests, we also help users find the most suitable authorities to send their requests to.
With that in mind, here are some balances we’ve put in place:
- Users are limited in how many batch requests they can send within any one month — so there’s no chance to go too wild.
- There’s a limit to the number of authorities that can be added to a single batch: we set this to be the number of local authorities in the UK, which is a logical sector to survey in this way.
- Before users do a batch mailout, we encourage the sending of an initial request that goes to just a few authorities. This safeguard can reveal where a request is flawed, so for example, if the data you get back is not what you need or in the wrong format, you don’t have to send to the full list all over again.
- We provide advice on cost limits to encourage succinct batch requests.
- Authorities have the facility to report a request which is unsuitable for review by our administrators.
- We’re rolling out the batch request functionality gradually to vetted WhatDoTheyKnow Pro users so that we can gradually learn how people use it in practice, and course-correct as necessary..
Testing and improvements
So far, the batch feature is only available to a select group of test users, who are giving us feedback on how they’re finding it. There’s certainly nothing like having your code being used by real people to help you see where improvements might be made!
That said, it’s been a very gratifying process. With the help of our test users, we’ve seen that the batch request functionality has the potential to be immensely helpful to professional users of FOI; even genuinely game-changing. We are certain that with the sending tools, we’ve created a service that really adds value for this sector.
We’re now in the next phase, and turning our attention to improving the functionality that helps users deal with incoming responses when they come in. This already exists in a basic form, and thanks to our testers, we’ve identified which improvements we need to make. We’re already working on incorporating them. But that is definitely material for the next update — for now, let’s take a look at just how the batch request function works.
Using batch request
There are three parts to making any request, whether you’re doing it yourself or using WhatDoTheyKnow Pro:
- Creating the request
- Managing the responses
- Analysing the results
The batch request functionality builds on our super-simple FOI workflow tools for WhatDoTheyKnow Pro, extending them to make larger investigations much easier.
Creating the Request
The first step is compiling a list of authorities to send the request to. From the compose screen, you might search on a keyword (for example, ‘hospital’, ‘Birmingham’, or ‘Birmingham hospitals’) and then add the authorities you’re interested in.
Each authority is added to a recipient list and WhatDoTheyKnow Pro creates a ‘mail merge’ setup. You’ll see how many authorities you’re writing to in the compose interface.
You can then draft your request. The special `Dear [Authority name]` salutation gets automatically replaced with each of the selected authorities when you send your batch.
Finally, before sending you can choose a privacy duration.
At this point you can either go straight ahead and send your request, or save the draft and come back to it later.
Once you’ve sent your batch request, you’re going to receive a lot of replies from authorities. This is where WhatDoTheyKnow Pro’s functionality really comes into its own, keeping all that clutter out of your email inbox.
Here’s what it looks like: the first thing you’ll see is a high-level progress bar showing you the overall progress of your batch. There are three main states that help you manage the requests in the batch:
- In progress (yellow): This means that there’s no action needed by you – you’re waiting on the authority to respond with an acknowledgement or the information you’ve requested.
- Action needed (red): When a request in the batch receives a response from the authority, you’ll need to check it out. We mark the response as “action needed” for you to review and decide what to do next.
- Complete (green): Once there’s no further action needed – either you’ve got the information you asked for, the authority didn’t have the information, or they’ve refused and you don’t want to challenge them – the request moves to the ‘complete’ state, so you know you don’t need to think about it until you come to analyse the data.
Clicking the title of the batch reveals the individual requests and their progress status. From there, you can click through, read the response and update the status.
Now you’ve got all your data, it’s time to compare the results from different authorities.
Sometimes authorities will reply in the main correspondence.
Other authorities respond with one or more attachments. You can view these inline or download them to your computer.
If you’re dealing with a batch sent to lots of authorities, sometimes it’s easier to just download everything. You can download a Zip file containing all the correspondence and attachments for each request via the “Actions” menu. From there you can pull out the attachments that contain the raw data and plug the numbers or answers in to your spreadsheet so that you can compare across authorities.
You can sign up to WhatDoTheyKnow Pro today and receive 1 month free with the voucher code BLOGMARCH18. Make some requests to try out the FOI workflow tools for professionals, and get in touch to request to join the waiting list for batch access.
If your FOI requests have made the news, let us know! Send us links to your published stories and we’ll throw in an extra month of WhatDoTheyKnow Pro for free. Your stories help us improve WhatDoTheyKnow Pro.
On 22 February 2008, we posted an announcement on this very blog: “the new mySociety Freedom of Information site is now live”.
More than 450,000 requests later, WhatDoTheyKnow is marking its tenth anniversary: as part of the celebrations, we’ve put together a timeline showing how WhatDoTheyKnow has intersected with the history of FOI in the UK since we first gained our right to information in 2005. If nothing else, you may enjoy looking at the site’s rather more primitive design back in its early days.
The past decade has seen legal challenges, contributions to Parliamentary inquiries, and the development of our code for use in other countries (26 and counting). It has proved that an ambitious project can be kept going thanks to the efforts of unpaid but skilled and dedicated volunteers.
Most of all, though, it’s seen you, the general public, submitting requests for information, and sharing the responses you receive. That was always the idea, and, it turns out, it’s a pretty sound one.
The latest installation of our Alaveteli software, OPRAmachine, is an interesting new use of the platform. Rather than covering a whole country, as most of the other Alaveteli installations around the world do, it services just a single US state.
OPRAmachine, which launched in October, allows citizens to request information from state and local governmental agencies in New Jersey, under the Open Public Records Act (OPRA).
We asked Gavin Rozzi, the local journalist who has built and runs OPRAmachine, about the site and its impacts so far:
Why did you decide to set up OPRAmachine?
I developed an interest in New Jersey’s Freedom of Information law in the course of my work as an independent journalist. I created OPRAmachine because there is a void in our state for a statewide Freedom of Information portal.
Historically, New Jersey has gained a reputation as a state with excessive spending on state and local government, along with an enduring “culture” of political corruption, as defined by The New York Times.
I have found that in all too many cases, a lack of transparency and compliance with OPRA disclosure requirements has gone hand in hand with instances of government mismanagement and corruption at the state and local level, some of which have been publicised over the years.
While working in my capacity as an independent journalist, I began making extensive use of the OPRA law in order to study the activities of local governments in New Jersey. I became very familiar with the process and the how the law is effective at bringing about vitally needed transparency through the right it gives citizens to obtain public records.
A year ago, we helped Belgian group Anticor launch the Alaveteli site for Belgium, Transparencia.be, and so far it’s been an incredible success. Since launch, the site’s had almost 60,000 visitors and over 375,000 page views— that’s unusually high for an FOI site in its infancy — and they’ve even brought about a change in local FOI law. So how exactly have Anticor achieved so much in such a short time?
We chatted to Claude, a key member of the Transparencia team, to learn more, and the first thing we discovered was that the stellar visitor numbers were actually news to him. That might give us a clue as to one factor in their success: they’ve been far too busy to check their site analytics!
Fortunately we keep track of traffic on all our partners’ sites (where we have permission to do so), and we were glad to be able to pass on the good news. That done, we were keen to ask what exactly brought so many people to the site — and to see what tips other Alaveteli sites might be able to pick up so that they can enjoy the same level of success. (more…)
Code for Croatia are one of many groups around the world who have used our software Alaveteli to set up a Freedom of Information site — ImamoPravoZnati (“We have the right to know”) was launched in 2015 and has processed more than 4,000 requests.
Many organisations might count that a success and leave it there, but Code for Croatia are clearly a little more ambitious. We’ve been interested to hear about their two latest projects.
A platform for consumer complaints
The Alaveteli code was written to send FOI requests to public authorities. But in essence, it’s little more than a system for sending emails to a predetermined list of recipients, and publishing the whole thread of correspondence online.
Change that list of recipients, and you can create a whole new type of site. Reklamacije (“Complaints”) puts the process of making consumer complaints online. It’s early days as yet — the site’s still in the beta phase, during which testers are putting it through its paces. There have been messages about bank closures, insurance policies… and even the inconsistent quality of the quesadillas at a Mexican food chain.
As we’ve often mentioned here on this blog, our FixMyStreet codebase has been put to many different purposes that require map-based reporting, but as far as we’re aware this is the first non-FOI use of Alaveteli so we’ll be watching with interest. Perhaps it might give you ideas about setting up a similar service elsewhere?
Probing travel expenses
Code for Croatia have also launched a campaign asking users to request details of ministers’ travel expenses.
If that sounds familiar, you’ll be remembering that back in January, AccessInfo did much the same with EU Commissioners and their expenses on the European Union FOI site AskTheEU. We can tentatively say that they were successful, too: it’s been announced that the EU expenses will be proactively published every two months. AskTheEU say they welcome the move ‘cautiously’, so let’s see how it all pans out.
The key to both these campaigns is pre-filled requests that make it really simple for supporters to make a request to a specific politician, while ensuring that the requests aren’t duplicated.
That’s something that Gemma explained how to do in this blog post — it’s a massive benefit of the friendly global Alaveteli community that we can all share insights like this, and especially that other groups can try out initiatives that have proved successful.
Access to information is a particularly powerful tool in countries like Colombia, where corruption is high and vital peace treaties are underway.
To make accessing information easier for citizens and public authorities alike, a group of journalists in Bogotá including DataSketch, have recently set up the Freedom of Information request platform QueremosDatos (the name of which translates as “We want data/information”).
The platform uses our Alaveteli software, and we thoroughly enjoyed working with the Colombian team to set the site up with them.
We asked María Isabel Magaña, who is coordinating the QueremosDatos project, about the site and its impacts so far:
Why did you decide to set up QueremosDatos?
I first learned about Alaveteli in Spain while I was doing my Masters in Investigative Journalism. There I was introduced to the platform TuDerechoASaber.es and to the power that FOIA and transparency had. I just knew Colombia needed something like that, especially since the Congress had just approved the first law regarding this matter.
What made you choose to use Alaveteli software for your platform?
What I love about Alaveteli is how easy it is to use for both users and admins. Designing the platform and making it useful for any type of person was the most attractive feature Alaveteli had. But also, because of the people behind it. Gemma, Gareth, and so many more people were ready to help me achieve this goal despite the different time zones and how much time it took to get it up and running.
What impact do you hope the site has?
It has been almost six months since we launched the site. The impact has been great! We have helped people make 274 requests to more than 6,000 authorities. The Government has been interested in the project and has helped us get in touch with different authorities to help them learn about FOIA and the Colombian law and how to work with people through the platform. Users love it, especially journalists.
Which responses on the site have you been most excited about seeing?
My favourite response so far has been one regarding victims of the Colombian conflict. It was very exciting to get the information because of what it meant for the person who was requesting it, and because of the historical context my country is going through. I also enjoyed seeing the transformation the police had when giving their answers: at first they always sent a response asking the user to call them. After a few explanations, they’re now sending complete answers to the requests via the site.
Do you know of examples where information obtained through the site has been used?
Yes! Journalists have used it mostly in ongoing investigations regarding medicines, drug trafficking, and abortion. Students have used it for journalism classes and homework too.
What are your future plans for QueremosDatos?
We are confirming an alliance with the government to promote the site in public offices and to teach public servants about what the Right to Know is, and their responsibilities with it. This pedagogy will be replicated in universities to teach different users about their power to request information.
Many thanks to María for answering our questions. It’s been great to see the impact the site has already had on authorities and citizens alike, especially the change in behaviour by certain public authorities.
We’re really looking forward to following the project’s continuing work, and wish the team the utmost success in their quest to make Colombia a much more transparent society!
We recently explained how to use pre-written Freedom of Information requests for a campaign. We’re glad to see this being used by AskTheEU, the Alaveteli site for Europe.
Today, AskTheEU launches a campaign to request the travel expenses of EU Commissioners — and they are calling on the public to help submit a total of 168 requests.
No matter what your feeling are towards the EU (let’s not even go there), we hope that everyone is in favour of transparency. AskTheEU’s campaign follows the discovery from a request that Commission President Jean-Claude Juncker spent €63,000 on an air taxi to Turkey for the G20 summit. Naturally, they were keen to know whether this level of spending is replicated across the organisation.
After a two-year battle, AskTheEU’s parent organisation Access Info has established that the European Commission will provide information on Commissioner’s travel expenses, but only in two-month bundles.
They’ve already made a start: after submitting legal appeals and new requests, Access Info won access to a handful of documents about the travel expenses of five Commissioners: these can be seen here.
But there’s plenty more to discover, and that’s where the general public comes in. Thanks to the pre-written requests function, all the hard work is already done: it’s just a matter of picking one or two time periods and submitting the already-composed request.
Anyone can participate by going to the campaign website from today. All requests and responses will be made public on AsktheEU.
Well, we’re delighted to say we’ve been shortlisted for a grant. innovateAFRICA judges will take a few weeks to consider shortlisted applications, and winners will be announced on 30th January.
In the meantime, we thought we’d ask the project’s coordinators, Henry Maina from ARTICLE 19 East Africa and Louise Crow from mySociety, to describe the project in a bit more detail and explain why they think it’s so important.
What is the Alaveteli Professional project?
Louise: Alaveteli Professional is a new toolset that we are currently building as a companion service to our existing Alaveteli software. Alaveteli is mySociety’s open-source platform for making public freedom of information (FOI) requests to public bodies.
Alaveteli Professional will provide journalists and those who use FOI in their work with extra functionality and training to ease the process of raising, managing and interpreting FOI requests, which can be a very time consuming and overwhelming task. This is so that they can spend their valuable time on creating more high-impact journalism and research that holds public authorities to account.
Why bring the Alaveteli Professional project to Kenya?
Henry: The project will enable more Kenyan journalists to utilise one critical tool in their armoury: namely the Freedom of Information law enacted on 31st August 2016. It will also complement our earlier training of 25 journalists on the FOI law.
Louise: innovateAFRICA funding will allow us to bring our newly developed toolset to the Kenyan context. The toolset will have already been tried and tested by journalists in the UK and Czech Republic, so we’ll use examples of how these European journalists have successfully used the platform to generate stories in our trainings with Kenyan media. Simply building these tools is not, on its own, enough. For this reason, the Alaveteli Professional project in Kenya will also involve refining the tools for the Kenyan context, the training of journalists, the creation of support materials and the provision of direct assistance in making and analysing requests.
From ARTICLE 19’s experience of training Kenyan journalists on the new FOI law, how will the Alaveteli Professional project help them with their work?
Henry: ARTICLE 19 has trained journalists on the Freedom of Information laws in Rwanda, Kenya, Uganda and South Sudan. In all our past training, we created manual request protocols and follow-up required making telephone calls. The Alaveteli Professional project will help most journalists to easily file, track and share information about information requests in an easy to engage, review platform.
Why is it so important for journalists and citizens alike to hold authorities to account in Kenya?
Henry: First, journalists and citizens are keen to understand why and how their public servants and officials take decisions. Second, citizens have a right to participate in the management of public affairs and effective engagement is only possible if the citizens are well informed.
Will the project also benefit Kenyan citizens who aren’t journalists?
Louise: Yes. Providing journalists with the extra toolset requires us to first install a standard version of Alaveteli. Therefore, alongside citizens in 25 other countries in the world, Kenyan citizens will be able to use the platform to easily send requests to public authorities, or, as all responses to requests are published on the site, browse already-released information.
Citizens will also benefit even if they don’t use the site at all: they’ll benefit from news stories that expose corruption and mismanagement or missing funds and so on, and thus hold those in power to account.
What impact will the project have on Kenyan information officers/civil servants?
Henry: The project is likely to have great impact on Kenyan information officers and public officials. First, it will offer an objective platform to recognise and reward civil servants that enhance access to information as they will be able to manage requests more efficiently. Second, given the trend in questions, officers will be aware of the information that they can and should proactively disclose to lessen individual requests. Third, it will bolster ARTICLE 19’s ongoing work of training information officers that seeks to help them better understand the law and their obligations under it. Four, most of the government decisions will gain traction with citizens as there will be publicly available information on why and how such decisions were arrived at.
What lasting impact do you hope the project will achieve?
Henry: The Kenyan government will be more transparent and accountable, journalists will be more professional and their stories more credible and factual, allowing the country to entrench democratic values.
Louise: As with all our Alaveteli projects, we hope the project will amplify the power of Freedom of Information and open government, by giving a broad swathe of citizens the information they need to hold those in power to account, and to improve their own lives.
How you can help
So there you are — a little more detail on why we hope to bring Alaveteli Professional to Kenya. We hope you can see the value as much as we can! If so, and you’d like to help support the project, please do tweet with the hashtag #innovateAFRICA: every such public show of support brings us a little closer to winning the grant.