We’ve just released two new versions of Alaveteli, our open source platform for running Freedom Of Information sites — one which we’ve packed all of bug fixes, new features and performance improvements into, and one with some important technical updates. Here are some of the highlights.

Making your first request easier

mySociety’s designer Zarino has subtly redesigned the authority search page – which is the first thing most users will see when making a request – to make it easier to find the help links at a glance, making the process feel much more approachable.

Before:

After:

Encouraging better quality requests

Part of our advice to users of WhatDoTheyKnow is to keep requests concise and focused. This new feature adds a visual reminder when the request text is approaching the point of being too long for an FOI officer to answer it efficiently:

With a second, stronger warning if the request becomes longer still:

We’ve also added an experimental feature on WhatDoTheyKnow to discourage users from making requests for personal information. (As well as not being valid FOI requests, publishing the responses would result in revealing confidential information about the requester.) By asking the user to clarify what they want to ask before sending a request to authorities which attract a high level of personal information requests, we can try to redirect them to the correct official channel instead:

One-click unsubscribe from notifications

An improvement for site owners and end users: it is now possible to unsubscribe from email notifications about requests that you’re following right from your inbox! (Previously – and still the case with other sorts of email that the site sends – you would have to visit the site and log in to change your email preferences.)

Allowing people to unsubscribe with ease – if they’ve forgotten that they signed up for notifications or have created a new track by mistake – should help to cut down on the number of messages being marked as spam which should in turn help improve the site’s mail sender reputation. (It also allows admins – if they receive feedback loop notification emails from email providers – to unsubscribe users who’ve marked these emails as spam, preventing further unwelcome emails being sent.)

Technical changes

In version 0.33, as well as our our features and fixes, we’ve added support for Ubuntu releases Xenial and Bionic and withdrawn support for Debian Jessie. The full list of highlights and upgrade notes for this release is in the changelog.

Version 0.34 contains our Rails 5.0 upgrade work which we’ve released separately to allow reusers time to adapt to new minimum requirements for operating systems and Ruby versions.
We’re no longer supporting Ubuntu Trusty and have also dropped Ruby versions older than 2.3 as that’s the minimum requirement for Rails 5. Upgrade notes are available in the changelog and, as ever, we recommend updating one version at a time to make sure everything’s working smoothly and reduce the risk of missing essential upgrade steps.

Moving to Rails 5.0 will allow us to retain support for major security issues when Rails 6 is released and dropping older Ruby versions removes some key technical barriers to modernising the Alaveteli codebase and allows us to focus on improving Alaveteli Pro so that it can be reused more widely.

Thanks again to everyone who’s contributed! Special thanks to Nigel Jones and Laurent Savaëte who contributed bug fixes for version 0.33!

 

—

Image: Landscape by mike138 (CC by-nd/2.0)

Next week Gareth and I will be heading to Tunis to attend the 8th edition of RightsCon. RightsCon is the world’s leading summit on human rights in the digital age, so we’re thrilled to be hosting a session at the conference about digital Access to Information platforms with our awesome friends at MuckRock.

If you’ll also be there we’d love to talk to you about your campaigns or investigations and how using access to information platforms could help.

As Jen said in her recent blog post,  we’ll be spending time this year developing our software platform Alaveteli Pro so more people across the world have access to its digital tools that help with the sending and management of information requests.

We’d love to get feedback on this work and would love to meet organisations who are interested in setting up Alaveteli Pro instances, in order to make access to information easier for citizens in their countries. We’re also very keen to talk to individuals and organisations who are interested in collaborating on cross-border public-interest investigations and campaigns using FOI-generated data.

We’d also love to talk to RightsCon attendees who might be interested in attending our AlaveteliCon event in Oslo on 23 and 24 September, where activists, journalists, technologists and campaigners from across the world will come together to discuss Freedom of Information technologies for creating public-interest investigations and campaigns.

And of course, our Call for Proposals is currently open for our TICTeC Local conference so it’d be great to chat to people interested in presenting their work using digital innovations to help local communities and/or public authorities to foster citizen engagement, drive efficiency, and combat social and environmental problems.

If you’re interested in chatting to us in Tunis drop us an email or give us a tweet 🙂

 

 

 

 

Using WhatdoTheyKnow Pro, this project pieced together a nationwide dataset, and generated important stories at both national and local levels.

Sold from Under You, a project from the Bureau of Investigative Journalism, revealed how much publicly-owned property has been sold off across England, as a response to austerity measures. In all, TBIJ discovered that over 12,000 buildings and pieces of land have been disposed of, bringing councils revenue of £9.1 billion — some of which has been spent on staff redundancies.

In collaboration with HuffPost, the findings were presented in the form of an interactive map which allows users to explore sales in their own area.

The investigation required a significant amount of data collection via FOI requests to 353 councils, work which was aided by WhatDoTheyKnow Pro. More than 150 people across the UK, including local journalists, took part in the collaborative investigation. As well as HuffPost’s coverage, stories were run in regional news outlets across the country. The project has now been shortlisted for the Data Journalism awards.

We spoke to Gareth Davies from TBIJ to understand how the organisation approached this ambitious project, and what part WhatDoTheyKnow Pro played in it. Here’s what he told us:

---

“The Bureau has been investigating the local government funding crisis in the UK for the last 18 months. The initial part of this particular investigation focused on the overall financial health of local authorities and used data to determine which were under the most pressure. We then wanted to look at the impact of the funding crisis so teamed up with Hazel Sheffield and her Far Nearer project to look at the public spaces that were being lost as a result.

“At the start of the investigation we undertook a research period to determine what local authorities are required to publish about the buildings and land they own, and how many of them were adhering to those rules.

“We discovered that while councils have to publish annual lists of the assets they own, this does not include vital information such as who assets were bought from or sold to and the prices paid.

“Also, two thirds of councils update the same spreadsheet each year, meaning change over time is lost. As a result it became apparent that FOI would be required to obtain the information we were interested in. FOI is a tool we have used for a number of stories, particularly those produced by our Bureau Local team.

“The information we wanted could be divided into two groups: what assets councils were buying and selling, and what they were doing with the money raised when an asset is sold. The research period showed we would need FOI to obtain this data.”

More than 700 FOI requests

“To reduce the risk of requests being refused for exceeding the cost/time limit, we needed to submit two separate requests to each of the 353 local authorities in England.

“Previously I had submitted and managed bulk FOI requests via email. However, staying on top of more than 700 requests would have proven very challenging. I was aware of the WhatDoTheyKnow Pro platform but hadn’t used it before, so thought this would be the ideal opportunity to test it out.

I don’t think I would have achieved that without WhatDoTheyKnow Pro

“It was useful to have up-to-date contact details for each authority and to be able to send the FOI requests in one go. But probably the most useful feature was the way in which WhatDoTheyKnow Pro tracks the status of each request and shows you when the public body in question has exceeded the statutory time limit. This made it a lot easier to stay on top of which councils needed to be chased and when I needed to do it.

“Managing so many FOI requests was still challenging and very time consuming but it would have been much harder by email. The first batch of requests had a success rate of more than 95% and the other (which was more detailed)  was around 85%.

“I don’t think I would have achieved that without WhatDoTheyKnow Pro and, as a result, the investigation and interactive map we created would not have been as comprehensive.”

Refining the requests

While councils have to publish annual lists of the assets they own, this does not include vital information such as who assets were bought from or sold to and the prices paid

“I sent requests to one of each type of local authority (London borough, metropolitan borough, unitary, county and district) to test what, if any, information councils would provide. The fact that all of those requests were successful meant I had confidence when submitting the batch requests.

“It also allowed me to include additional information in the bulk requests, because some of the test councils erroneously withheld, under Section 40, the identities of companies. As a result I added a note to the request highlighting that this would not be a correct application of that exemption.

“As each response came in I recorded them in two separate spreadsheets — one showing what assets had been bought/sold and another containing information about how the money raised from asset sales had been used. Gradually we built a comprehensive picture of what was happening with public spaces, and that was crucial for our story.”

Bringing about change

There have been tangible results from this investigation.

“The government launched an investigation into the sale of assets by Peterborough Council as a result of this particular story, focusing on that area.

“We submitted our findings to an inquiry currently being held by the Communities and Local Government select committee and were mentioned by name during the first day of oral hearings.

“And last month the Public Accounts Committee announced it would hold a similar inquiry into the sale of public land. Several councils halted their property investment policies after our coverage revealed how much they had borrowed to fund the purchases.”

—

Thank you very much to Gareth Davies for talking to us about the Sold From Under You project.

Find out more about WhatDoTheyKnow Pro.

—

Image: Daniel von Appen

WhatDoTheyKnow Pro, mySociety’s subscription service offering extra tools for journalists and other professional users of FOI, has been running in the UK for just about two years.

During that time we’ve launched, worked closely with users to refine the service, and — happily — watched it play a vital part in the making of several important data-driven news stories, on topics as diverse as Brexit campaign funding and the results of austerity cuts on councils. Journalists, in particular, have appreciated tools such as the ability to send and manage bulk requests to multiple authorities; and the embargo tool that keeps requests and responses hidden until the story has been published.

Now, thanks to support from Adessium Foundation, we are able to bring the same benefits to countries across Europe, and — we hope — some additional synergies that will be borne of organisations working across boundaries. The same functionality that extends WhatDoTheyKnow into the Pro version will be available to FOI sites run on the Alaveteli platform, under the name Alaveteli Pro.

The ultimate aim is to enable journalists, campaigners and citizens in Europe to make greater and more effective use of their right to access information; and in particular to generate public interest stories and campaigns that will hold power to account.

We’ll be focusing on three areas in order to achieve this aim:

• We’ll give selected existing Alaveteli sites in Europe the technical help they need to upgrade to the Pro version;
• We’ll be helping organisations in three new European jurisdictions to launch brand new Alaveteli sites, making access to information easier for citizens in these countries. The first site will be launched by VVOJ from the Netherlands.
• We’ll encourage cross-border collaborations between journalists and organisations using the sites (both the existing ones and the new ones) to investigate stories that span more than one EU country.

So watch this space: we’ll be sure to keep you posted as the work progresses. The planned start date is next month, and the project is set to run for three years.

We’re looking forward to sharing stories resulting from this initiative once they start rolling out, and supporting the incredible work that journalists do in putting them together.

—

Image: Emiliano Vittoriosi

We’re delighted to be hosting the third AlaveteliCon, our Freedom of Information (FOI) technologies conference, on 23 and 24 September 2019 in Oslo, Norway.

A few days ahead of International Right to Know Day (28 September), AlaveteliCon will bring together activists, journalists, technologists and campaigners from across the world who use Freedom of Information laws, data and technologies to create public-interest investigations and campaigns.

This time we’ll be including a focus on catalysing collaborations on cross-border European public interest investigations and campaigns, in order to strengthen the use of FOI across the region. This focus is in part due to our latest Transparency project, funded by Adessium Foundation. This Transparency project seeks to enable journalists, campaigners and citizens in Europe to make greater and more effective use of their right to access information, in particular to generate public interest stories and campaigns that hold power to account.

Here are some of the other things AlaveteliCon attendees will get up to this time:

• Exchanging insights and ideas on how to run and improve open-source Freedom of Information platforms such as WhatDoTheyKnow, MuckRock and FragDenStaat
• Hearing how journalists and campaigners around the world have used FOI to power high-profile investigations and campaigns
• Making connections with journalists and campaigners who would like to collaborate on cross-border investigations that hold governments to account
• Learning about data sources available on FOI platforms around the world, waiting to be analysed and turned into public-interest stories
• Hearing tips from FOI activists on getting governments to release information in their countries including going to court
• Contributing to the further development of mySociety’s open-source FOI toolkit for journalists, Alaveteli Pro, which helps journalists and campaigners to manage their FOI investigations
• Connecting to a global network of FOI experts and advocates
• Working together to plan potential joint activities to celebrate International Right to Know Day on 28th September 2019

Apply to attend

As spaces at AlaveteliCon are limited, we’re asking those wishing to come to fill out this application form to tell us what unique perspective they can bring that’s relevant to the above conference themes/topics.

Are you a journalist or campaigner who uses FOI for investigations/campaigns? Do you want to collaborate on FOI-generated investigations? Or interested in setting up an FOI platform in your country? Or want to learn how to use FOI to its fullest potential? Or interested in any of the above topics? If so, then please submit your application by 2nd September at the very latest.

We’ll let you know if you’ve been allocated a place by Friday 6th September at the latest. However, we will endeavour to reply to you sooner than this.

AlaveteliCon is free to attend, but delegates will be required to arrange their own travel, accommodation and costs in Oslo.

Many thanks to Adessium Foundation and NUUG Foundation for supporting AlaveteliCon.

 

 

We’ve just released version 0.32 of Alaveteli, our open source platform for running Freedom Of Information sites. Here are some of the highlights.

Making correspondence threads easier to navigate

Thanks to our designers, it’s now possible to collapse individual messages in a correspondence thread in order to focus on just the parts you’re trying to read. Plus you can quickly collapse (or expand) all the messages in the thread using the “Collapse all” and “Expand all” links from the “Actions” menu.

Alaveteli Pro users gain the additional benefit of a redesigned sidebar which allows for easier navigation of lengthy correspondence and avoids having to scroll to the top of the request thread to update its status. See Martin’s full explanation here.

Better password security

We’ve started enforcing stricter password length constraints wherever a password is set or updated to help users keep their accounts secure. And we’re also using a stronger encryption method for storing password data, using bcrypt rather than the older SHA1 algorithm to obscure the actual password. (Be sure to run the rake task documented in the release upgrade notes to upgrade secure password storage for all existing users.)

You can read more about what this does and why it’s important if you’re interested in the technical details behind this upgrade.

Authorities not subject to FOI law

We’ve adopted WhatDoTheyKnow’s foi_no tag for authorities to indicate that although the authority is listed on the site, it is not legally subject to FOI law. This could be for advocacy purposes – if it’s felt an authority should be covered by legislation – or where the authority has agreed to respond on a voluntary basis.

Adding the foi_no tag now causes an extra message to appear under the authority’s name on their page and on all related requests, and removes language about legal responsibilities to reply from the messages sent to users.

To improve the UI, we’ve made a similar change for authorities with the eir_only tag to make it clearer that such authorities are only accepting requests about the environment.

(Don’t worry admins, you don’t need to remember all this – we’ve updated the documentation on the edit page to reflect the new functionality!)

Improvements for site admins

We’ve made it easier for admins to ban users who sign up to post spam links in their profile. There’s now a “Ban for spamming” button which is available on the user edit page or as soon as you expand the user’s details in the listing rather than having to manually edit user metadata.

We’ve also made it harder to leave requests flagged as vexatious (or “not_foi”) in an inconsistent state. Previously the site just assumed that vexatious requests would always be hidden. Now the admin interface enforces the hiding of vexatious requests by showing warnings when a request is set as vexatious while it’s visible on the site, and prevents the updated request from being saved until a valid state is selected.

Announcements

And last but not least – introducing the new Announcements feature!

Easier popup banner management

Site admins will be relieved to hear that they can now update the popup banner message on the site without needing to schedule developer time.

This feature supports multi-language sites so if you set the announcement for your main (default) language, it will appear across all language versions that you have not added a specific translation for.

Admin-only announcements

You can set announcements that will only be seen by fellow administrators when they visit the summary page. (If you’re running a Pro site, you can also have announcements that will only be seen by your Pro admins.)

Pro announcements

Announcements for Pro users appear as a carousel at the top of their dashboard. So far we’ve used it on WhatDoTheyKnow Pro to publicise new features, offer discount codes, and encourage people to share their published stories with us.

---

The full list of highlights and upgrade notes for this release is in the changelog.

Thanks again to everyone who’s contributed!

Last year, we launched WhatDoTheyKnow Pro, our service for journalists and other professional users of Freedom of Information.

As it’s a new venture, we’re keen to track whether it’s achieving everything we’d hoped for when first planning the service. One of the targets which we set, as a measure of success, is the number of impactful press stories generated by its use.

What might count as impactful? Well, that’s obviously up for debate, but loosely we’d say that we’re looking for news stories that have a wide readership, and uncover previously unknown facts, offer new insights, or bring about change.

Stories in the news

A couple of recent stories, generated through WhatDoTheyKnow Pro, have ticked at least a few of those boxes. At mySociety, we keep a position of political neutrality — our services are available to everyone of any persuasion, and we don’t campaign on any political issue — so we present these stories not to comment on their substance, but to note that they certainly fit the criteria above.

Brexit is clearly one of the most vital stories of our day, here in the UK, and while many might feel that we’ve had a surfeit of commentary on the issue, we can only benefit from understanding the facts.

One of WhatDoTheyKnow Pro’s earliest users, Jenna Corderoy, broke two important stories on that topic. First, that UK parliamentary standards watchdog IPSA is investigating Jacob Rees Mogg’s hard Brexit European Research Group over their second bank account and ‘informal governance structure’. This also ran in the Daily Mail.

Secondly (and this progresses a previous story about expenses – also uncovered thanks to Jenna’s use of WhatDoTheyKnow Pro) there is the widely-reported story that the Electoral Commission had misinterpreted laws around campaigning expenses, allowing Vote Leave to overspend. This was picked up by the BBC and Guardian, among many others.

No matter which side of the Brexit debate you support, hopefully you will agree that it benefits society as a whole to have the facts out in the open.

From FOI request to the national news

As a last thought: it’s interesting to us to see how a story grows from one or more FOI requests into something that hits the national news platforms. We think these stories were broken using a tried and true method that goes like this:

• As a journalist, campaigner or researcher, you might be investigating a topic. Perhaps you’ve heard a rumour that you’re hoping to substantiate, or perhaps you’re inquiring more deeply into a story that’s already in the air. Using FOI, you can retrieve facts and figures to bolster your investigation.
• Once you have a story, you can publish it in a smaller publication like Open Democracy, testing the water to see if it gains any traction.
• If the story is well-received there, it’s easy to approach larger outlets with the proof (in the form of FOI responses) underpinning it.

If you’re a journalist or you use FOI in your professional life and you’d like to try WhatDoTheyKnow Professional for yourself, then head over to whatdotheyknow.com/pro. Put in the code WELCOME18 when you sign up, and you’ll get 25% off your first month’s subscription.

—

Image: Roman Kraft

WhatDoTheyKnow Pro is our Freedom of Information service for journalists, and campaigners, and we’ve recently rolled out some major changes to the request sidebar to make reading, navigating, and classifying Pro requests a lot easier.

Since the very first Alpha version of WhatDoTheyKnow Pro we’ve been receiving feedback from our users, which we have been feeding directly into our future development plans. The sidebar changes are the first round of changes that have come as result of direct Pro user feedback, and there will be more to follow.

Current state

In WhatDoTheyKnow a member of the public can send a Freedom of Information request to an authority, which they receive in the form of an email. The request, as well as any replies or follow up from the requester, or the authority, are published on WhatDoTheyKnow. If the request is made by a Pro user, they have the added option of making a request private for a limited time.

In the request process we observed the following:

• A new response from an authority goes to the bottom of the thread (the bottom of the page)
• The user interface for updating the status of a request is located at the top of the page (a request’s’ status is a way of keeping track of where it is in the request process – for example ‘awaiting response’, ‘needs clarification’, or ‘refused’)
• A longstanding or complicated request will often consist of many, many messages. So scrolling to the bottom of the page to read the most recent response, then back to the top to update its status involves a lot of interaction that we can remove.

We can’t say for sure that a user will always be at the bottom of the request thread when updating the status of a request, but we can safely assume they are sometimes.

Goals

For these changes we set ourselves the following goals:

• Speed up the process of updating the status of requests
• Improve the experience of navigating requests with a long history.

In previous research we established that a typical workflow for dealing with request responses is:

Get reply → Read reply → Take action (typically reply, or update the status of the request)

It’s a short, three step process, but a busy user catching up with a backlog may do this hundreds of times a day, so if we can optimise this workflow we can save a lot of time and frustration.

So what have we done?

For desktop users we’ve made the sidebar controls (where the ‘update status’ button is) “sticky”, so it will follow you as you scroll up and down the page, meaning you can update the request status from any position on the page. This really helps as requests get longer, as you no longer need to scroll back to the top to classify the latest response.

We’ve added new message navigation buttons. This is to enable you to move through a request thread message-by-message by clicking the up and down arrow buttons, or using the arrow keys on your keyboard. We’ve also added a counter so that it’s easier to see where you are in the list, and to go back and forth to specific messages.

We’ve also taken this opportunity to make some key information about the privacy of your request visible at all times (this was previously hidden behind a click), and to tweak the design of the sidebar – making it easier to read and removing some visual noise.

More to do

We’re looking at a way to add similar functionality to mobiles and other small screen devices. As screen space is limited it will require a separate design process.

We’re aware that the problems we’re trying to solve aren’t unique to our Pro customers, so if the features work, and are well received, we’ll be making a similar feature available to all WhatDoTheyKnow users in the future.

Keeping in mind that as our public users have different needs to our Pro users there are some design challenges to overcome beforehand. For example – the public request page has more features to help less-frequent users, because we’re keen to ensure that everyone can participate in the FOI process, not just experts. Conversely, Pro users are by their very nature more likely to require less guidance. We’re going to need to do more research on this shortly.

Got some feedback?

Whether you’re a WhatDoTheyKnow Pro customer or not, we’d love feedback on this feature — or any other. Drop us an email to pro_team@whatdotheyknow.com.

Back in March, we flagged up the ‘batch request’ feature we’d been working on for the WhatDoTheyKnow Pro service. Batch requests are now switched on for every WhatDoTheyKnow Pro subscriber, by default.

Batch enables users to send the same Freedom of Information request to several bodies at once, and we spent a substantial amount of time building and testing it because we wanted to be confident that the feature wouldn’t be abused — or if it was, that we could catch irregular behaviour.

Part of that testing has involved making the feature available to a limited number of WhatDoTheyKnow Pro subscribers, and loosely monitoring how it was used. We’re glad to say that during this four-month period, the activity was all acceptable.

However, we also realised that we should tighten up our terms and conditions to reflect our expectations around usage of Batch, and add some advice to our Help pages about making responsible and effective requests, both of which we’ve now done. We’ve also added some automatic notifications that will alert the team when multiple batch requests are made, so that we can check that everything is in order.

If you think Batch might be useful in your own work or campaigning, and you’d like to find out more about WhatDoTheyKnow Pro, you can do that here.

 

—

Image: Ankush Minda

When we started building WhatDoTheyKnow Pro, our toolkit for professional users of FOI, we knew that there was one feature which would be a game-changer for such users: the ability to send a request to multiple authorities at once.

In this blog post, we examine what we wanted the tool to do, how we are guarding against abuse, and finally we’ll give a step by step walkthrough of the interface.

Investigative news stories or in-depth pieces of research often require information from a multitude of different sources. By gathering statistics or information from multiple authorities, journalists, activists and researchers can build up a previously-unseen picture, for example of how widespread a particular problem is, or where there are inequities in medical provision across the country.

It’s something that many professional users of FOI are doing already, usually with the aid of their own homemade spreadsheets on which they keep track of requests made, dates by which replies should be expected, which bodies have responded, which need chasing, and of course the information held in the responses themselves.

The standard WhatDoTheyKnow website already provides several helpful features that you just don’t get with a DIY system: it has all the right email addresses for authorities, for example; it guides you through the FOI process; and it will send you an email reminder when the deadline for response arrives — even taking bank holidays into account.

But we knew that in order for our batch request feature to woo people away from their spreadsheets, it needed to do more than those homebuilt systems, some of which have been refined over several years and work well, even if a bit clunkily, for their owners.

Power and responsibility

One important consideration was uppermost in our minds when it came to batch requests: it costs authorities time and money to respond to each request, and of course that multiplies with batch requests. We are keen to promote responsible use of FOI, so we want to fold appropriate safeguards and guidance into whatever system we build.

As mentioned, with WhatDoTheyKnow Pro we’re focusing on features that are genuinely useful for professional users of FOI, but we also want to help those users make better, more focused requests — ones that are more likely to get useful responses and see the light of day as news stories. So it was important that, in making it simple to send multiple requests, we also help users find the most suitable authorities to send their requests to.

With that in mind, here are some balances we’ve put in place:

• Users are limited in how many batch requests they can send within any one month — so there’s no chance to go too wild.
• There’s a limit to the number of authorities that can be added to a single batch: we set this to be the number of local authorities in the UK, which is a logical sector to survey in this way.
• Before users do a batch mailout, we encourage the sending of an initial request that goes to just a few authorities. This safeguard can reveal where a request is flawed, so for example, if the data you get back is not what you need or in the wrong format, you don’t have to send to the full list all over again.
• We provide advice on cost limits to encourage succinct batch requests.
• Authorities have the facility to report a request which is unsuitable for review by our administrators.
• We’re rolling out the batch request functionality gradually to vetted WhatDoTheyKnow Pro users so that we can gradually learn how people use it in practice, and course-correct as necessary..

Testing and improvements

So far, the batch feature is only available to a select group of test users, who are giving us feedback on how they’re finding it. There’s certainly nothing like having your code being used by real people to help you see where improvements might be made!

That said, it’s been a very gratifying process. With the help of our test users, we’ve seen that the batch request functionality has the potential to be immensely helpful to professional users of FOI; even genuinely game-changing. We are certain that with the sending tools, we’ve created a service that really adds value for this sector.

We’re now in the next phase, and turning our attention to improving the functionality that helps users deal with incoming responses when they come in. This already exists in a basic form, and thanks to our testers, we’ve identified which improvements we need to make. We’re already working on incorporating them. But that is definitely material for the next update  — for now, let’s take a look at just how the batch request function works.

Using batch request

There are three parts to making any request, whether you’re doing it yourself or using WhatDoTheyKnow Pro:

1. Creating the request
2. Managing the responses
3. Analysing the results

The batch request functionality builds on our super-simple FOI workflow tools for WhatDoTheyKnow Pro, extending them to make larger investigations much easier.

Creating the Request

The first step is compiling a list of authorities to send the request to. From the compose screen, you might search on a keyword (for example, ‘hospital’, ‘Birmingham’, or ‘Birmingham hospitals’) and then add the authorities you’re interested in.

Each authority is added to a recipient list and WhatDoTheyKnow Pro creates a ‘mail merge’ setup. You’ll see how many authorities you’re writing to in the compose interface.

You can then draft your request. The special `Dear [Authority name]` salutation gets automatically replaced with each of the selected authorities when you send your batch.

Finally, before sending you can choose a privacy duration.

At this point you can either go straight ahead and send your request, or save the draft and come back to it later.

Managing Responses

Once you’ve sent your batch request, you’re going to receive a lot of replies from authorities. This is where WhatDoTheyKnow Pro’s functionality really comes into its own, keeping all that clutter out of your email inbox.

Here’s what it looks like: the first thing you’ll see is a high-level progress bar showing you the overall progress of your batch. There are three main states that help you manage the requests in the batch:

• In progress (yellow): This means that there’s no action needed by you – you’re waiting on the authority to respond with an acknowledgement or the information you’ve requested.
• Action needed (red): When a request in the batch receives a response from the authority, you’ll need to check it out. We mark the response as “action needed” for you to review and decide what to do next.
• Complete (green): Once there’s no further action needed – either you’ve got the information you asked for, the authority didn’t have the information, or they’ve refused and you don’t want to challenge them – the request moves to the ‘complete’ state, so you know you don’t need to think about it until you come to analyse the data.

Clicking the title of the batch reveals the individual requests and their progress status. From there, you can click through, read the response and update the status.

Analysing Results

Now you’ve got all your data, it’s time to compare the results from different authorities.

Sometimes authorities will reply in the main correspondence.

Other authorities respond with one or more attachments. You can view these inline or download them to your computer.

If you’re dealing with a batch sent to lots of authorities, sometimes it’s easier to just download everything. You can download a Zip file containing all the correspondence and attachments for each request via the “Actions” menu. From there you can pull out the attachments that contain the raw data and plug the numbers or answers in to your spreadsheet so that you can compare across authorities.

 

If your FOI requests have made the news, let us know! Send us links to your published stories and we’ll throw in an extra month of WhatDoTheyKnow Pro for free. Your stories help us improve WhatDoTheyKnow Pro.

—

Image: Peretz Partensky (CC by-sa/2.0)