With the aim of making large scale Freedom of Information investigations easier for community newsrooms and campaigning organisations, we’ve spent the first half of 2020 developing collaboration tools for WhatDoTheyKnow to speed up and bring others into the FOI management process.
In an initial pilot, 17 contributors saved a journalist 6.5 hours by taking on half of the work of managing responses to requests.
We’re actively looking to partner with membership-driven news organisations or impactful campaign groups to run further pilot projects to help refine the features. If that’s you, please get in touch.
FOI can be hard without dedicated tools
We know FOI can be hard work, especially when you make large batch requests that return a huge amount of data.
While our Pro tools make life easier, much of the work simply involves triaging whether you got a response or just an automated acknowledgement, and whether the authority actually released the information you requested.
After that, you then need to sift through various different formats of data, different understandings of the questions, and follow up with clarifications.
All this comes before you can start analysing the data to build up a narrative for a story.
A compelling membership proposition
News organisations are increasingly looking for sustainability by offering memberships – where you pay a monthly fee to support the organisation – instead of relying on advertising revenue to support themselves.
Memberships are still a relatively unproven and unexplored area, and organisations are still in the process of discovery over what makes someone want to pay for their news output. Is it just being able to read the stories, or do people want more involvement?
There’s evidence to suggest that members do want to get more involved.
Crowdsourcing some of the work of the FOI process from the membership presents an opportunity to help take some of the load off journalists, while also bringing members into the reporting process so that they value the final output more.
Many hands make light work
With this new functionality, once you’ve made your requests – either individually or as part of a batch – they can be added to a Project. Contributors can then be invited to the project where they are briefed on what the project is about and the tasks they can help with.
Helping to classifying responses
When you’re making FOI requests, each response to each request needs to be read to establish whether the authority has provided the information asked for – a process that is difficult to automate, given the huge variety of language that can be deployed by authorities. With large batch requests this can be a time-consuming process.
Projects creates a pool of responses that need classifying that contributors can work through to take some of the onus off the project owner.
Contributors read the original FOI request and latest response, and then classify its current status appropriately. This doesn’t take much specialist understanding of FOI, so it’s a really easy way to get lots of people to help out.
Helping to extract data
In larger FOI investigations requesters are usually looking to build up a dataset so that they can compare responses from different authorities.
This usually involves lots of spreadsheets, copy & paste, and hours of hard work.
Projects provides dedicated tools to help build this dataset by creating a pool of requests that contributors can extract data points from using structured forms.
Allowing contributors to help build up a dataset that will be used for real-life reporting and research helps them feel more directly involved and connected to the organisation, hopefully adding value to the membership proposition.
Project owners are then able to download the crowdsourced dataset to investigate, using their analysis tools of choice.
What we learned from our pilot
In our pilot project contributors took on 50% of the classification tasks, accounting for 57% of the 14.8 hours overall spent classifying, saving the journalist around 6.5 hours of the administrative work required before she could start reviewing the data releases. This is a clear indication that crowdsourcing key parts of the FOI investigation process can save a significant amount of time.
The journalist we worked with was enthusiastic about using the Projects interface again in the future, even if she wouldn’t be inviting external contributors. She expressed that it would be ideal to collaborate with interns to help sift through classifications and responses.
With an 82% conversion rate from joining to taking action and nearly 40% of contributors returning for more than one session there’s clearly an appetite from contributors to get involved and help out. The contributors we interviewed understood that by helping with menial tasks, they were allowing the journalist more time to focus on work which required specialist expertise.
A potential for global benefit
Through the Nesta Future News Fund we worked with openDemocracy to design and develop WhatDoTheyKnow Projects to support this collaboration, and ran a pilot collaborative project made up from a batch of over 800 FOI requests.
Projects is of course built into Alaveteli – the platform that powers WhatDoTheyKnow and many other FOI sites around the world, so it’s not just going to be of use in the UK, but for every jurisdiction where an Alaveteli site is utilising the Pro add-on.
Image: Duy Pham
Investigative journalism platform The Ferret has just launched an online training course on using Freedom of Information — and all trainees get a free subscription to our WhatDoTheyKnow Pro service for professional users of FOI.
Based in Edinburgh, the Ferret is a community journalism initiative that describes itself as ‘for Scotland and beyond’. Since 2012 its members’ investigations have rooted out the truth around local, national and international issues including coronavirus, Brexit, dark money — and much more. They’re a co-operative, so supporters become part-owners. If they want to, they can also access the resources and training to pursue their own stories.
And now, the Ferret’s online Freedom of Information course shares everything the founders know about the use of FOI for tracking down facts. This resource would be useful for anyone wanting to know the ins and outs of the act and how to use it, not just for journalism but potentially for campaigning or research purposes too. And it’s not just restricted to the use of FOI in Scotland: you’ll learn everything you need to know to use FOI across the UK… and beyond.
The course costs £30, but six months’ WhatDoTheyKnow Pro usage is bundled in. Since that’s worth £60 on its own, you’re ahead before you even begin.
We’re big fans of the Ferret at mySociety, and we have every confidence that this course will be a springboard for a new generation of great investigative journalists. If you think you might like to be one of them, then why not give it a try? More details here, and in this Twitter thread.
You may remember our recent post on the surveillance techniques in use by police forces, as investigated by the campaign group Privacy International.
Several of you tweeted or commented that you were concerned to read of these new technologies. Well, here’s a way that you can get involved in finding out more.
Sounds fluffy? The reality is a bit more chilling. Cloud extraction technology allows the police to gain access from a citizen’s mobile phone to cloud based services such as their email, browser activity, and social media. So, if you are stopped and your phone is examined then handed back, surveillance might not stop there. Even after the phone is returned, using this tech police can monitor your online activity on an ongoing basis, seeing what you search for, trawling through your social media posts, and even accessing your location data.
Whether or not you’ve ever been detained by the police, you might like to know whether this sort of surveillance is in action in your own local neighbourhood. And that’s where FOI comes in.
To make everything as easy as possible for you, Privacy International have used pre-filled FOI requests* and provided the wording you should include. You can also see which forces have already been contacted, so as not to waste time making duplicate requests. Here’s where to get started.
Camilla Graham Wood, a Legal Officer at Privacy International, is clear about the benefits WhatDoTheyKnow has brought to their campaigning: “Using WhatDoTheyKnow we have created a way for members of the public to quickly and easily contact their local police force and ask them about intrusive surveillance tech. We were able to embed this on our own website and to pre-fill certain boxes as well as adding a tag so we can follow the progress of the campaign.
“Engaging the public in this way shows the level of public interest in policing technologies and introduces those who might not have used Freedom of Information request before to this valuable transparency tool”.
*If you’re running a campaign and you’d like to know how to set up something similar, take a look at this blog post where Gemma explained it all, back in 2016.
Image: Gilles Lambert
We’ve recently made a small change to WhatDoTheyKnow. Now, if your request has resulted in a news story, you can add the link as a ‘citation’.
If you’re a journalist or a campaigner, we hope this is a useful way to give your stories some more readership (not to mention a nice inbound link from a high-ranked site for your search engine ratings).
And if you’re simply a citizen whose request was picked up by the press, there’s now a way to share how the information you’ve obtained fits into the news agenda.
There’s a benefit for the wider transparency community, too. We think that, in aggregate, these links will serve to show others just what a simple FOI request can do.
Requests often start off as nothing more than an inkling or a nagging question, but there’s always the chance that one of them will reveal important or interesting information, hitting the news, reaching a wider readership and — who knows? — maybe even changing hearts and minds. For big stories, it will be good to be able to create a permanent record of where it all began.
More broadly, when you use this feature you’ll be helping us to understand what sort of impact the site is having, too. We’re always keen to spot news stories based on WhatDoTheyKnow requests, but papers don’t always cite a source or link back to the site, meaning that our monitoring is often dependent on a manual search where stories look like they might have originated with one of our users.
How to add a citation
You’ll find the feature in the right hand column of your request page. Just click on ‘Let us know’:
…and paste the URL in:
If there’s more than one story, you can click ‘New Citation’ to add another one.
The way we’ve set this feature up, WhatDoTheyKnow users can add a citation to any of their own requests — but if you spot a news story that’s linked to a request that isn’t yours, please do contact the WhatDotheyKnow team.
They’ll assess it and input it if they find it to be valid. Our aim here is, of course, to prevent spammers from adding irrelevant links to the site.
Users of WhatDoTheyKnow Pro, on the other hand, have the ability to add citations to any request.
This work is one of the ongoing improvements that we’re working on thanks to a grant from Adessium.
Image: Kaboom Pics
In a strike for transparency, journalist Jenna Corderoy has secured the release of documents from the European Research Group (ERG), the pro Brexit lobby of which Jacob Rees-Mogg is a prominent member.
For more than a year, Jenna has been striving to ensure that the facts around Brexit — and the funding that drives it — reach the public domain: she also broke the now-infamous revelations about Vote Leave’s campaign overspending.
The release of material such as this into the public domain is beneficial to all, as it means that public debate is based on facts rather than conjecture. FOI can be a vital tool in ensuring that the documents shaping our society’s future direction are available for scrutiny.
On this latest release, a piece by Jenna and Peter Geoghegan reports:
“The ERG is part-funded by subscriptions paid out of MPs’ parliamentary expenses. As a consequence the group has to supply samples of its research for scrutiny to the Independent Parliamentary Standards Authority [IPSA] to ensure public money is being properly spent and not used for party political campaigning.”
Using WhatDoTheyKnow Pro — our service for professional users of FOI, which among other features, allows users to hold off from putting request correspondence in public until a story has been published — in January 2018 a request was made to IPSA to see these materials.
The request’s WhatDoTheyKnow page is now public. IPSA initially argued that the release of these materials was exempted under section 43 of the FOI Act as it would prejudice the commercial interests of the ERG, whose research is ordinarily available only to those paying a £2,000 annual subscription.
Subsequently Jenna referred this refusal to the Information Commissioner, who upheld the decision. Determined that the public has the right to see the research, Jenna and Peter did not leave the matter there, taking it to an information tribunal.
The tribunal made the final decision that the material must indeed be released, vindicating the effort and determination Jenna put into pursuing this request and stating that to make the documents available would:
“further transparency, accountability and public trust with respect to the working of Parliament”.
As a result, the documents will be made available on 11 July — keep an eye on OpenDemocracy for news of their release — and we’ll make sure we update the annotations on the original request as further details unfold. Meanwhile, you can see the full tribunal decision here.
Image: Udur Akdemir
Last year, we launched WhatDoTheyKnow Pro, our service for journalists and other professional users of Freedom of Information.
As it’s a new venture, we’re keen to track whether it’s achieving everything we’d hoped for when first planning the service. One of the targets which we set, as a measure of success, is the number of impactful press stories generated by its use.
What might count as impactful? Well, that’s obviously up for debate, but loosely we’d say that we’re looking for news stories that have a wide readership, and uncover previously unknown facts, offer new insights, or bring about change.
Stories in the news
A couple of recent stories, generated through WhatDoTheyKnow Pro, have ticked at least a few of those boxes. At mySociety, we keep a position of political neutrality — our services are available to everyone of any persuasion, and we don’t campaign on any political issue — so we present these stories not to comment on their substance, but to note that they certainly fit the criteria above.
Brexit is clearly one of the most vital stories of our day, here in the UK, and while many might feel that we’ve had a surfeit of commentary on the issue, we can only benefit from understanding the facts.
One of WhatDoTheyKnow Pro’s earliest users, Jenna Corderoy, broke two important stories on that topic. First, that UK parliamentary standards watchdog IPSA is investigating Jacob Rees Mogg’s hard Brexit European Research Group over their second bank account and ‘informal governance structure’. This also ran in the Daily Mail.
Secondly (and this progresses a previous story about expenses – also uncovered thanks to Jenna’s use of WhatDoTheyKnow Pro) there is the widely-reported story that the Electoral Commission had misinterpreted laws around campaigning expenses, allowing Vote Leave to overspend. This was picked up by the BBC and Guardian, among many others.
No matter which side of the Brexit debate you support, hopefully you will agree that it benefits society as a whole to have the facts out in the open.
From FOI request to the national news
As a last thought: it’s interesting to us to see how a story grows from one or more FOI requests into something that hits the national news platforms. We think these stories were broken using a tried and true method that goes like this:
- As a journalist, campaigner or researcher, you might be investigating a topic. Perhaps you’ve heard a rumour that you’re hoping to substantiate, or perhaps you’re inquiring more deeply into a story that’s already in the air. Using FOI, you can retrieve facts and figures to bolster your investigation.
- Once you have a story, you can publish it in a smaller publication like Open Democracy, testing the water to see if it gains any traction.
- If the story is well-received there, it’s easy to approach larger outlets with the proof (in the form of FOI responses) underpinning it.
If you’re a journalist or you use FOI in your professional life and you’d like to try WhatDoTheyKnow Professional for yourself, then head over to whatdotheyknow.com/pro. Put in the code WELCOME18 when you sign up, and you’ll get 25% off your first month’s subscription.
Image: Roman Kraft
The Freedom of Information Act allows us to keep check on authorities — not least, how they are spending public money.
These sorts of requests can be of particular interest to regional press, who perform a vital function in keeping citizens informed about where their taxes go. Journalists might be applauding good spending; equally they could be uncovering waste or corruption — both narratives are highly relevant in times of budget cuts and austerity.
Here are examples of stories arising from council expenditure requests, covering the length and breadth of the UK.
Spending on administration
It’s often worth scrutinising council expenditure on day to day items like office supplies, furniture or computer hardware.
While stories here may be quite mundane on the face of it, they become more interesting when the question arises of how much is reasonable to spend on these overheads.
In North Warwickshire, the Birmingham Mail reported that over £5,000 went on tea, coffee & biscuits (to be fair, over a period of five years). The story goes as far as to detail each brand of teabag. It’s important, said the council, to offer visitors a decent spread.
That’s as may be, but how about the same region’s County Council splashing out £13,700 on furniture for their leader’s new office — and not from Ikea, by the sounds of it. This story was run by the Rugby Advertiser, and just as with the tea and biscuits, the defence was that it’s important to make visitors comfortable.
But how far does such comfort extend? Let’s consider social functions and parties. Dundee was highlighted in the Courier for its awards bashes, into which a round £266,000 was sunk. Perhaps easier to defend was the same authority’s £150,000 expenditure on tablets and computers.
Spending on infrastructure and services
Fixed penalty notices for littering, often farmed out to third party fine collectors, can be highly contentious. Citizens might be somewhat mollified when they see a boost to their councils’ coffers and some expenditure that directly relates to the misdemeanours in question. In Wirral, for example, the Globe tells how £176,000 from littering fines has been spent on CCTV surveillance, anti-rubbish and fly-tipping campaigns, research and clean-ups.
In Borehamwood, money gleaned from developers within the town paid for £1.7M worth of school expansions, crossroad junctions and traffic calming measures, as reported in the Borehamwood Times.
And then there are the oddities, anomalies or outliers of spending that always make a good story, too. The Scotsman found that fees for taxis to take children to schools that offer subjects they can’t study in their own schools added up to £1M across Scotland.
Spending on staff… and disputes
Of course, every council has staffing costs — it’s how they’re distributed that can make a gripping story for a regional publication like Wokingham Today. When does expenditure on temporary employees, for example, start indicating that permanent staff might have been a better bet? We don’t know the full circumstances, but Wokingham’s £7.8M does seem like an awful lot.
In Norfolk and Suffolk, £200,000’s worth of ‘golden hellos’ to welcome new staff to the area were under scrutiny by the Eastern Daily Press. But that kind of expenditure seems perfectly wholesome when contrasted with Wiltshire’s £275,000 on gagging orders and compromise agreements.
The picture can sometimes be a little more complex than first viewing might suggest. In South Gloucestershire for example, the Bristol Post reports that £15M worth of redundancy payments is expected to save the council money in the long run.
In Birmingham, the Mail had a fine old time trying to get their FOI request responded to, and were finally successful in revealing that overtime paid to binmen in the run-up to a strike amounted to over £1M.
Spending on compensation
We mentioned that expenditure stories can be timely in the current age of austerity — well, not only that, but there’s a particular irony to be found in expenditure that’s potentially caused by austerity measures.
Two recent stories looked at the compensation councils were paying out for damage caused by potholes. Authorities everywhere are making cutbacks to their expenditure on roads, but if this results in payouts that are bigger than the savings involved, there’s obviously a false economy at play.
In Lancashire, the Telegraph discovered that the council had paid over £1M in pothole compensation in one financial year, while in Cambridge the local paper went back as far as 1992 to break down the expenditure year by year.
Have we sparked ideas?
If you are a journalist and you’d like to use Freedom of Information to uncover this kind of story — while keeping your research under wraps until it has been published — check out our WhatDoTheyKnow Pro service. It provides all the tools that you need to pursue your investigations, large or small.
As a regional journalist, you might also find it useful to follow your local council on WhatDoTheyKnow, so you know what information our users are asking for. We’ll send you an email when requests are made to your chosen authority — just click the grey ‘follow’ button on any council’s page.
We’ve talked a lot about our new service for journalists and other professional users of Freedom of Information — but it’s not always the professionals who uncover the news stories.
This week, we mark WhatDoTheyKnow’s tenth anniversary. As part of the celebrations, we thought we’d look back on the news stories that came about because of requests made through the site. Many of these began with an FOI request submitted by a user with no links to the press, and were picked up by news outlets because the response was of public interest.
From the restrictions on what names can be given to a baby in this country, to an accidental torpedo release, and via a geographically-accurate Tube map, it makes for fascinating reading. You can see them all here.
Our Freedom of Information toolkit for journalists, WhatDoTheyKnowPro, will have its official launch very soon — and we’re glad to see that it’s already beginning to help generate high-profile news stories based on FOI requests.
During development, several journalists have been putting it through its paces and offering us invaluable feedback which has helped us shape the service — and meanwhile their activity is also uncovering stories of genuine interest. These give a taste of exactly what kind of investigative stories can be supported by WhatDoTheyKnowPro, which makes requests to multiple bodies simpler, as well as organising the responses so that they’re easier to manage.
Here are the WhatDoTheyKnowPro-generated stories we’ve heard about so far:
Student Brexit campaign received ‘as much funding as necessary to win’
Open Democracy and the Ferret uncovered how Vote Leave used a loophole to funnel hundreds of thousands of pounds through a student’s small-scale campaign. The story was subsequently run by multiple other news outlets and legal proceedings towards a judicial review have begun.
British police trained officers in repressive regimes
Scrutiny of documents from the College of Policing revealed that much of its income was coming from countries where there is concern about human rights. The story, by Lucas Amin, was run by the Guardian in September.
Public servants and Scottish ministers paid thousands of pounds to dine with Obama
Investigative journalism platform the Ferret uncovered this story in July, detailing how much public money was spent on senior staff attending a charity dinner with Barack Obama.
Links between Mark Hoban and Price Waterhouse Cooper
The Times and the Daily Mail both ran this story from Patrick Hosking, which revealed the former financial secretary’s ‘forgetfulness’ over prior links with Price Waterhouse Cooper.
Infighting in UKIP over the name Patriotic Alliance
Both anti-far right activists and UKIP officials tried to stop Arron Banks from registering a new political party called the Patriotic Alliance, another story run by the Ferret revealed.
Where British holiday-makers get arrested most often
Back in June, this story by analysed figures from the FCO on where Brits had been detained and obtained consular support, allowing them to state which countries had the most arrests, and how figures had changed over time. The story ran in the Birmingham Mail and was also picked up by other publications in the Trinity Mirror Group.James Rodger
We’re delighted to see such good use being made of WhatDoTheyKnowPro, and we anticipate many more stories emerging once it has fully launched.
Jenna Corderoy, Alaveteli Professional Advocate, brings us an update on the project.
Since our last blog post on Alaveteli Professional — our Freedom of Information toolkit for journalists, campaigners and activists — there have been a few exciting developments.
The batch request feature is coming along nicely: this will allow users of the service to send one Freedom of Information request to multiple authorities and help them to easily manage large volumes of responses.
We’re going to be working with a small group of our beta testers to develop this feature and make sure we release it in a useful and responsible form (click here to apply as a beta tester and get a year’s free access to WhatDoTheyKnowPro, the UK version of the service).
We’ve been pleased to see the first news story to emerge as the result of a request made through WhatDoTheyKnowPro: a response to the Foreign and Commonwealth Office showed which are the countries where UK holidaymakers are most likely to get arrested. The full list was covered in the Birmingham Mail.
But we also have plans for this Freedom of Information toolkit to go international: Alaveteli Professional will be a bolt-on option for anyone already running an FOI site on our software platform Alaveteli.
In April, mySociety team members traveled to roll out the first such project, with Info Pro Všechny, the Czech Republic’s Alaveteli site.
We were able to introduce beta users to the features we’ve been developing, such as the ability to keep requests private until the story has been published.
While in the Czech Republic, we held a roundtable discussion with journalists and campaigners, swapping Freedom of Information battle stories and sharing tips and tricks for getting the best results when submitting requests for information, as well as experiences of filing requests to European Union institutions.
mySociety was also invited to give a talk to student journalists based in Olomouc about Info Pro Všechny and Alaveteli Professional in general, discussing success stories generated from our Freedom of Information sites from around the world.
We’re currently working on subscription options, which will allow us to officially launch WhatDoTheyKnowPro as a paid-for service in the UK, and later in the year, we plan to introduce the Pro toolkit to the Belgian Alaveteli site Transparencia.be, which has been making a splash in Belgian politics.