GCloud 11 is live: it’s the latest iteration of GOV.UK’s Digital Marketplace, making it easier for those in the public sector to find and procure cloud-based software services — including ours.
Regular followers will be well aware that FixMyStreet Pro is a street fault reporting service which can integrate with any existing council system, offering great opportunities to cut costs and increase efficiency.
Meanwhile our FOI for Councils service streamlines authorities’ FOI workflows and reduces unnecessary requests, relieving the burden in what is often an overstretched resource.
The great benefit of GCloud from the public sector point of view is that suppliers come ready-verified, saving the time and inconvenience of going through the regular procurement process. All the information you need about the service is readily accessible, and then when you’ve made your decision it’s very simple to get things moving.
We’re pleased to offer these two services via GCloud — and will be equally happy to answer any questions you may have.
Every quarter the Cabinet Office releases Freedom of Information statistics for a collection of central government ministries, departments and agencies. This provides a benchmark for understanding how requests made from WhatDoTheyKnow relate to FOI requests made through other methods. From 2017, mySociety started retrospectively tracking the proportion of FOI requests sent via WhatDoTheyKnow to central government using a minisite — https://research.mysociety.org/sites/foi/ — that explores the data.
This report explores what had and hadn’t changed in the last few years, as well as the number of requests made through WhatDoTheyKnow Pro — a new service being piloted that allows embargos of the results of FOI requests for a period — with the goal of bringing more people making FOI requests professionally (such as journalists) into the system and leading to more raw results being made available after the conclusion of a project.
We found that:
- WhatDoTheyKnow accounted for between 15-17% of audited bodies and between 18-21% of ministerial departament FOI requests.
- While the proportion of requests have grown most years since 2010, there was no real change from 2017 to 2018.
- Requests made to central government via WhatDoTheyKnow only make-up around 9-10% of all requests sent via WhatDoTheyKnow in 2018.
- WhatDoTheyKnow Pro requests made up 1% of FOI requests to central government — but most requests using this service went to other areas of the public sector.
In a strike for transparency, journalist Jenna Corderoy has secured the release of documents from the European Research Group (ERG), the pro Brexit lobby of which Jacob Rees-Mogg is a prominent member.
For more than a year, Jenna has been striving to ensure that the facts around Brexit — and the funding that drives it — reach the public domain: she also broke the now-infamous revelations about Vote Leave’s campaign overspending.
The release of material such as this into the public domain is beneficial to all, as it means that public debate is based on facts rather than conjecture. FOI can be a vital tool in ensuring that the documents shaping our society’s future direction are available for scrutiny.
On this latest release, a piece by Jenna and Peter Geoghegan reports:
“The ERG is part-funded by subscriptions paid out of MPs’ parliamentary expenses. As a consequence the group has to supply samples of its research for scrutiny to the Independent Parliamentary Standards Authority [IPSA] to ensure public money is being properly spent and not used for party political campaigning.”
Using WhatDoTheyKnow Pro — our service for professional users of FOI, which among other features, allows users to hold off from putting request correspondence in public until a story has been published — in January 2018 a request was made to IPSA to see these materials.
The request’s WhatDoTheyKnow page is now public. IPSA initially argued that the release of these materials was exempted under section 43 of the FOI Act as it would prejudice the commercial interests of the ERG, whose research is ordinarily available only to those paying a £2,000 annual subscription.
Subsequently Jenna referred this refusal to the Information Commissioner, who upheld the decision. Determined that the public has the right to see the research, Jenna and Peter did not leave the matter there, taking it to an information tribunal.
The tribunal made the final decision that the material must indeed be released, vindicating the effort and determination Jenna put into pursuing this request and stating that to make the documents available would:
“further transparency, accountability and public trust with respect to the working of Parliament”.
As a result, the documents will be made available on 11 July — keep an eye on OpenDemocracy for news of their release — and we’ll make sure we update the annotations on the original request as further details unfold. Meanwhile, you can see the full tribunal decision here.
Image: Udur Akdemir
We’ve just released two new versions of Alaveteli, our open source platform for running Freedom Of Information sites — one which we’ve packed all of bug fixes, new features and performance improvements into, and one with some important technical updates. Here are some of the highlights.
Making your first request easier
mySociety’s designer Zarino has subtly redesigned the authority search page – which is the first thing most users will see when making a request – to make it easier to find the help links at a glance, making the process feel much more approachable.
Encouraging better quality requests
Part of our advice to users of WhatDoTheyKnow is to keep requests concise and focused. This new feature adds a visual reminder when the request text is approaching the point of being too long for an FOI officer to answer it efficiently:
With a second, stronger warning if the request becomes longer still:
We’ve also added an experimental feature on WhatDoTheyKnow to discourage users from making requests for personal information. (As well as not being valid FOI requests, publishing the responses would result in revealing confidential information about the requester.) By asking the user to clarify what they want to ask before sending a request to authorities which attract a high level of personal information requests, we can try to redirect them to the correct official channel instead:
One-click unsubscribe from notifications
An improvement for site owners and end users: it is now possible to unsubscribe from email notifications about requests that you’re following right from your inbox! (Previously – and still the case with other sorts of email that the site sends – you would have to visit the site and log in to change your email preferences.)
Allowing people to unsubscribe with ease – if they’ve forgotten that they signed up for notifications or have created a new track by mistake – should help to cut down on the number of messages being marked as spam which should in turn help improve the site’s mail sender reputation. (It also allows admins – if they receive feedback loop notification emails from email providers – to unsubscribe users who’ve marked these emails as spam, preventing further unwelcome emails being sent.)
In version 0.33, as well as our our features and fixes, we’ve added support for Ubuntu releases Xenial and Bionic and withdrawn support for Debian Jessie. The full list of highlights and upgrade notes for this release is in the changelog.
Version 0.34 contains our Rails 5.0 upgrade work which we’ve released separately to allow reusers time to adapt to new minimum requirements for operating systems and Ruby versions.
We’re no longer supporting Ubuntu Trusty and have also dropped Ruby versions older than 2.3 as that’s the minimum requirement for Rails 5. Upgrade notes are available in the changelog and, as ever, we recommend updating one version at a time to make sure everything’s working smoothly and reduce the risk of missing essential upgrade steps.
Moving to Rails 5.0 will allow us to retain support for major security issues when Rails 6 is released and dropping older Ruby versions removes some key technical barriers to modernising the Alaveteli codebase and allows us to focus on improving Alaveteli Pro so that it can be reused more widely.
Thanks again to everyone who’s contributed! Special thanks to Nigel Jones and Laurent Savaëte who contributed bug fixes for version 0.33!
Our own FOI site WhatDoTheyKnow is always interesting to browse, but we suspect that even the gems waiting to be discovered there might pale in comparison to the 13 million pages of declassified files released by the CIA over in the States.
These are available to American citizens — and indeed the world — thanks to sustained efforts from our friends over at MuckRock, the US FOI site*.
In 2016 MuckRock won a three year fight compelling the CIA to abide by the nation’s FOIA law and release their files: the history of how the CIA had dodged their obligations for so long is amusingly written up in this post.
Now MuckRock are encouraging users — including you, if you would like — to browse the content and let them know of anything interesting you discover. They’re always happy to share the more useful, fascinating or downright bizarre information unearthed.
Never let it be said that FOI is dull or dry: so far they’ve written up almost 300 findings, including a recipe for borscht, an Edgar Allen Poe parody, a guide to christening ships and the very mysterious picture of a man.
You can find guidance on how to tackle this vast archive on the Muckrock site. If you discover anything worth telling MuckRock about, please let them know we sent you.
Kudos to MuckRock and their tenacious users for their work in getting these files into the public domain.
* Unlike many of the international FOI sites we write about, MuckRock isn’t run on our Alaveteli platform, but it shares the same aims and we’re proud to be working together. In fact, as we were writing this, members of mySociety’s Transparency team were over in Tunis at RightsCon, giving a joint presentation with MuckRock, and they’ll be coming to our FOI technologies conference AlaveteliCon in September. And here’s a picture of us meeting up in the UK.
Image: President Ford meets with CIA Director-designate George Bush (via Wikimedia; public domain)
On 26 – 27 of June, scholars and practitioners from all over the world will be meeting in Rio de Janeiro for the 6th Global Conference on Transparency Research. The conference focuses on measuring transparency, exploring how this can be achieved, what the barriers are, whether metrics are useful, and how current interventions are shaping transparency around the world.
mySociety’s Head of Research Rebecca Rumbul will be attending, and will be presenting some of mySociety’s recent research into the transparency of parliamentary information in sub-Saharan Africa. Examining transparency through a digital lens, this research broke new ground in understanding how digital tools are shaping parliamentary transparency in sub-Saharan Africa, and how barriers to transparency are affecting how citizens engage with public institutions. You can read the full report here.
Rebecca will be speaking at 4pm on Thursday 27 June, so please do come along and say hello. She says, “Transparency, digital and citizen engagement are core themes of our research at mySociety, and we love to talk to other people working in these areas. Meeting new people and sharing ideas are the best parts of any conference, so do grab me for a chat if you are attending.”
If you are unable to join Rebecca in Rio, but you are interested in talking research, we’re always happy to receive email. And keep your eyes peeled for our TICTeC conference announcement for April 2020. We will be opening our Call for Papers in early September.
Image: Jaime Spaniol
Next week Gareth and I will be heading to Tunis to attend the 8th edition of RightsCon. RightsCon is the world’s leading summit on human rights in the digital age, so we’re thrilled to be hosting a session at the conference about digital Access to Information platforms with our awesome friends at MuckRock.
If you’ll also be there we’d love to talk to you about your campaigns or investigations and how using access to information platforms could help.
As Jen said in her recent blog post, we’ll be spending time this year developing our software platform Alaveteli Pro so more people across the world have access to its digital tools that help with the sending and management of information requests.
We’d love to get feedback on this work and would love to meet organisations who are interested in setting up Alaveteli Pro instances, in order to make access to information easier for citizens in their countries. We’re also very keen to talk to individuals and organisations who are interested in collaborating on cross-border public-interest investigations and campaigns using FOI-generated data.
We’d also love to talk to RightsCon attendees who might be interested in attending our AlaveteliCon event in Oslo on 23 and 24 September, where activists, journalists, technologists and campaigners from across the world will come together to discuss Freedom of Information technologies for creating public-interest investigations and campaigns.
And of course, our Call for Proposals is currently open for our TICTeC Local conference so it’d be great to chat to people interested in presenting their work using digital innovations to help local communities and/or public authorities to foster citizen engagement, drive efficiency, and combat social and environmental problems.
Using WhatdoTheyKnow Pro, this project pieced together a nationwide dataset, and generated important stories at both national and local levels.
Sold from Under You, a project from the Bureau of Investigative Journalism, revealed how much publicly-owned property has been sold off across England, as a response to austerity measures. In all, TBIJ discovered that over 12,000 buildings and pieces of land have been disposed of, bringing councils revenue of £9.1 billion — some of which has been spent on staff redundancies.
In collaboration with HuffPost, the findings were presented in the form of an interactive map which allows users to explore sales in their own area.
The investigation required a significant amount of data collection via FOI requests to 353 councils, work which was aided by WhatDoTheyKnow Pro. More than 150 people across the UK, including local journalists, took part in the collaborative investigation. As well as HuffPost’s coverage, stories were run in regional news outlets across the country. The project has now been shortlisted for the Data Journalism awards.
We spoke to Gareth Davies from TBIJ to understand how the organisation approached this ambitious project, and what part WhatDoTheyKnow Pro played in it. Here’s what he told us:
“The Bureau has been investigating the local government funding crisis in the UK for the last 18 months. The initial part of this particular investigation focused on the overall financial health of local authorities and used data to determine which were under the most pressure. We then wanted to look at the impact of the funding crisis so teamed up with Hazel Sheffield and her Far Nearer project to look at the public spaces that were being lost as a result.
“At the start of the investigation we undertook a research period to determine what local authorities are required to publish about the buildings and land they own, and how many of them were adhering to those rules.
“We discovered that while councils have to publish annual lists of the assets they own, this does not include vital information such as who assets were bought from or sold to and the prices paid.
“Also, two thirds of councils update the same spreadsheet each year, meaning change over time is lost. As a result it became apparent that FOI would be required to obtain the information we were interested in. FOI is a tool we have used for a number of stories, particularly those produced by our Bureau Local team.
“The information we wanted could be divided into two groups: what assets councils were buying and selling, and what they were doing with the money raised when an asset is sold. The research period showed we would need FOI to obtain this data.”
More than 700 FOI requests
“To reduce the risk of requests being refused for exceeding the cost/time limit, we needed to submit two separate requests to each of the 353 local authorities in England.
“Previously I had submitted and managed bulk FOI requests via email. However, staying on top of more than 700 requests would have proven very challenging. I was aware of the WhatDoTheyKnow Pro platform but hadn’t used it before, so thought this would be the ideal opportunity to test it out.
I don’t think I would have achieved that without WhatDoTheyKnow Pro
“It was useful to have up-to-date contact details for each authority and to be able to send the FOI requests in one go. But probably the most useful feature was the way in which WhatDoTheyKnow Pro tracks the status of each request and shows you when the public body in question has exceeded the statutory time limit. This made it a lot easier to stay on top of which councils needed to be chased and when I needed to do it.
“Managing so many FOI requests was still challenging and very time consuming but it would have been much harder by email. The first batch of requests had a success rate of more than 95% and the other (which was more detailed) was around 85%.
“I don’t think I would have achieved that without WhatDoTheyKnow Pro and, as a result, the investigation and interactive map we created would not have been as comprehensive.”
Refining the requests
While councils have to publish annual lists of the assets they own, this does not include vital information such as who assets were bought from or sold to and the prices paid
“I sent requests to one of each type of local authority (London borough, metropolitan borough, unitary, county and district) to test what, if any, information councils would provide. The fact that all of those requests were successful meant I had confidence when submitting the batch requests.
“It also allowed me to include additional information in the bulk requests, because some of the test councils erroneously withheld, under Section 40, the identities of companies. As a result I added a note to the request highlighting that this would not be a correct application of that exemption.
“As each response came in I recorded them in two separate spreadsheets — one showing what assets had been bought/sold and another containing information about how the money raised from asset sales had been used. Gradually we built a comprehensive picture of what was happening with public spaces, and that was crucial for our story.”
Bringing about change
There have been tangible results from this investigation.
“The government launched an investigation into the sale of assets by Peterborough Council as a result of this particular story, focusing on that area.
“We submitted our findings to an inquiry currently being held by the Communities and Local Government select committee and were mentioned by name during the first day of oral hearings.
“And last month the Public Accounts Committee announced it would hold a similar inquiry into the sale of public land. Several councils halted their property investment policies after our coverage revealed how much they had borrowed to fund the purchases.”
Thank you very much to Gareth Davies for talking to us about the Sold From Under You project.
Image: Daniel von Appen
WhatDoTheyKnow Pro, mySociety’s subscription service offering extra tools for journalists and other professional users of FOI, has been running in the UK for just about two years.
During that time we’ve launched, worked closely with users to refine the service, and — happily — watched it play a vital part in the making of several important data-driven news stories, on topics as diverse as Brexit campaign funding and the results of austerity cuts on councils. Journalists, in particular, have appreciated tools such as the ability to send and manage bulk requests to multiple authorities; and the embargo tool that keeps requests and responses hidden until the story has been published.
Now, thanks to support from Adessium Foundation, we are able to bring the same benefits to countries across Europe, and — we hope — some additional synergies that will be borne of organisations working across boundaries. The same functionality that extends WhatDoTheyKnow into the Pro version will be available to FOI sites run on the Alaveteli platform, under the name Alaveteli Pro.
The ultimate aim is to enable journalists, campaigners and citizens in Europe to make greater and more effective use of their right to access information; and in particular to generate public interest stories and campaigns that will hold power to account.
We’ll be focusing on three areas in order to achieve this aim:
- We’ll give selected existing Alaveteli sites in Europe the technical help they need to upgrade to the Pro version;
- We’ll be helping organisations in three new European jurisdictions to launch brand new Alaveteli sites, making access to information easier for citizens in these countries. The first site will be launched by VVOJ from the Netherlands.
- We’ll encourage cross-border collaborations between journalists and organisations using the sites (both the existing ones and the new ones) to investigate stories that span more than one EU country.
So watch this space: we’ll be sure to keep you posted as the work progresses. The planned start date is next month, and the project is set to run for three years.
We’re looking forward to sharing stories resulting from this initiative once they start rolling out, and supporting the incredible work that journalists do in putting them together.
Image: Emiliano Vittoriosi
‘Sold From Under You’ project used WhatDoTheyKnow Pro
Not long ago, we let you know about the Bureau of Investigative Journalism’s project to map and quantify the scale of properties being sold by councils up and down the country as they try to manage with reduced budgets under austerity.
The investigation, which made use of our WhatDoTheyKnow Pro service to send and manage hundreds of Freedom of Information requests, has now been shortlisted for a Data Journalism Award in the Open Data category.
We’re delighted that our platform for professional users of FOI could be of help; this is just the sort of broad data-driven investigation, requiring FOI requests to multiple authorities, that it was conceived for.
You can read BIJ’s interesting account of their methodology and the impact that the project has had here. We wish them the very best of luck for the award finals next month.