Every quarter the Cabinet Office releases Freedom of Information statistics for a collection of central government ministries, departments and agencies. This provides a benchmark for understanding how requests made from WhatDoTheyKnow relate to FOI requests made through other methods. From 2017, mySociety started retrospectively tracking the proportion of FOI requests sent via WhatDoTheyKnow to central government using a minisite — https://research.mysociety.org/sites/foi/ — that explores the data.
This report explores what had and hadn’t changed in the last few years, as well as the number of requests made through WhatDoTheyKnow Pro — a new service being piloted that allows embargos of the results of FOI requests for a period — with the goal of bringing more people making FOI requests professionally (such as journalists) into the system and leading to more raw results being made available after the conclusion of a project.
We found that:
- WhatDoTheyKnow accounted for between 15-17% of audited bodies and between 18-21% of ministerial departament FOI requests.
- While the proportion of requests have grown most years since 2010, there was no real change from 2017 to 2018.
- Requests made to central government via WhatDoTheyKnow only make-up around 9-10% of all requests sent via WhatDoTheyKnow in 2018.
- WhatDoTheyKnow Pro requests made up 1% of FOI requests to central government — but most requests using this service went to other areas of the public sector.
In a strike for transparency, journalist Jenna Corderoy has secured the release of documents from the European Research Group (ERG), the pro Brexit lobby of which Jacob Rees-Mogg is a prominent member.
For more than a year, Jenna has been striving to ensure that the facts around Brexit — and the funding that drives it — reach the public domain: she also broke the now-infamous revelations about Vote Leave’s campaign overspending.
The release of material such as this into the public domain is beneficial to all, as it means that public debate is based on facts rather than conjecture. FOI can be a vital tool in ensuring that the documents shaping our society’s future direction are available for scrutiny.
On this latest release, a piece by Jenna and Peter Geoghegan reports:
“The ERG is part-funded by subscriptions paid out of MPs’ parliamentary expenses. As a consequence the group has to supply samples of its research for scrutiny to the Independent Parliamentary Standards Authority [IPSA] to ensure public money is being properly spent and not used for party political campaigning.”
Using WhatDoTheyKnow Pro — our service for professional users of FOI, which among other features, allows users to hold off from putting request correspondence in public until a story has been published — in January 2018 a request was made to IPSA to see these materials.
The request’s WhatDoTheyKnow page is now public. IPSA initially argued that the release of these materials was exempted under section 43 of the FOI Act as it would prejudice the commercial interests of the ERG, whose research is ordinarily available only to those paying a £2,000 annual subscription.
Subsequently Jenna referred this refusal to the Information Commissioner, who upheld the decision. Determined that the public has the right to see the research, Jenna and Peter did not leave the matter there, taking it to an information tribunal.
The tribunal made the final decision that the material must indeed be released, vindicating the effort and determination Jenna put into pursuing this request and stating that to make the documents available would:
“further transparency, accountability and public trust with respect to the working of Parliament”.
As a result, the documents will be made available on 11 July — keep an eye on OpenDemocracy for news of their release — and we’ll make sure we update the annotations on the original request as further details unfold. Meanwhile, you can see the full tribunal decision here.
Image: Udur Akdemir
We’ve just released two new versions of Alaveteli, our open source platform for running Freedom Of Information sites — one which we’ve packed all of bug fixes, new features and performance improvements into, and one with some important technical updates. Here are some of the highlights.
Making your first request easier
mySociety’s designer Zarino has subtly redesigned the authority search page – which is the first thing most users will see when making a request – to make it easier to find the help links at a glance, making the process feel much more approachable.
Encouraging better quality requests
Part of our advice to users of WhatDoTheyKnow is to keep requests concise and focused. This new feature adds a visual reminder when the request text is approaching the point of being too long for an FOI officer to answer it efficiently:
With a second, stronger warning if the request becomes longer still:
We’ve also added an experimental feature on WhatDoTheyKnow to discourage users from making requests for personal information. (As well as not being valid FOI requests, publishing the responses would result in revealing confidential information about the requester.) By asking the user to clarify what they want to ask before sending a request to authorities which attract a high level of personal information requests, we can try to redirect them to the correct official channel instead:
One-click unsubscribe from notifications
An improvement for site owners and end users: it is now possible to unsubscribe from email notifications about requests that you’re following right from your inbox! (Previously – and still the case with other sorts of email that the site sends – you would have to visit the site and log in to change your email preferences.)
Allowing people to unsubscribe with ease – if they’ve forgotten that they signed up for notifications or have created a new track by mistake – should help to cut down on the number of messages being marked as spam which should in turn help improve the site’s mail sender reputation. (It also allows admins – if they receive feedback loop notification emails from email providers – to unsubscribe users who’ve marked these emails as spam, preventing further unwelcome emails being sent.)
In version 0.33, as well as our our features and fixes, we’ve added support for Ubuntu releases Xenial and Bionic and withdrawn support for Debian Jessie. The full list of highlights and upgrade notes for this release is in the changelog.
Version 0.34 contains our Rails 5.0 upgrade work which we’ve released separately to allow reusers time to adapt to new minimum requirements for operating systems and Ruby versions.
We’re no longer supporting Ubuntu Trusty and have also dropped Ruby versions older than 2.3 as that’s the minimum requirement for Rails 5. Upgrade notes are available in the changelog and, as ever, we recommend updating one version at a time to make sure everything’s working smoothly and reduce the risk of missing essential upgrade steps.
Moving to Rails 5.0 will allow us to retain support for major security issues when Rails 6 is released and dropping older Ruby versions removes some key technical barriers to modernising the Alaveteli codebase and allows us to focus on improving Alaveteli Pro so that it can be reused more widely.
Thanks again to everyone who’s contributed! Special thanks to Nigel Jones and Laurent Savaëte who contributed bug fixes for version 0.33!
A study published in 2017 by Reka Solymosi, Kate J Bowers, and Taku Fujiyama used publicly available data for FixMyStreet to investigate (among other things) whether men and women reported different things using the site, and found a gender divide relating how people were moving around when they found the problem:
[M]en are more likely to report in categories related to driving (potholes and road problems), whereas women report more in categories related to walking (parks, dead animals, dog fouling, litter.(p. 954).
A potential limitation of this study was that it could only use reports that weren’t publicly anonymous, as the reporter’s name was used to approximate gender. If there was a gender skew in terms of which users were more likely to report anonymously, this might mistakenly pick up differences in anonymisation as a gender divide (for instance, if a lot of women were reporting potholes, but were more likely to do so anonymously).
To investigate this, we internally replicated the study on both anonymous and non-anonymous reports. This found that there was a gender skew related to anonymisation, with women being 10% more likely to report anonymously and that some types of report were more likely to be reported anonymously than others.
However, despite this factor, the original study’s conclusion was validated by this analysis. The categories highlighted are differently gendered when including the non-anonymous data, with men reporting far more problems with road surfaces and women reporting more litter related issues.
Future blog posts will further explore reasons and implications of this divide. The replication can be read online or downloaded as a PDF.
If you’re a councillor who’d like to find out how our services can help you work more efficiently — and bring benefits to your residents — please do swing by for a chat at stand BL3.
We’ve written a lot about our street reporting service for councils — how it can integrate with existing back-end systems; how it can encourage channel shift and thus bring savings; and the many new features we’ve introduced in response to what councils tell us they need. You can read all our past posts on the FixMyStreet Pro blog.
But as a councillor, you may be interested in other aspects of the service. Here are a few highlights:
- FixMyStreet lets you subscribe to the reports being made in your ward — you’ll get an email every time someone makes a new report. This allows you to monitor issues as they occur, and take action if it’s warranted.
- You can also access a map showing every report ever made in your ward. If desired, you can filter reports by category or by status to get a picture of how each type of report, from graffiti to potholes, is impacting your residents.
- If your council is one of the many who use FixMyStreet Pro as their main reporting system, you’ll also have access to more refined analysis via the dashboard, which allows you to compare reports and fulfillment over different periods of time.
- You can make reports on the go, so if you spot something that needs fixing while you’re out and about, it’s quick and easy to get a report filed.
Keep It In The Community
Also come and discover Keep It In The Community, an England-wide online mapping of the spaces and places that are valuable to local communities, created in partnership with Power To Change.
Under the Localism Act of 2011, every council is obliged to retain a list of Assets of Community Value (ACVs): Keep it In The Community turns this obligation into a benefit for all, allowing you to store and share your data while contributing to a national picture.
Thanks to a recent update, Keep It In The Community also displays buildings and spaces currently under community ownership. As a councillor, we think you’ll like this service because:
- It’s completely free.
- It provides an attractive way for councils to display ACVs and community-run spaces, and invites residents to add richer detail such as memories and photographs.
- It’s a great way to demonstrate the community activity that’s taking place within your ward.
- It helps popularise the concept of community ownership, encouraging more residents to take action and preserve the spaces that matter to them.
If this has whetted your interest, don’t forget to come and meet the friendly mySociety and Power to Change folk on stand BL3.
Buckinghamshire County Council have revealed the cost savings brought to them by FixMyStreet Pro.
The authority switched over to FixMyStreet Pro as their official fault reporting system in April 2018. They’re now able to assess a year’s worth of data and compare it to the year previous. The findings are gratifying, to say the least — and set out a real case for councils who are considering opting for the service themselves.
Saving staff time and resources
The council reports that they’ve seen a 13% decrease in calls and a 40% reduction in emails about street faults since FixMyStreet Pro was introduced.
In case you’re wondering how that translates into monetary savings, well, on average they reckon that a single call costs £5.88 in staff time, while a report made by email, with its potential for back and forth communication to pin down the precise details, chalks up £7.81.
In comparison, because FixMyStreet Pro places reports directly into the system, and little staff time is required to administer them, the perceived cost is just 9p per report.
Additionally, Buckinghamshire has seen a 29% drop in calls where residents are chasing progress: report makers no longer need to get on the phone to check whether their issue is being seen to, because updates are pushed directly back to them as the report progresses through the system.
And there’s been a 59% decrease in unnecessary clarification, that is, when the council need to go back to the report-maker to check the exact location or nature of an issue. FixMyStreet can be set up to the council’s exact specifications to ensure that the user is prompted to provide all the information they’ll need, which accounts for this impressive drop.
Avoiding unnecessary reports
It can be a frustrating waste of time and resources when a council receives reports about issues which are not their responsibility: with the UK’s two tier system, it’s almost inevitable that citizens get confused about which authority deals with which category of street fault — and on top of that, there are the reports that are dealt with by other bodies such as TfL or Highways England.
FixMyStreet has always done a good job of routing reports to the right council, though, and the improvements we’ve made to the service over the last few years mean we can also make sure the relevant reports go through to TfL and Highways England too. Bucks say that since introducing FixMyStreet Pro, they’ve seen a 19% decrease in misrouted reports that have to be forwarded elsewhere.
Finally, they can see a 30% decrease in street light reports. Since Bucks are one of the councils who display all their streetlights on FixMyStreet it’s now very easy for a resident to check online whether an issue has already been reported for any specific lamp post. If it has, they can also see its progress towards resolution — so there’s no need for them to open a new report.
These figures illustrate very clearly what is meant by channel shift: real, tangible results that save money for councils, and thus ultimately, for residents too. It’s great to have this confirmation that FixMyStreet Pro brings results — and we’re still in a continual process of development in consultation with councils, to keep making more improvements wherever we can.
Come and talk to us at the LGA conference next week
We’d be delighted to answer your questions and give you a demo if you’re planning on being in Bournemouth for next week’s LGA conference. You’ll find us on stand BL3 in the Purbeck Hall.
As part of the recent work we’ve been doing around meaningful citizen participation in democratic decision making, mySociety have been investigating how digital tools can be used as part of the process of a Citizens’ Assembly.
We reviewed how Citizens’ Assemblies to date have used digital technology, and explored where lessons can be learned from other deliberative or consultative activities.
While there is no unified digital service for Citizens’ Assemblies, there are a number of different, individual tools that can be used to enhance the process — and most of these are generic and well-tested products and services. We also tried to identify where innovative tools could be put to new uses, while always bearing in mind the core importance of the in-person deliberative nature of assemblies.
We found that digital tools have potential uses in many parts of the process, which we grouped in three areas:
Preparation: bringing the public in
- Question forming
- Public submissions
- Finding experts and stakeholders to give evidence
Internal: facilitating assemblies
- Attendance management
- Tools for coming to decisions in the assembly (voting)
- Sharing assembly materials to members
- Including a wider range of experts
- Enabling online deliberation for assembly members outside the face-to-face sessions
External: sharing products
- Sharing the conclusions of the assembly
- Streaming of evidence/plenary sessions
- Sharing evidence submitted to inquiry
- Tracking implementation of recommendations
- Communicating participants’ experiences
- Allowing feedback from non-participants on the outcome
Above all when considering the use of digital tools, it’s important that the final choice is appropriate to the aims of the project — and will typically be complementary rather than taking a centre-stage role. Digital tools can reduce costs and enhance the process by creating resources that add greater depth and knowledge to the process, but shouldn’t detract focus from the importance of the core deliberative activity of the assembly.
After over five years of active development we have decided to pause work on the EveryPolitician project for the foreseeable future.
In this post we’ll outline where we are leaving things, how you can make use of the data that does exist, and how you might be able to help migrate or transfer some of what we’ve collected over to services like Wikidata.
What’s in place today
The EveryPolitician project is, as its name suggests, based on the simple idea to gather accurate and up-to-date data on every politician in the world, collated and shared in a consistent format for free download and use by researchers, democracy projects, campaigners and individual citizens.
Over the course of the project we have gathered, structured and shared data on 78,382 politicians from 233 countries and territories presented on EveryPolitician.org via hundreds of scrapers run on morph.io and hosted on GitHub, producing the data on everypolitician-data.
Mostly the data covers the main chambers of recent parliaments around the world, but it also includes thousands of entries for previous parliaments, in some cases going back decades.
This has been a sizeable undertaking, involving a handful of very talented developers and colleagues within mySociety, as well as contributions from dozens of other organisations and individuals, many of whom make use of the data within their own projects.
The reality is that this work is hugely time consuming, complex and requires not just expert knowledge but a commitment to go deep into the intricacies of parliamentary data in order to make it comprehensible to a wider group of users. And looking to the next couple of years this task is only ever going to increase in complexity — too much for one underfunded organisation.
We therefore intend to freeze the current data as it currently stands, and it will continue to be available for download and reuse. We just can no longer commit to keeping this data up to date.
Always playing catch up
The challenge with data projects like EveryPolitician, beyond the complexity of understanding the structures and relationships within hundreds of individual parliaments (every parliament is an edge case in some way), is that the data is always steadily going out of date.
Across the world’s national parliaments there is an election somewhere roughly once a week, and that’s often when parliaments choose to update their websites, sometimes breaking our scrapers and changing the format of the data. Throughout the life of a parliament you might expect a few percent of MPs to change, sometimes more in different systems, so keeping on top of those individual changes is a sizeable task – especially where errors or duplications occur.
In addition to managing the hundreds of scrapers, we also included data from other sources — increasingly from Wikidata. Over the past 18 months we’ve been attempting to migrate more and more of what we’ve learned on EveryPolitician over to Wikidata via the WikiProject every politician.
Where the project goes next
EveryPolitician was built on the many years of work we had already delivered in this area, through PopIt, Poplus and working with Popolo. We knew what was needed, what worked and what didn’t.
We saw the potential to create an Open Corporates for political data, and hoped that EveryPolitician would be able to attract grant funding to grow, and potentially develop appropriate commercial services in support.
However, after five years of significant investment we just don’t have the funding to continue this work on our own.
In time we hope to be able to continue to contribute again to the wider availability of political data, and with hindsight it’s clear that Wikidata should be the natural global home for this type of data – benefitting from much greater reach, the contribution of motivated individuals in each country, and from the wider Wiki community.
As part of our contribution to Wikidata, we’ve created numerous tools to support the cross-referencing, verification, and supported update of data between EveryPolitician and the Wikiproject. This is still something of a work in progress, but we see it as a key way that others might contribute and take on aspects of the project in the future.
In the meantime we hope that many people continue to make use of the wealth of data that’s already been collected.
If you have a specific interest in a country, group of legislatures or some other combination, perhaps you can consider adding the kind of data that EveryPolitician has collected to Wikidata. We have no further resources to devote to this work; however if you do have an interest in taking some of this on then we will try to advise what options might best suit.
Image: Jelle van Leest
Our own FOI site WhatDoTheyKnow is always interesting to browse, but we suspect that even the gems waiting to be discovered there might pale in comparison to the 13 million pages of declassified files released by the CIA over in the States.
These are available to American citizens — and indeed the world — thanks to sustained efforts from our friends over at MuckRock, the US FOI site*.
In 2016 MuckRock won a three year fight compelling the CIA to abide by the nation’s FOIA law and release their files: the history of how the CIA had dodged their obligations for so long is amusingly written up in this post.
Now MuckRock are encouraging users — including you, if you would like — to browse the content and let them know of anything interesting you discover. They’re always happy to share the more useful, fascinating or downright bizarre information unearthed.
Never let it be said that FOI is dull or dry: so far they’ve written up almost 300 findings, including a recipe for borscht, an Edgar Allen Poe parody, a guide to christening ships and the very mysterious picture of a man.
You can find guidance on how to tackle this vast archive on the Muckrock site. If you discover anything worth telling MuckRock about, please let them know we sent you.
Kudos to MuckRock and their tenacious users for their work in getting these files into the public domain.
* Unlike many of the international FOI sites we write about, MuckRock isn’t run on our Alaveteli platform, but it shares the same aims and we’re proud to be working together. In fact, as we were writing this, members of mySociety’s Transparency team were over in Tunis at RightsCon, giving a joint presentation with MuckRock, and they’ll be coming to our FOI technologies conference AlaveteliCon in September. And here’s a picture of us meeting up in the UK.
Image: President Ford meets with CIA Director-designate George Bush (via Wikimedia; public domain)
On 26 – 27 of June, scholars and practitioners from all over the world will be meeting in Rio de Janeiro for the 6th Global Conference on Transparency Research. The conference focuses on measuring transparency, exploring how this can be achieved, what the barriers are, whether metrics are useful, and how current interventions are shaping transparency around the world.
mySociety’s Head of Research Rebecca Rumbul will be attending, and will be presenting some of mySociety’s recent research into the transparency of parliamentary information in sub-Saharan Africa. Examining transparency through a digital lens, this research broke new ground in understanding how digital tools are shaping parliamentary transparency in sub-Saharan Africa, and how barriers to transparency are affecting how citizens engage with public institutions. You can read the full report here.
Rebecca will be speaking at 4pm on Thursday 27 June, so please do come along and say hello. She says, “Transparency, digital and citizen engagement are core themes of our research at mySociety, and we love to talk to other people working in these areas. Meeting new people and sharing ideas are the best parts of any conference, so do grab me for a chat if you are attending.”
If you are unable to join Rebecca in Rio, but you are interested in talking research, we’re always happy to receive email. And keep your eyes peeled for our TICTeC conference announcement for April 2020. We will be opening our Call for Papers in early September.
Image: Jaime Spaniol