Over the last year, mySociety’s research team has been trying to build a picture of how Freedom of Information functions in local government. This research project became our report into FOI in Local Government (which can be read in full here).

One of the key questions for this research project is how many FOI requests are received by local government.

We believe that use of WhatDoTheyKnow has benefits beyond people who submit requests because requests made through the site are available publicly — increasing the sum of knowledge available to all. Given this, a good metric for us to understand is what percentage of all information being released through FOI is being stored on WhatDoTheyKnow. Unfortunately, there isn’t a lot of good data in this area that allows us to make a clear comparison.

The Cabinet Office release annual statistics about FOI requests made to central government, which can be used for comparison. In 2017, 16.8% of requests sent to central government were sent via WhatDoTheyKnow — but this represents a very small volume of all use of WhatDoTheyKnow. 88% of FOI requests were sent to public authorities outside  central government, suggesting that the majority of FOI activity is elsewhere, but there is no official figure of the total number of FOI requests received by all public bodies.

In 2010, UCL’s Constitution Unit estimated a figure for all local authorities in England of 197,000 FOI requests received. We wanted to understand if this was still a good baseline for FOI requests to local government and gain new understanding of local authorities beyond England.

To do this, we sent an FOI request to every local authority (except those in Scotland, who publish these figures in a central repository) asking for a set of FOI statistics for the year 2017.

This presented an immediate set of problems. There was a split in how authorities understood ‘2017’, with internal statistics recorded in a split of financial and calendar year — a choice that has a demonstrable difference in the volume of FOIs recorded that year. A minority of councils did not respond to the FOI requests – which unaddressed would lead to an under-count in the total number of FOI requests.

To correct these problems, using the requests that were returned and other sources of public information, we constructed a model to address the issue of the split in recording year and predict a range of values for councils that didn’t return data.

The result of this is an estimate of 468,780 FOI requests received in the calendar year 2017. There is a 95% confidence this value falls between 467,587 and 469,975 (range of 2,387).  

On average an individual council receives around 1,120 requests in a given year. But as the graph below shows, this has substantial variation:

[

And the type of council has substantial impact on the number of requests recorded — with London boroughs receiving over three times as many requests as authorities in Northern Ireland.

What does this mean for WhatDoTheyKnow? Comparing totals, the 28,282 requests made through WhatDoTheyKnow in 2017 represented 6% of all FOI requests made to local authorities. Similarly, there is a lot of variation across authorities some councils have around 2% of requests start on WhatDoTheyKnow, others have around 13%.  This shows that the overwhelming majority of requests to local authorities are made through other means.

Part two of this blog post discusses what we learned about the administration of FOI from this research. You can read the full report online, or download as a pdf.

This blog post is licenced under a Creative Commons Attribution Licence (https://creativecommons.org/licenses/by/3.0/)  — it can be re-posted and adapted without commercial restriction as long as the  original article/author is credited and by noting if the article has been edited from the original.

—
Image: Ula Kuźma

Our previous blog post about our new report, Freedom of Information in Local Government, discussed our findings about the volume of requests received by local government. This second post explores our findings about how FOI is administered, working from information received via FOI requests to all councils and an anonymous survey of FOI officers.

Staff responsible for the administration of FOI in local government tend to hold this as one responsibility among several. FOI teams are generally embedded in larger teams, with few staff solely working on FOI. As such, FOI administration rarely appears as a specific budget item.

While this makes the data patchy, from the information that is available, staffing levels (and hence budget) seem to be responsive to the number of FOIs received by a council. Every thousand additional FOI requests increases the number of staff dealing with FOI by 0.75 (95% confidence this is between 0.35 and 1.15).  Similarly, use of a case management system was associated with a greater number of requests — with the use of an organised system, and then use of specialist software being predicted by increases in the number of FOI requests received.

However, use of a case management system was not associated with any increase in the percentage of requests being replied to within the statutory limit (20 days), which suggests that differences in delays are caused elsewhere than the management of incoming FOI requests. Some requests are more complex in this respect than others, with FOI officers estimating that 38% of requests required responses from multiple departments or teams in the authority and 23% required ‘double handling’ — additional sign-off from senior or specialist staff.  The number of requests appealed to internal review was low (1.4%), but within these the success rate was quite high — between 36% and 49% were successful in changing some component of the original outcome.

Councils fairly universally keep records on the number of requests received, and time taken to reply — but have fewer records on the volume of information disclosed, or on the status of appeals.

The highest availability of knowledge were figures on numbers of FOI requests received. The two areas where almost all authorities had records was the number of FOI requests received (98% recorded these figures) and how many were completed inside the statutory deadline (92%). Records of internal review were held in 87% of cases and records of appeals to ICO in 86% of cases. The questions with the most missing information related to how much of a request had been delivered. 73% had records of the number that were completely granted; 70% had records of the number that were entirely withheld; and 65% had records of partially withheld/disclosed requests.

Most councils do not publish a disclosure log (a record of FOI requests received and their responses). Adding this factor into the model used to predict missing values for the number of FOI requests received found that there was no positive or negative effect of publishing a disclosure log on the number of FOIs received. In individual responses, while many FOI Officers expressed a desire to publish more (or steps taken towards that), there was also a strong skepticism of the value of doing this, and concerns that people do not check the log before submitting their requests, meaning logs do not reduce the volume of incoming requests. Several councils that had previously run disclosure logs had discontinued them due to low usage.

An upcoming blog post will talk about what we learned about using a front end interface to reduce FOI requests by searching the disclosure log. Sign up to our FOI newsletter to hear more when released.

Part one of this blog post discusses what we learned about the administration of FOI from this research. You can read the full report online, or download as a pdf.

This blog post is licenced under a Creative Commons Attribution Licence (https://creativecommons.org/licenses/by/3.0/)  — it can be re-posted and adapted without commercial restriction as long as the  original article/author is credited and by noting if the article has been edited from the original.

—

Image: Martin Adams

—
Image: Ula Kuźma

We’ve been working over the last few years to make our research as easy to read and explore as we can. However, because we release a lot of open data (and are usually open to sharing other data with researchers) there’s also been a lot of research written by researchers outside mySociety, which of course also forms part of the knowledge base about our services.  

As such, we’re expanding the scope of the research store to include work about mySociety’s services that has been produced by researchers beyond our own team.

Where papers have been released under a Creative Commons licence but there is only a PDF file available, we will sometimes create more accessible versions. For instance, we have already done so with Emily Shaw’s research into Civic Tech Cities and Frederik M Sjoberg, Jonathan Mellon, & Tiago Peixoto’s exploration of how receiving a response through FixMyStreet affects the probability of making future reports.

This isn’t yet a comprehensive collection, but we plan to add new research as it is published, and retrospectively add older research on a rolling basis. Sign up for our newsletter to hear when new research is added.

While we’re making things easier to find — we’ve also started including mySociety’s responses to calls for evidence and consultations on the research portal, and you can see those here.

With funding from the Consumer Data Research Centre (CDRC) we’ve been working with researchers from the University of Sheffield and University of Sterling to open up FixMyStreet data for researchers.

For an example of the kind of thing that can be done with this data, this group have produced maps for every local authority in the UK, mapping FixMyStreet reports against indices of deprivation (a few examples: Sheffield, Harrogate and Cardiff). These can be explored on our mini-site, where for each authority you can also download a printable poster with additional statistics.

If you’d like to know more about what these maps mean and what we learned from the process, there’s a report exploring what we learned here.

The Inter Parliamentary Union release a report each year detailing changes in the representation of women across the world. In 2017, women represented 23.4% of all MPs – which is less than half of the proportion of women in the population at large.

While the picture for the last decade shows a positive trend, there is nothing inevitable about ever-increasing representation of women. The IPU report notes that while Albania and France’s representation of women rose by 10% and 12% respectively, other countries saw a decline. Improved representation of women is often a result of decisions deliberately taken to improve representation, rather than being a natural outcome of unstoppable social forces.

One of the pitfalls of international comparisons is that it obscures some of the drivers of good and poor representation. Increased representation of women is often uneven, and concentrated more in some parties rather than others. As Miki Caul points out, international comparisons of relative representation of women overlook “the fact that individual parties vary greatly in the proportion of women MPs within each nation”. Similarly, Lena Wängnerud argues “cross-country studies tend to miss variations between parties within a single system. Variations in the proportion of women to men are even greater across parties than across nations”.

To understand more about this, we’ve built an experimental mini site to examine the roles of parties in driving the representation of women. Using data from EveryPolitician.org (which contains gender and party information for a number of countries), we can explore the respective contributions of different parties to representation of women.

For this it’s not enough to look at the gender ratios of all the parties individually, as those with the best proportional representation of women are often quite small — for instance,  the Green Party in the UK has 100% female representation, in the form of its one MP.

Instead, what we look at is the respective contributions to the total gender ratio. For each party we look at how much better or worse the proportional representation of women would be if you ignored that party’s MPs.

For instance in the UK, while the gender ratio of the current House of Commons is around 32%, the Labour Party’s ratio is around 44%. If you take out the Labour Party the representation of women in the House of Common as a whole drops to 23%.

For our purposes, the Labour Party is the UK’s Most Valuable Party (MVP) — ignoring it leads to the largest reduction in the representation of women. For each country, the gap between the ‘gender ratio’ and the ‘gender ratio ignoring the MVP’ gives a new metric of how to understand the gap in gender representation. Where this number is high, it means that the role of individual parties is very important; where it is lower it means that the ratio is not strongly driven by party effects. For instance, the gender ratio in the United States is strongly driven by party effects, while in Bolivia it is not.

Countries with a wide gap between the ‘ratio ignoring the best party’ and ‘ratio ignoring the worst party’ tend to be countries that use majoritarian electoral systems, like the UK. Pippa Norris shows that systems using majoritarian electoral systems tend to have a poorer representation of women than those using proportional representation, but also that there is a lot of variation within each family of electoral systems and “the basic type of electoral system is neither a necessary nor a sufficient condition to guarantee women’s representation”.

Our analysis shows that parties have different levels of agency to improve the overall representation of women depending on the party structure created by the electoral system. Countries that use proportional representation tend to show smaller party effects because there are usually more parties with fewer MPs — and so the ability of any one party to shift the overall representation is reduced. Conversely, in FPTP parliaments with only a few major parties, a large amount of change can happen by only one of these major parties taking measures to improve their internal representation of women.

For example, while Germany’s CDU and the UK’s Conservative party have a similar representation of women at the national level (20.5% and 21.14% respectively), the Conservative party has more than twice the leverage to affect the overall representation of women simply by changing their own policy.

There are limits to using the proportional representation of women as a single measure for the political representation of women. As mySociety’s Head of Research Rebecca Rumbul has previously shown, even bodies with relatively good representation of women like the National Assembly for Wales can then fall down on other areas – with a low proportion of oral evidence to consultations and committees coming from women. While the UK’s Conservative party performs poorly on the proportion of MPs, it has conversely selected more female party leaders and Prime Ministers.

Importantly, looking at the representation of women as a single figure also obscures the important role of social factors as such class or race in shaping which women are represented. Creating a metric for comparison across many different countries is inherently reductive and discards important information about local context in every instance.

Our goal with this website has been to re-complicate the international comparison by moving away from a single national statistic for representation in a way that assigns agency to political actors within each country. Variations among these parties (and international variations in this variation) reflect that representation of currently under-represented groups isn’t a natural fact of life in a given country, but reflects choices made – and that other choices can lead to different outcomes.

This is still a work in progress and we acknowledge there will be holes in how this data has been applied. Lack of gender information for all countries means that some countries that have high representation of women (such as Rwanda) are not addressed. This means that it shouldn’t be taken as a comprehensive ranking — but we hope it is useful as a jumping off point for thinking about the representation of women in parliaments across the world.

We have detailed our methodology here, including known issues with the data. This is an early experiment with the data and we welcome feedback on the website here; or get in touch through the contact details here.

The data the site is built on can be downloaded from everypolitician.org.

You can explore the website here, or sign up to the research newsletter here.

—
Image: Photo by Joakim Honkasalo on Unsplash

A key part of mySociety’s research agenda is understanding how Civic Technology is (or isn’t) helping under-represented groups in society access government services and their representation. In 2015 we released a report  Who Benefits from Civic Technology, that explored variations in usage of Civic Tech in various countries and demographics. You can read or download it here.

In this blog post I’m going to talk a bit about how we’ve internally tried to apply our data to understanding the under-representation of women in politics and as users of our services, as well as some interesting things that external researchers have found using our data.

EveryPolitician

Our EveryPolitician dataset contains information on current (and in some cases historical) politicians for a large number of countries around the world. For a large number of representatives, this includes gender information.

However, a key problem of international comparisons of the representation of women is, as Miki Caul points out, that it “overlooks the fact that individual parties vary greatly in the proportion of women MPs within each nation”. Similarly, Lena Wängnerud argues “cross-country studies tend to miss variations between parties within a single system. Variations in the proportion of women to men are even greater across parties than across nations”.

Fortunately, this is exactly the kind of problem that an international dataset like EveryPolitician is well placed to examine – on Thursday we’ll be using a new mini-site to explore the gender and party information contained in EveryPolitician to give a sense of the international picture and the party-level differences within each country. Stay tuned! Or you can download the data yourself (there are APIs for Python, Ruby and R) and try and beat us to it.

TheyWorkForYou

TheyWorkForYou makes it easy to search through the history of what has been said in Parliament, and we make the data (based on the Hansard dataset but more consistently formatted) freely available to download. As essentially a download of a very large amount of text, getting insights from this dataset is a bit more complicated, but potentially very rewarding.

Jack Blumenau has a paper based on TheyWorkForYou data using language to analyse whether appointing female ministers changes how other female MPs participate in debates. Looking at “half a million Commons’ speeches between 1997 and 2017, [he demonstrates] that appointing a female minster increases the participation of women MPs in relevant debates by approximately one third over the level of female participation under male ministers” – and that “female MPs also became more influential in debates under the purview of female ministers […] female ministers respond in a systematically different fashion to the speeches of female MPs.” In this case, influence is a measure of whether the language an individual used is then taken up by others, and this kind of analysis shows how the TheyWorkForYou dataset can be used to demonstrate not just counts of how many women were in Parliament, but the substantive effects of women holding office on the political process.

As Myf talked about yesterday, TheyWorkForYou’s Commons content now extends back to 1918, and so includes every speech by a female MP ever made. We hope this is a useful resource for anyone interested in exploring the history of the representation of women in the UK and have plans for a small project in the upcoming months to show in a simple way how this data can be used (please sign up to our mailing list if you’re interested in hearing about this when it’s completed).

TheyWorkForYou data can either be accessed through an API, or downloaded as formatted XML files.

FixMyStreet and WriteToThem

Understanding the under-representation of women is important across our services. Where men and women are experiencing different issues and concerns, imbalances in access (or use of access) potentially lead to differences in resource allocation.

The majority of reports on FixMyStreet.com are reported by men – but to make things more complicated, it’s not just that women make fewer reports, but women report substantively different kinds of reports.

Reka Solymosi, Kate Bowers and Taku Fujiyama investigated FixMyStreet reports and found (by determining gender from names of problem reporters) that different kinds of reports are more likely to be reported by men and women – they suggest that at “first glance it appears that men are more likely to report in categories related to driving (potholes and road problems), whereas women report more in categories related to walking (parks, dead animals, dog fouling, litter)”.

If different kinds of reports are differently gendered, this complicates thinking about how to improve how women use the website – as potential users are having substantially different experiences of problems in the real world well before they interact with the site. We have to engage with the nuance of this kind of finding to understand how to redress issues of access to services.

We’re currently in the process of extending this kind of analysis to our other service. For WriteToThem, we’ve learned that while the majority of people using the service to write to MPs are male (around 60%), this picture is different depending on the level of government – for instance the gender balance for people writing to councils is pretty close to 50/50.

As part of this, we’re investigating whether having the same gender as their representative makes people more likely to make contact. This has some interesting preliminary findings, and we hope to have more to say about this towards the end of the year.

Our research in this area is ongoing, and we’re keen to help people use our data to investigate under-representation – especially where you have expertise or knowledge that we don’t. If you’d like to discuss potential uses of the data please get in touch, or sign up to our mailing list to hear about future research releases.

—

Image: Theresa May’s first PMQs: © UK Parliament/Jessica Taylor (CC by-nc/2.0)

Two weeks after you write to a representative on WriteToThem we send you a survey asking if they wrote back. We’ve traditionally used the data from these surveys to compare the responsiveness of individual MPs – but something we’re interested in at the moment is understanding more about systematic drivers of responsiveness. What features of a representative’s position or background makes them more or less likely to respond to messages?

The first fruit of that research is a paper in Parliamentary Affairs talking about using WriteToThem data to explore differences in responsiveness between representatives elected from constituencies and those elected from party lists in the Scottish Parliament, National Assembly for Wales, and the London Assembly.

We understand that most readers will not have journal access, so we’ve also written a summary for Democratic Audit that everyone can read here.

We’re actively investigating other factors that affect responsiveness (especially at the Westminster Parliament) and will write more in the coming months. If you’d like to make sure you don’t miss our findings, you can sign up to the research mailing list here.

—
Image: Chris Flexen (Unsplash

mySociety services produce a lot of useful (and interesting!) data. Over the years we’ve often made components or the results of mySociety services available through APIs (like our MapIt service) or as open data to download (such as our EveryPolitician data).

What we haven’t been good at is showing you the full breadth of what we have available, or how component parts can be used together. Sometimes we find users of one aspect of mySociety data being unaware of other relevant datasets.

To fix this problem, we’ve created a new data portal – data.mysociety.org –  to bring all the data we publish into one place. From the politicians of Albania to data about all ministerial and parliamentary roles UK MPs have held, everything can be found on one site.

Our research team will also use this site to publish supplementary materials to papers and blog posts that might be of use to others (such as a lookup table for the different codes used for UK Local Authorities). So we plan to keep adding data whenever we can!

—
Image: Ash Edmonds (Unsplash)

How many Freedom of Information requests are sent through WhatDoTheyKnow as compared to those made directly to public bodies? Our new mini-site lets you explore Cabinet Office statistics in comparison to numbers from WhatDoTheyKnow.

Every quarter, the Cabinet Office releases Freedom of Information stats for a collection of central government ministries, departments and agencies. This provides a good benchmark for understanding how requests made from WhatDoTheyKnow relate to requests made through other routes. Back in 2010 we ran several blog posts about this, though we haven’t released any comparisons in recent years — and we’re now making up for lost time.

In 2016, WhatDoTheyKnow was the source of 17.14% of requests to audited public bodies. On the other hand, most WhatDoTheyKnow requests (88.51%) went to public bodies that the Cabinet Office figures don’t cover.

One interesting conclusion from this is that most FOI activity in the UK is not immediately visible from the official statistics. You can read more about what we learned from the numbers, or explore the data for yourself on the mini-site.

—
Image: Jerry Kiesewetter (Unsplash)

For the last few years mySociety’s research output has been living in its own little area of the main website. At the start this was fine, but as we’ve produced more research (which is good!) the website was not good at making clear what we had previously released and why you should read it (which is bad!).

To fix that we’ve brought all our research reports, papers and blog post together in one place. We also wanted to take the opportunity to make our research easier to access. For all our research going back to 2015, we now have a nice, mobile-responsive, easy-to-read version, as well as a text and a kindle .mobi file to go along with that. In several cases papers that had been published externally were released by the publisher under a Creative Commons licence – meaning these could be converted to the new format.

So have a look around the new site! Learn about Civic Tech Cities, or Who Benefits From Civic Technology.

And don’t forget that you can sign-up for our research newsletter for exciting research updates!

_

Image: Nico Kaiser