This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here. 

The FixMyStreet section of the Explorer mini-site helps explore the relationship between demographic features and FixMyStreet reports.

In one use case, it maps the location a report was made to a ‘neighbourhood’ sized area, and then in turn to sets of statistics measured against those areas — most importantly, the indices of multiple deprivation.  These areas are Lower Super Output Areas (LSOAs) in England and Wales, Data Zones (DZ) in Scotland and Lower Output Areas (LOA) in Northern Ireland (although NI is not covered separately in the Explorer site due a relative lack of data). These can be seen as equivalent to census tracts in the US and each LSOA has a population of around 1,500 people, while Data Zones have around 500-1000 people.

While this statistical unit feels neighbourhood-sized and so is used to examine data for effects that may result from being in the same neighbourhood, the approach has the significant problem that what people on the ground perceive as their “neighbourhood” is unlikely to exactly overlap onto the statistical unit. On the edge of a LSOA, even a 50m radius around a home will cross into another statistical area.

Making the problem worse is that the idea of a neighbourhood is very variable. People can disagree with each other about the boundaries of their area. Claudia Coluton, Jill Korbin, Tsui Chan, Marilyn Su (2001) found that when citizens were asked to draw the boundaries of their neighbourhood these very rarely aligned with US census tracts. As the gif in this tweet shows a set of citizen-drawn boundaries for Stoke Newington in East London, and while there is a clear core, there is substantial disagreement between residents about the size of this area.

Laura Macdonald, Ade Kearns and Anne Ellaway (2013)  found that residents in West Central Scotland had a different perception of how well placed they were for ‘local’ amenities compared to the geographic distance. This reflects that what was viewed as local from the outside might not be viewed the same way by locals: there is a context gap that just cannot be bridged at this scale of analysis.

Understanding of neighbourhood effects is often positioned in terms of guardianship of a home area, and this means that certain kinds of reports might be more apparent in areas where these boundaries are less clear — leading to conflict. Joscha Legewie and Merlin Shaeffer (2016) used New York 311 calls to demonstrate that complaints about blocked driveways, noise from neighbours and drinking in public were more frequent on the boundaries of areas with differing demographics. This can also be seen in the idea that complaints about dog fouling are used for score-settling between neighbours in Chicago. Complaints can be about conflicts as well as actual problems reported.

In a related problem, Alasdair Rae and Elvis Nyanzu show in some areas the most deprived 10% of areas and the least deprived 10% are not far from each other. This means that relationships between reports and the features of deprivation might be harder to detect. The less homogenous the area, the greater the chance that features affecting how likely a person is to report will result in reports in a LSOA that is substantially different from their ‘home’ area.

This blog post is exploring a potential problem with the explorer minisite methodology. A big part of what the explorer site is doing is trying to show how much different kinds of reports are “explained” by different local features — but because of various forms of fuzziness the differences it detects may be less sharp than actually exists. In general, however, not detecting things that are there is a better problem to have than the opposite.

This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here. 

A common feature in websites and services where users generate data is that a small amount of users are responsible for a large percent of the activity. For instance, 77% of Wikipedia is written by 1% of editors (with most of that being done by an even smaller fraction) and for OpenStreetMap 0.01% of users contribute a majority of the information.

This also applies to plenty of offline activities — for instance, half of the 25,000 noise complaints about Heathrow Airport were made by 10 people. People who dedicate significant time to an activity can quickly outpace a much larger group who only use the service once.

For FixMyStreet (where people report issues like littering and potholes to local authorities), the top 0.1% of users made 16% of the reports and 10% of users account for 62% of reports. Starting from the most prolific users, increasing the number of users by a factor of 10 roughly doubles the number of reports:

• 418 users (0.1%) account for 224,775 reports (16%)
• 4,181 users (1%) account for 470,384 reports (33%)
• 41,814 users (10%) account for 881,481 reports (62%)

This reflects that at any scale in the data, around half the activity is happening in the top 10%. Overall, two-thirds of users made only one report — but the reports made by this large set of users only makes up 20% of the total number of reports.

This means that different questions can lead you to very different conclusions about the service. If you’re interested in the people who are using FixMyStreet, that two-thirds is where most of the action is. If you’re interested in the outcomes of the service, this is mostly due to a much smaller group of people.

Reka Solymosi (2018) investigated the behaviour of the top 1% of reporters and found that they tended to report a wide range of categories: only “16 of the 415 contributors reported only one type of issue. The other 399 reported issues in more than one category” with an average of six categories. These also tended to cover a wide area and “there were only six people who reported in only one neighborhood [LSOA], fewer than the number of people who reported in only one category. The other 409 contributors all reported in at least two neighborhoods”. Solymosi finds four clusters of these super-contributors:

• Traditional guardians – these report in a small number of neighbourhoods covered but represent the largest number of users.
• Large-neighbourhood guardians – Report in a larger number of connected neighbourhoods.
• Super-neighbourhood guardians – People who report in a high number of connected neighbourhoods; this is the largest group.
• Neighbourhood agnostic guardians – reports are made in disconnected areas.

Collectively, this can have a wide impact — 18% of LSOAs in England have at least one report from a user who has made more than 100 reports (which is only around 900 people).

Looking at the general picture through the Explorer minisite, it’s not just that serial reporters report widely; certain kinds of reports are more likely to be made by users who are reporting more issues:

Incivilities, rubbish, road safety and bus stop damage are all categories more likely to be reported by users who have made 50+ reports. While users who make lots of reports tend to make reports across a few categories, they are often specialised in their output.

59% reports of flyposting, 57% of graffiti, 52% of litter problems are made by users who have reported more than 50 times.

It’s important to remember that these aren’t hard divides. Single report users are less likely to report potholes than serial reporters, but it is also true that one in five people who only report one issue report a pothole.

For the bundle model of understanding FixMyStreet, thinking about this group of super contributors is important, because they represent a minority of users, yet generate most of the value and impact of the site.

But this comes with a cost. People living in the same area as super contributors benefit from their efforts – but where these super contributors have different concerns or priorities from the area as a while this might shift the outcomes of the service.

As Muki Haklay argues:

The specific background and interests of high contributors will, by necessity, impact on the type of data that is recorded. This is especially important in VGI [volunteered geographic information] projects where the details of what to record are left to the participants.

Where resources are allocated on the basis of data generated by a service, the behaviour of this small group can have an outsized effect. Future blog posts in this series will explore what this looks like in practice.

This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here.

A key question when looking at the role of the internet in civic life is whether it changes the demographics of who participates; or whether it simply changes the methods by which already engaged citizens participate. The two sides in this argument can be described as mobilisation and reinforcement.

The mobilisation argument says that the internet reduces the cost of communication and action, which means that more people can be involved and access becomes more broad.

The reinforcement argument says that the reduced costs of connectivity will mostly reinforce existing participation divides, making it cheaper for people already engaged to participate, but not necessarily reaching disengaged people.

This is a fundamental question for civic tech: how are these online tools used? Are they mobilising everyone or just providing more efficient processes for people who are already engaged?

This is explored in mySociety’s 2015 report Who benefits from Civic Technology?, and is a recurring question in much of our research since, such as our work on FixMyStreet, and digital technologies in sub-Saharan Africa.

Two themes we are currently investigating in this area are proxies and bundles.

Proxies are where services are used by intermediaries, on behalf of — and bringing benefits to — others: for instance, where charities engage in more effective lobbying as a result of free access to TheyWorkForYou, or where case workers find it easier to identify and write to a client’s local councillor using WriteToThem.

Bundles are about exploring how different groups of users use a service in different ways, to such an extent that one service can in fact be understood as a bundle of services serving different kinds of users.

This is the first in a series of blog posts investigating  bundles.

A common finding across mySociety services is that most people only use “transactional” services (like WhatDoTheyKnow, FixMyStreet or WriteToThem) once, to do one thing. Repeat users make up a minority of users (even if they account for the majority of actual usage).

From a technology point of view or an organisational point of view, it makes sense to understand that there is a website called FixMyStreet.com run by mySociety. But from the point of view of the majority of users, it makes sense to think of a website like FixMyStreet as dozens of different services, most of which they will never use. For one user,  FixMyStreet is a tool for reporting potholes, for another it is for reporting littering. Similarly, WriteToThem is most often used as a tool to write to MPs — but the profile of people who use it to write to their local councillors is very different.

Some services in a bundle are used by a different demographic to other uses of the same website. Understanding how to encourage FixMyStreet use in underrepresented groups requires an understanding of how there are already differences in usage across all the “services” in the FixMyStreet bundle.

To get more information about these different uses of a website, we’ve built a mini-site that helps to explore basic demographic information about each use type. Starting with FixMyStreet, personal information (names) have been anonymised and converted to gender (approximately), while coordinates are grouped into Lower Super Output Areas (LSOA) — geographic areas commonly used for statistical purposes. This means that we can look at a general, anonymised set of data representing people making FixMyStreet reports, and match this grouped data against various measures of deprivation.

Understanding more about these different patterns of users suggests possible ways a service can be used and helps sharpen new research questions.

When examining uses of one element of a bundle, the key question is whether the pattern observed reflects just the individual, or the overall pattern of the bundle. To answer this, a chi-square test is used to tell if the distribution of a sub-use of the site is different to a statistically significant extent to all other uses of the site (this method was inspired by an analysis of gender of reporters in Reka Solymosi, Kate Bowers and Taku Fujiyama’s 2018 paper on FixMyStreet). The groupings of categories in FixMyStreet use Elvis Nyanzu’s meta categories.  The mini-site highlights in red and green areas where a distribution differs from how patterns on the site as a whole respond.

We’ll be writing a number of blog posts over the next few months covering things we’ve learned from the mini-site. The first two are already up (and linked below):

• What’s a neighbourhood?
• Power laws and super contributors in FixMyStreet

This page will be updated as new blog posts are released.

Every quarter the Cabinet Office releases Freedom of Information statistics for a collection of central government ministries, departments and agencies. This provides a benchmark for understanding how requests made from WhatDoTheyKnow relate to FOI requests made through other methods. From 2017, mySociety started retrospectively tracking the proportion of FOI requests sent via WhatDoTheyKnow to central government using a minisite —  https://research.mysociety.org/sites/foi/ —  that explores the data.

This report explores what had and hadn’t changed in the last few years, as well as the number of requests made through WhatDoTheyKnow Pro — a new service being piloted that allows embargos of the results of FOI requests for a period — with the goal of bringing more people making FOI requests professionally (such as journalists) into the system and leading to more raw results being made available after the conclusion of a project. 

We found that:

• WhatDoTheyKnow accounted for between 15-17% of audited bodies and between 18-21% of ministerial departament FOI requests.
• While the proportion of requests have grown most years since 2010, there was no real change from 2017 to 2018.
• Requests made to central government via WhatDoTheyKnow only make-up around 9-10% of all requests sent via WhatDoTheyKnow in 2018. 
• WhatDoTheyKnow Pro requests made up 1% of FOI requests to central government — but most requests using this service went to other areas of the public sector.

You can read the whole report online, download the PDF, or explore the data.

A study published in 2017 by Reka Solymosi, Kate J Bowers, and Taku Fujiyama used publicly available data for FixMyStreet to investigate (among other things) whether men and women reported different things using the site, and found a gender divide relating how people were moving around when they found the problem:

[M]en are more likely to report in categories related to driving (potholes and road problems), whereas women report more in categories related to walking (parks, dead animals, dog fouling, litter.(p. 954).

This study is open access and available online, and you can also watch Reka’s 2018 TICTeC presentation on the subject. 

A potential limitation of this study was that it could only use reports that weren’t publicly anonymous, as the reporter’s name was used to approximate gender. If there was a gender skew in terms of which users were more likely to report anonymously, this might mistakenly pick up differences in anonymisation as a gender divide (for instance, if a lot of women were reporting potholes, but were more likely to do so anonymously).

To investigate this, we internally replicated the study on both anonymous and non-anonymous reports. This found that there was a gender skew related to anonymisation, with women being 10% more likely to report anonymously and that some types of report were more likely to be reported anonymously than others.

However, despite this factor, the original study’s conclusion was validated by this analysis. The categories highlighted are differently gendered when including the non-anonymous data, with men reporting far more problems with road surfaces and women reporting more litter related issues.

Future blog posts will further explore reasons and implications of this divide. The replication can be read online or downloaded as a PDF.

As part of the recent work we’ve been doing around meaningful citizen participation in democratic decision making, mySociety have been investigating how digital tools can be used as part of the process of a Citizens’ Assembly.

We reviewed how Citizens’ Assemblies to date have used digital technology, and explored where lessons can be learned from other deliberative or consultative activities.

While there is no unified digital service for Citizens’ Assemblies, there are a number of different, individual tools that can be used to enhance the process — and most of these are generic and well-tested products and services. We also tried to identify where innovative tools could be put to new uses, while always bearing in mind the core importance of the in-person deliberative nature of assemblies.

We found that digital tools have potential uses in many parts of the process, which we grouped in three areas:

Preparation: bringing the public in 

• Question forming
• Public submissions
• Finding experts and stakeholders to give evidence

Internal: facilitating assemblies

• Attendance management
• Tools for coming to decisions in the assembly (voting)
• Sharing assembly materials to members
• Including a wider range of experts
• Enabling online deliberation for assembly members outside the face-to-face sessions

External: sharing products

• Sharing the conclusions of the assembly
• Streaming of evidence/plenary sessions
• Sharing evidence submitted to inquiry
• Tracking implementation of recommendations
• Communicating participants’ experiences
• Allowing feedback from non-participants on the outcome

Above all when considering the use of digital tools, it’s important that the final choice is appropriate to the aims of the project — and will typically be complementary rather than taking a centre-stage role. Digital tools can reduce costs and enhance the process by creating resources that add greater depth and knowledge to the process, but shouldn’t detract focus from the importance of the core deliberative activity of the assembly. 

The document can be downloaded as a PDF, but we’d also like to be able to respond to feedback and update as time goes on, so the document is also available as a Google Doc open for comments. 

This work was supported by the Department for Digital, Culture, Media and Sport and by Luminate, through the Public Square programme. 

Over the last year, mySociety’s research team has been trying to build a picture of how Freedom of Information functions in local government. This research project became our report into FOI in Local Government (which can be read in full here).

One of the key questions for this research project is how many FOI requests are received by local government.

We believe that use of WhatDoTheyKnow has benefits beyond people who submit requests because requests made through the site are available publicly — increasing the sum of knowledge available to all. Given this, a good metric for us to understand is what percentage of all information being released through FOI is being stored on WhatDoTheyKnow. Unfortunately, there isn’t a lot of good data in this area that allows us to make a clear comparison.

The Cabinet Office release annual statistics about FOI requests made to central government, which can be used for comparison. In 2017, 16.8% of requests sent to central government were sent via WhatDoTheyKnow — but this represents a very small volume of all use of WhatDoTheyKnow. 88% of FOI requests were sent to public authorities outside  central government, suggesting that the majority of FOI activity is elsewhere, but there is no official figure of the total number of FOI requests received by all public bodies.

In 2010, UCL’s Constitution Unit estimated a figure for all local authorities in England of 197,000 FOI requests received. We wanted to understand if this was still a good baseline for FOI requests to local government and gain new understanding of local authorities beyond England.

To do this, we sent an FOI request to every local authority (except those in Scotland, who publish these figures in a central repository) asking for a set of FOI statistics for the year 2017.

This presented an immediate set of problems. There was a split in how authorities understood ‘2017’, with internal statistics recorded in a split of financial and calendar year — a choice that has a demonstrable difference in the volume of FOIs recorded that year. A minority of councils did not respond to the FOI requests – which unaddressed would lead to an under-count in the total number of FOI requests.

To correct these problems, using the requests that were returned and other sources of public information, we constructed a model to address the issue of the split in recording year and predict a range of values for councils that didn’t return data.

The result of this is an estimate of 468,780 FOI requests received in the calendar year 2017. There is a 95% confidence this value falls between 467,587 and 469,975 (range of 2,387).  

On average an individual council receives around 1,120 requests in a given year. But as the graph below shows, this has substantial variation:

[

And the type of council has substantial impact on the number of requests recorded — with London boroughs receiving over three times as many requests as authorities in Northern Ireland.

What does this mean for WhatDoTheyKnow? Comparing totals, the 28,282 requests made through WhatDoTheyKnow in 2017 represented 6% of all FOI requests made to local authorities. Similarly, there is a lot of variation across authorities some councils have around 2% of requests start on WhatDoTheyKnow, others have around 13%.  This shows that the overwhelming majority of requests to local authorities are made through other means.

Part two of this blog post discusses what we learned about the administration of FOI from this research. You can read the full report online, or download as a pdf.

This blog post is licenced under a Creative Commons Attribution Licence (https://creativecommons.org/licenses/by/3.0/)  — it can be re-posted and adapted without commercial restriction as long as the  original article/author is credited and by noting if the article has been edited from the original.

—
Image: Ula Kuźma

Our previous blog post about our new report, Freedom of Information in Local Government, discussed our findings about the volume of requests received by local government. This second post explores our findings about how FOI is administered, working from information received via FOI requests to all councils and an anonymous survey of FOI officers.

Staff responsible for the administration of FOI in local government tend to hold this as one responsibility among several. FOI teams are generally embedded in larger teams, with few staff solely working on FOI. As such, FOI administration rarely appears as a specific budget item.

While this makes the data patchy, from the information that is available, staffing levels (and hence budget) seem to be responsive to the number of FOIs received by a council. Every thousand additional FOI requests increases the number of staff dealing with FOI by 0.75 (95% confidence this is between 0.35 and 1.15).  Similarly, use of a case management system was associated with a greater number of requests — with the use of an organised system, and then use of specialist software being predicted by increases in the number of FOI requests received.

However, use of a case management system was not associated with any increase in the percentage of requests being replied to within the statutory limit (20 days), which suggests that differences in delays are caused elsewhere than the management of incoming FOI requests. Some requests are more complex in this respect than others, with FOI officers estimating that 38% of requests required responses from multiple departments or teams in the authority and 23% required ‘double handling’ — additional sign-off from senior or specialist staff.  The number of requests appealed to internal review was low (1.4%), but within these the success rate was quite high — between 36% and 49% were successful in changing some component of the original outcome.

Councils fairly universally keep records on the number of requests received, and time taken to reply — but have fewer records on the volume of information disclosed, or on the status of appeals.

The highest availability of knowledge were figures on numbers of FOI requests received. The two areas where almost all authorities had records was the number of FOI requests received (98% recorded these figures) and how many were completed inside the statutory deadline (92%). Records of internal review were held in 87% of cases and records of appeals to ICO in 86% of cases. The questions with the most missing information related to how much of a request had been delivered. 73% had records of the number that were completely granted; 70% had records of the number that were entirely withheld; and 65% had records of partially withheld/disclosed requests.

Most councils do not publish a disclosure log (a record of FOI requests received and their responses). Adding this factor into the model used to predict missing values for the number of FOI requests received found that there was no positive or negative effect of publishing a disclosure log on the number of FOIs received. In individual responses, while many FOI Officers expressed a desire to publish more (or steps taken towards that), there was also a strong skepticism of the value of doing this, and concerns that people do not check the log before submitting their requests, meaning logs do not reduce the volume of incoming requests. Several councils that had previously run disclosure logs had discontinued them due to low usage.

An upcoming blog post will talk about what we learned about using a front end interface to reduce FOI requests by searching the disclosure log. Sign up to our FOI newsletter to hear more when released.

Part one of this blog post discusses what we learned about the administration of FOI from this research. You can read the full report online, or download as a pdf.

This blog post is licenced under a Creative Commons Attribution Licence (https://creativecommons.org/licenses/by/3.0/)  — it can be re-posted and adapted without commercial restriction as long as the  original article/author is credited and by noting if the article has been edited from the original.

—

Image: Martin Adams

—
Image: Ula Kuźma

We’ve been working over the last few years to make our research as easy to read and explore as we can. However, because we release a lot of open data (and are usually open to sharing other data with researchers) there’s also been a lot of research written by researchers outside mySociety, which of course also forms part of the knowledge base about our services.  

As such, we’re expanding the scope of the research store to include work about mySociety’s services that has been produced by researchers beyond our own team.

Where papers have been released under a Creative Commons licence but there is only a PDF file available, we will sometimes create more accessible versions. For instance, we have already done so with Emily Shaw’s research into Civic Tech Cities and Frederik M Sjoberg, Jonathan Mellon, & Tiago Peixoto’s exploration of how receiving a response through FixMyStreet affects the probability of making future reports.

This isn’t yet a comprehensive collection, but we plan to add new research as it is published, and retrospectively add older research on a rolling basis. Sign up for our newsletter to hear when new research is added.

While we’re making things easier to find — we’ve also started including mySociety’s responses to calls for evidence and consultations on the research portal, and you can see those here.

With funding from the Consumer Data Research Centre (CDRC) we’ve been working with researchers from the University of Sheffield and University of Sterling to open up FixMyStreet data for researchers.

For an example of the kind of thing that can be done with this data, this group have produced maps for every local authority in the UK, mapping FixMyStreet reports against indices of deprivation (a few examples: Sheffield, Harrogate and Cardiff). These can be explored on our mini-site, where for each authority you can also download a printable poster with additional statistics.

If you’d like to know more about what these maps mean and what we learned from the process, there’s a report exploring what we learned here.