This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here.
A common feature in websites and services where users generate data is that a small amount of users are responsible for a large percent of the activity. For instance, 77% of Wikipedia is written by 1% of editors (with most of that being done by an even smaller fraction) and for OpenStreetMap 0.01% of users contribute a majority of the information.
This also applies to plenty of offline activities — for instance, half of the 25,000 noise complaints about Heathrow Airport were made by 10 people. People who dedicate significant time to an activity can quickly outpace a much larger group who only use the service once.
For FixMyStreet (where people report issues like littering and potholes to local authorities), the top 0.1% of users made 16% of the reports and 10% of users account for 62% of reports. Starting from the most prolific users, increasing the number of users by a factor of 10 roughly doubles the number of reports:
- 418 users (0.1%) account for 224,775 reports (16%)
- 4,181 users (1%) account for 470,384 reports (33%)
- 41,814 users (10%) account for 881,481 reports (62%)
This reflects that at any scale in the data, around half the activity is happening in the top 10%. Overall, two-thirds of users made only one report — but the reports made by this large set of users only makes up 20% of the total number of reports.
This means that different questions can lead you to very different conclusions about the service. If you’re interested in the people who are using FixMyStreet, that two-thirds is where most of the action is. If you’re interested in the outcomes of the service, this is mostly due to a much smaller group of people.
Reka Solymosi (2018) investigated the behaviour of the top 1% of reporters and found that they tended to report a wide range of categories: only “16 of the 415 contributors reported only one type of issue. The other 399 reported issues in more than one category” with an average of six categories. These also tended to cover a wide area and “there were only six people who reported in only one neighborhood [LSOA], fewer than the number of people who reported in only one category. The other 409 contributors all reported in at least two neighborhoods”. Solymosi finds four clusters of these super-contributors:
- Traditional guardians – these report in a small number of neighbourhoods covered but represent the largest number of users.
- Large-neighbourhood guardians – Report in a larger number of connected neighbourhoods.
- Super-neighbourhood guardians – People who report in a high number of connected neighbourhoods; this is the largest group.
- Neighbourhood agnostic guardians – reports are made in disconnected areas.
Collectively, this can have a wide impact — 18% of LSOAs in England have at least one report from a user who has made more than 100 reports (which is only around 900 people).
Looking at the general picture through the Explorer minisite, it’s not just that serial reporters report widely; certain kinds of reports are more likely to be made by users who are reporting more issues:
Incivilities, rubbish, road safety and bus stop damage are all categories more likely to be reported by users who have made 50+ reports. While users who make lots of reports tend to make reports across a few categories, they are often specialised in their output.
59% reports of flyposting, 57% of graffiti, 52% of litter problems are made by users who have reported more than 50 times.
It’s important to remember that these aren’t hard divides. Single report users are less likely to report potholes than serial reporters, but it is also true that one in five people who only report one issue report a pothole.
For the bundle model of understanding FixMyStreet, thinking about this group of super contributors is important, because they represent a minority of users, yet generate most of the value and impact of the site.
But this comes with a cost. People living in the same area as super contributors benefit from their efforts – but where these super contributors have different concerns or priorities from the area as a while this might shift the outcomes of the service.
As Muki Haklay argues:
The specific background and interests of high contributors will, by necessity, impact on the type of data that is recorded. This is especially important in VGI [volunteered geographic information] projects where the details of what to record are left to the participants.
Where resources are allocated on the basis of data generated by a service, the behaviour of this small group can have an outsized effect. Future blog posts in this series will explore what this looks like in practice.
Research Mailing List
Sign up to our mailing list to hear about future research.
This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here.
A key question when looking at the role of the internet in civic life is whether it changes the demographics of who participates; or whether it simply changes the methods by which already engaged citizens participate. The two sides in this argument can be described as mobilisation and reinforcement.
The mobilisation argument says that the internet reduces the cost of communication and action, which means that more people can be involved and access becomes more broad.
The reinforcement argument says that the reduced costs of connectivity will mostly reinforce existing participation divides, making it cheaper for people already engaged to participate, but not necessarily reaching disengaged people.
This is a fundamental question for civic tech: how are these online tools used? Are they mobilising everyone or just providing more efficient processes for people who are already engaged?
This is explored in mySociety’s 2015 report Who benefits from Civic Technology?, and is a recurring question in much of our research since, such as our work on FixMyStreet, and digital technologies in sub-Saharan Africa.
Two themes we are currently investigating in this area are proxies and bundles.
Proxies are where services are used by intermediaries, on behalf of — and bringing benefits to — others: for instance, where charities engage in more effective lobbying as a result of free access to TheyWorkForYou, or where case workers find it easier to identify and write to a client’s local councillor using WriteToThem.
Bundles are about exploring how different groups of users use a service in different ways, to such an extent that one service can in fact be understood as a bundle of services serving different kinds of users.
This is the first in a series of blog posts investigating bundles.
A common finding across mySociety services is that most people only use “transactional” services (like WhatDoTheyKnow, FixMyStreet or WriteToThem) once, to do one thing. Repeat users make up a minority of users (even if they account for the majority of actual usage).
From a technology point of view or an organisational point of view, it makes sense to understand that there is a website called FixMyStreet.com run by mySociety. But from the point of view of the majority of users, it makes sense to think of a website like FixMyStreet as dozens of different services, most of which they will never use. For one user, FixMyStreet is a tool for reporting potholes, for another it is for reporting littering. Similarly, WriteToThem is most often used as a tool to write to MPs — but the profile of people who use it to write to their local councillors is very different.
Some services in a bundle are used by a different demographic to other uses of the same website. Understanding how to encourage FixMyStreet use in underrepresented groups requires an understanding of how there are already differences in usage across all the “services” in the FixMyStreet bundle.
To get more information about these different uses of a website, we’ve built a mini-site that helps to explore basic demographic information about each use type. Starting with FixMyStreet, personal information (names) have been anonymised and converted to gender (approximately), while coordinates are grouped into Lower Super Output Areas (LSOA) — geographic areas commonly used for statistical purposes. This means that we can look at a general, anonymised set of data representing people making FixMyStreet reports, and match this grouped data against various measures of deprivation.
Understanding more about these different patterns of users suggests possible ways a service can be used and helps sharpen new research questions.
When examining uses of one element of a bundle, the key question is whether the pattern observed reflects just the individual, or the overall pattern of the bundle. To answer this, a chi-square test is used to tell if the distribution of a sub-use of the site is different to a statistically significant extent to all other uses of the site (this method was inspired by an analysis of gender of reporters in Reka Solymosi, Kate Bowers and Taku Fujiyama’s 2018 paper on FixMyStreet). The groupings of categories in FixMyStreet use Elvis Nyanzu’s meta categories. The mini-site highlights in red and green areas where a distribution differs from how patterns on the site as a whole respond.
We’ll be writing a number of blog posts over the next few months covering things we’ve learned from the mini-site. The first two are already up (and linked below):
Research Mailing List
Sign up to our mailing list to hear about future research.
mySociety’s latest research looks into the impacts of Legal Information Institutes in sub-Saharan Africa. You can read the full paper here.
If you wanted to find out what a specific law covered, how would you do it? Google what you thought the law was and hope that it came up in an internet search? Go to the local public library and look for law books? Ask a friend? In many cases around the world, especially in developing countries, it is almost impossible to get access to the law through these channels. Many sub-Saharan African countries do not routinely publish their legislation online, and fewer still publish the case-law judgments made by the courts.
So where is this information? Does it exist?
The answer is, that it does mostly exist, but much of it is either held in hard copy format within expensive and rare legal textbooks (the kind that can only be found in shiny law offices or prestigious university libraries), or, it is held behind an electronic paywall by a private, profit-making organisation, which requires often eye-watering subscription fees to access.
How, then, can individuals working in the legal field without significant financial backing, access and use the law? Online Legal Information Institutes are the primary answer.
In more than 60 countries around the world, Legal Information Institutes make significant volumes of legal information — legislation, case law, judgements etc — freely available on the internet. They can provide valuable resources to legal students, practitioners and stakeholders.
Despite the fact that a substantial movement exists promoting the principles of free access to law, these services have received relatively little charitable or philanthropic funding — particularly when compared to services that provide information relating to political or fiscal transparency. Is it the case that LIIs are primarily used by comparatively well-paid professionals, and hence deliver little true, positive, impact? Or do LIIs in developing countries, where domestic case law and legislation is already difficult to access, and where social mobility within the professions remains low, perform a greater societal service?
As a financial supporter of a number of African LIIs since 2013, the Indigo Trust, a UK-based philanthropic foundation, commissioned a report to examine the impacts of the LIIs, and whether those impacts could reasonably be amplified with greater investment.
The research identified clear, positive impacts resulting from the existence and use of the LIIs, most notably in South Africa, where the LII proved to be a key tool in increasing access to the legal profession for economically disadvantaged groups. Across the countries studied, the LIIs were also benefiting the development of high quality domestic case law, which had been underdeveloped prior to digitisation; and were considered to be useful tools for citizens in developing a more meaningful understanding of the law.
The publication of this research serves to demonstrate this positive work, and the further development work that can strengthen the LIIs.
Image: Gemma Moulder
As part of our work investigating the digital side of Citizens’ Assemblies (see our previous report), mySociety have started writing a guide on what the website for a Citizens’ Assembly should look like.
A dedicated website can be important before, during and after the event. It can help you to recruit, inform and communicate during the whole process, from planning to sharing of results. But beyond that, it helps ensure you meet two of the most crucial standards suggested in Marcin Gerwin’s well-regarded list for Citizens’ Assemblies: Visibility and Transparency.
It can also help with the further standards of Impact: making clear from the outset what will result from the outcome of the Assembly; and Openness: providing a forum where everyone can contribute to the process.
In this guide we discuss broad design and editorial principles, as well as information that should be included. While we include examples of what we consider good practice from previous Assembly websites, this is very much a first attempt at consolidating good practice rather than a definitive document.
Image: Markus Spiske
mySociety’s Head of Research Dr Rebecca Rumbul will be speaking at the first ever Welsh Citizens’ Assembly next week. She’ll be exploring how citizens might more easily feed into the questions posed to ministers and the First Minister in the National Assembly for Wales.
Questions are a fundamental part of all of the UK’s parliaments, most famously in the form of PMQs, the half hour every Wednesday when MPs can raise any issue they deem important with the Prime Minister.
In the devolved parliaments there are also various formats for Q&As, both written and oral. But, Rebecca will argue, there are fundamental problems inherent in all of them, from a lack of representation of the views of the general public, to the political motivations that lead to many questions lacking meaningful substance.
Of course, a Citizens’ Assembly is most concerned with hearing from the general populace, and Rebecca will go on to present our recent research into the digital tools that can help with that process, while examining the pros and cons of each.
Rebecca is one of several speakers who will also include Dr Diana Stirbu and Professor Graham Smith. The event is being co-facilitated by Involve and you can keep up to date with the Citizens’ Assembly’s activities on their dedicated website.
Every quarter the Cabinet Office releases Freedom of Information statistics for a collection of central government ministries, departments and agencies. This provides a benchmark for understanding how requests made from WhatDoTheyKnow relate to FOI requests made through other methods. From 2017, mySociety started retrospectively tracking the proportion of FOI requests sent via WhatDoTheyKnow to central government using a minisite — https://research.mysociety.org/sites/foi/ — that explores the data.
This report explores what had and hadn’t changed in the last few years, as well as the number of requests made through WhatDoTheyKnow Pro — a new service being piloted that allows embargos of the results of FOI requests for a period — with the goal of bringing more people making FOI requests professionally (such as journalists) into the system and leading to more raw results being made available after the conclusion of a project.
We found that:
- WhatDoTheyKnow accounted for between 15-17% of audited bodies and between 18-21% of ministerial departament FOI requests.
- While the proportion of requests have grown most years since 2010, there was no real change from 2017 to 2018.
- Requests made to central government via WhatDoTheyKnow only make-up around 9-10% of all requests sent via WhatDoTheyKnow in 2018.
- WhatDoTheyKnow Pro requests made up 1% of FOI requests to central government — but most requests using this service went to other areas of the public sector.
A study published in 2017 by Reka Solymosi, Kate J Bowers, and Taku Fujiyama used publicly available data for FixMyStreet to investigate (among other things) whether men and women reported different things using the site, and found a gender divide relating how people were moving around when they found the problem:
[M]en are more likely to report in categories related to driving (potholes and road problems), whereas women report more in categories related to walking (parks, dead animals, dog fouling, litter.(p. 954).
A potential limitation of this study was that it could only use reports that weren’t publicly anonymous, as the reporter’s name was used to approximate gender. If there was a gender skew in terms of which users were more likely to report anonymously, this might mistakenly pick up differences in anonymisation as a gender divide (for instance, if a lot of women were reporting potholes, but were more likely to do so anonymously).
To investigate this, we internally replicated the study on both anonymous and non-anonymous reports. This found that there was a gender skew related to anonymisation, with women being 10% more likely to report anonymously and that some types of report were more likely to be reported anonymously than others.
However, despite this factor, the original study’s conclusion was validated by this analysis. The categories highlighted are differently gendered when including the non-anonymous data, with men reporting far more problems with road surfaces and women reporting more litter related issues.
Future blog posts will further explore reasons and implications of this divide. The replication can be read online or downloaded as a PDF.
As part of the recent work we’ve been doing around meaningful citizen participation in democratic decision making, mySociety have been investigating how digital tools can be used as part of the process of a Citizens’ Assembly.
We reviewed how Citizens’ Assemblies to date have used digital technology, and explored where lessons can be learned from other deliberative or consultative activities.
While there is no unified digital service for Citizens’ Assemblies, there are a number of different, individual tools that can be used to enhance the process — and most of these are generic and well-tested products and services. We also tried to identify where innovative tools could be put to new uses, while always bearing in mind the core importance of the in-person deliberative nature of assemblies.
We found that digital tools have potential uses in many parts of the process, which we grouped in three areas:
Preparation: bringing the public in
- Question forming
- Public submissions
- Finding experts and stakeholders to give evidence
Internal: facilitating assemblies
- Attendance management
- Tools for coming to decisions in the assembly (voting)
- Sharing assembly materials to members
- Including a wider range of experts
- Enabling online deliberation for assembly members outside the face-to-face sessions
External: sharing products
- Sharing the conclusions of the assembly
- Streaming of evidence/plenary sessions
- Sharing evidence submitted to inquiry
- Tracking implementation of recommendations
- Communicating participants’ experiences
- Allowing feedback from non-participants on the outcome
Above all when considering the use of digital tools, it’s important that the final choice is appropriate to the aims of the project — and will typically be complementary rather than taking a centre-stage role. Digital tools can reduce costs and enhance the process by creating resources that add greater depth and knowledge to the process, but shouldn’t detract focus from the importance of the core deliberative activity of the assembly.
On 26 – 27 of June, scholars and practitioners from all over the world will be meeting in Rio de Janeiro for the 6th Global Conference on Transparency Research. The conference focuses on measuring transparency, exploring how this can be achieved, what the barriers are, whether metrics are useful, and how current interventions are shaping transparency around the world.
mySociety’s Head of Research Rebecca Rumbul will be attending, and will be presenting some of mySociety’s recent research into the transparency of parliamentary information in sub-Saharan Africa. Examining transparency through a digital lens, this research broke new ground in understanding how digital tools are shaping parliamentary transparency in sub-Saharan Africa, and how barriers to transparency are affecting how citizens engage with public institutions. You can read the full report here.
Rebecca will be speaking at 4pm on Thursday 27 June, so please do come along and say hello. She says, “Transparency, digital and citizen engagement are core themes of our research at mySociety, and we love to talk to other people working in these areas. Meeting new people and sharing ideas are the best parts of any conference, so do grab me for a chat if you are attending.”
If you are unable to join Rebecca in Rio, but you are interested in talking research, we’re always happy to receive email. And keep your eyes peeled for our TICTeC conference announcement for April 2020. We will be opening our Call for Papers in early September.
Image: Jaime Spaniol
Following on from the success of our first TICTeC Local conference in Manchester last year, we’re delighted to be hosting the second TICTeC Local on 1st November 2019.
Thanks to support from London’s Chief Digital Officer, Theo Blackwell, we’re thrilled to be hosting TICTeC Local 2019 at London City Hall, home to the Mayor of London and the London Assembly.
TICTeC Local is part of mySociety’s global The Impacts of Civic Technology Conference (TICTeC) series, which has been examining Civic Tech at the local, national and global levels since 2015. TICTeC Local narrows the lens, focusing on where and how Civic Tech connects with and impacts local communities and local government.
What are the digital innovations that are helping local communities and local government to foster citizen engagement, drive efficiency, and combat social and environmental problems? Join us at City Hall this November to find out.
Call for Papers now open
If you’d like to give a presentation or run a workshop at TICTeC Local 2019, please submit your proposals now. You have until Friday 6th September 2019.
We encourage presentation submissions to focus on specific digital innovations that are helping local communities and/or public authorities to foster citizen engagement/participation, drive efficiency, and combat social and environmental problems.
We welcome examples from across the UK and from around the world, but the focus must be the local level.
We will prioritise submissions that focus on the specific impacts of these digital innovations, rather than those that simply showcase new tools that are as yet untested.
Workshop proposals should be relevant to the conference theme. Submit your proposal now.
Registration is now open. Tickets are free for the public sector. Early bird tickets provide a significant discount, so it’s well worth registering before early bird ticket sales end on 13th September 2019.
mySociety is a charity and relies on sponsorship and ticket sales to make events like TICTeC Local happen. If you’re interested in hearing about sponsorship opportunities at TICTeC Local or our global TICTeC events then please get in touch to discuss further.
We look forward to seeing you in London in November! Meanwhile, if you’d like to see what TICTeC Local is all about, you can browse all the resources from last year’s conference.