Today, mySociety, in partnership with Microsoft, launch Civic Tech Cities, a new piece of research looking at the technologies local governments implement to serve and communicate with their citizens. You can download it here.
Civic Tech: whose job is it?
Debating and making decisions on behalf of the people; managing services, disseminating information — all of these have been the agreed tasks of local government for a very long time. But has citizen-facing technology now also become a core function of government? And if so, how are they doing?
We often say that mySociety was originally set up to show governments how they could be using digital better, and that one day we hope to have done ourselves out of a job.
But perhaps it’s wrong to foresee a time when we’ll be able to pack up and go home. Perhaps those within government will never be able to escape internal bureaucracies and budget constraints to provide the software that their citizens will really benefit from; perhaps the provocative NGO, one step ahead with citizen-to-government technologies, will always be a necessary agent.
We won’t know for sure until we start researching beyond our own sphere.
A vital new area for research
When we set up the mySociety research programme, as you’d expect, our first priority was to look at the impact of the services we, and other organisations like us, were providing.
Around the same time, the term ‘Civic Tech’ was gaining traction, and it carried with it an implicit reference to applications made outside government, by organisations like us, cheekily providing the tools the citizens wanted rather than those the government decided they needed.
If our aim was to wake governments up to the possibilities of digital, to some extent it has been successful. Governments around the world, at all levels, have seen the financial and societal benefits, and are producing, buying in, and commissioning civic software for their own online offerings.
It is, then, high time that the sphere of government-implemented civic technologies were more closely examined: how effective are they? Who is using them? What changes are they wreaking on the relationship between citizen and government? How, indeed, are governments themselves changing as a result of this new direction?
Civic Tech Cities
Thanks to generous funding from Microsoft, we were able to conduct research that seeks to answer these questions, in the context of municipal-level council digital offerings in five US cities.
Emily Shaw, in collaboration with mySociety’s Head of Research Rebecca Rumbul, examined standalone projects in Austin, Chicago, Oakland, Washington DC and Seattle, to produce case studies that cast a light on the state of institutional civic tech in the current age.
The technologies chosen for scrutiny were diverse in some ways, but the challenges they faced were often alike: and we can all, whether inside or outside government, recognise common pitfalls such as failing to budget for ongoing maintenance of a service that was expected to roll happily along, untended, for the foreseeable future; or building a world-changing digital service that fails to gain traction because its potential users never get to hear about it.
It’s our hope that local governments everywhere will benefit from this in-depth look at the tools US municipal governments have put in place, from LargeLots in Chicago which sold disused land in disadvantaged neighbourhoods for a nominal $1 fee, to RecordTrac in Oakland, a request and response tool for those seeking information under California’s Public Record Act.
Better tools make better policy
Interestingly, one of the key findings of this report is that developing digital tools alongside policy, rather than bolting these tools on afterwards, results not only in better tools, but better policy too.
The user-centred design principles that have been central to the Civic Tech movement had a knock-on effect beyond the software development departments of municipal government. They began to shape the ways in which policy itself was developed, resulting in services that were more accessible and appropriate to the communities they serve.
Finally, it’s not just governments who will learn from this examination of best practices, potential problems and unexpected bonuses; we, and other NGOs like us, can gain crucial insights from the sector which, after all, is pursuing the same aim that we are.
You can read the research paper here. Many thanks to Microsoft for making it possible, and to Emily Shaw for putting in the time and effort to make it a reality.
Image: Jindong H
Researching in an unstable environment
It’s been nearly two years since the InfoLib Liberia project with iLab Liberia started. In that time the project has faced many hurdles, some predicted, and some completely unforeseen.
The iLab team have seen their country devastated by Ebola, only 11 years after the end of their second civil war, bringing tragedy and instability along with it. As you can probably imagine, the impact of curfews, fear and death in communities has made it difficult for people to continue with their daily lives. The social impact of such a disease is wide-reaching. Distrust, marginalisation and exclusion can be directed at those who show symptoms, or even who suffered and survived.
These are challenges that our local partners have had to contend with every day, both when holding training sessions and more crucially when researching the impact of the project on people’s lives.
However, by far the largest hurdle for this particular project has been a mixture of low internet penetration and lack of government will to release information. The team on the ground have been working tirelessly to create an ecosystem of requesting and training Public Information Officers (PIOs) to reply – even providing them with tablets to scan documents without needing electricity, let alone a computer. But if those officers have no access to the information that has been requested, their jobs become virtually impossible.
The project is now drawing to a close and we’re undertaking our final research survey. It seemed like a good time to take a look at what we’ve learnt about the impact of our joint Freedom of Information project in Liberia.
When designing the project we decided that impact could best be measured in terms of whether or not the project increased confidence in government transparency.
We carried out surveys in January 2016 and April 2016, to provide a baseline picture and then an assessment of impact at midline. The final survey is being conducted in August 2016 just as the project ends.
The first survey – the baseline – was carried out mainly in the rural areas. iLab Liberia teamed up with LFIC to survey 152 participants who had been involved in the FOI workshops that LFIC had held in the counties.
We had to attempt the second survey twice, as it turned out to be more challenging than we’d expected. We needed the participants from the first questionnaire to answer the same questions we’d asked them initially, in order to measure change — but it proved hard to locate all of them.
There were many factors which caused this, but the main one was economic drivers, forcing people to move to where the opportunities are. It’s a problem many researchers must run into working in the field.
Carter, the project lead at iLab Liberia told us:
“There are several reasons why this happens […]. People migrate a lot between markets, farms. Several persons who participated in the baseline could not be reached as they [had] travelled to other cities/counties. [Or] the job that allowed them to reside in that city/county is no longer available so they might have left seeking after another job.”
Our second attempt was more successful. We managed to contact a large percentage of the original participants in the survey: 112 of the 152.
We’ve found out some interesting things from doing this research. We saw that 74% of people who use the internet daily say it’s their main source of information, though it is still only a small percentage of the population who have access to the internet.
So the next biggest source of information? Radio! 85% of people with with no access to the internet give radio as their main source of information. Thinking of the migration of workers between cities and counties – you suddenly appreciate why Radio is such an important medium for getting hold of information. Thankfully, as you’ll remember from our original blog post, we’re covering both of these media in the InfoLib project.
In the months since we began studying the impact of this project we also learned that fear of making a request has dropped by 5% in the individuals surveyed . The amount of people who reported that they didn’t know how to ask for information dropped from 24% to 21%. This is pretty great news to us as it shows that our training and our encouragement is working – albeit slowly.
Finally we saw the percentage of people who believe government would be more transparent if citizens could see the information they hold rise by 3% to 93% of the surveyed respondents. Even if this figure hadn’t risen, this demonstrates a clear existing demand from the citizens of Liberia for the Government to release more information about its activities which is great news overall!
No project is without its challenges, and as you’ve seen above one of the big ones is ensuring that the same people respond from survey to survey. Not being able to pin down precisely the same set of people means that we can’t say with 100% certainty that we have a true measure in the difference in attitude.
As a result of the economic and social drivers mentioned above, the workforce in Liberia is very transient. This makes disseminating information through radio and internet mediums even more important. This research has shown that these are the primary sources of news and official information for the majority of Liberians, and continuing to improve knowledge about, and access to, information via these sources will empower the population further.
Finally, it can be challenging to demonstrate impact in projects like these, simply because research is not the main focus for our local partners. We partner with local groups because they are passionate, capable, and able to engage and mobilise citizens around a certain issue. We cannot expect small grassroots groups to have the resources or experience to conduct academic surveying, sampling or interviewing that could detect and definitively isolate the short term impact of a small project. This piece of research has provided some encouraging interim results, but most of all, it has provided valuable lessons to us at mySociety in trying to conduct this kind of impact research remotely and in partnership.
While we wait for the outcome of the final survey we can feel cautiously hopeful that this project has caused a small change in the way access to information operates in Liberia. infoLib will continue to run after the project officially ends, and mySociety will continue to support the work that iLab does in this area . However it may take longer than we had expected or hoped, to see the governmental shift towards releasing information.
TICTeC is our annual conference on the impacts of civic technologies.
This year, we’ll be in Barcelona, Spain, with a diverse programme of speakers from all over the world.
Guy Grossman, assistant professor at the University of Pennsylvania’s political science department, will deliver one of two keynotes. See details of our other keynote speaker, Helen Milner OBE, here.
Guy has had a long history in civic tech research, with a special focus on Uganda and Sub-Saharan Africa, and we are delighted that he will be sharing his insights to our audience of practitioners and researchers.
Hi Guy! Tell us what you’ll be covering in your TICTeC keynote.
The state of academic knowledge about the promises and pitfalls of ICT applications designed to increase voice and accountability in low-income countries. Specifically:
- Under what conditions are citizens more likely to communicate with government officials using mobile-platforms? If we “build them – will they come”?
- Do mobile platforms exacerbate inequalities in political access, privileging groups (men, urban, better-off) that are heavier users of ICTs?
- What can be done to help increase the participation rates of marginalised populations (such as women and the poor)?
Low-income countries have leapfrogged past the landline era and directly into the mobile era. Citizens can now connect with each other and with their public officials in unprecedented ways.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited.
The massive penetration of mobile technology, even to remote areas of the least developed countries, has great potential to reshape both the social and political landscapes.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited. My goal at TICTeC is to help take stock of what we know and don’t know in this emerging field.
What are you hoping to get out of TICTeC?
I’m hoping to interact with practitioners, policy makers, technologists and fellow academics to figure out where the overlap is between practical needs and academic research.
I’m also hoping to get exposed to promising and innovative applications that I am unaware of.
Your civic tech research focuses especially on Uganda and Sub-Saharan Africa. What led you down this path?
I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries.
I care deeply about poverty and social and gender inequality, so working and studying in Africa has been a natural choice.
How did I get involved with civic tech? In the past decade I have witnessed an increasing number of mobile-based applications designed to enhance such things as agriculture productivity and financial transactions or match between buyers and sellers.
On the other hand, I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries. I came to believe that this was a great void that needed to be filled.
In fact, unfortunately, we are still quite far from saturation in the development of governance-related ICT applications.
What do you think are the big unanswered questions when it comes to civic tech?
Does participation in mobile communication spill into more traditional forms of political participation?
Here’s a very partial list of some of the answered questions when it comes to civic tech in the global south:
Can simple innovations in mobile technology be used to facilitate new meaningful forms of political participation?
If so, what types of mobile-based political communication are most likely to be adopted by voters?
What is the potential of ICTs to flatten political access to marginalised populations?
What are some of the ways to increase the usage of mobile-based political communication in the face of clear collective action problems?
How can ICT be used to overcome political economy constraints to growth (e.g. by resolving monitoring challenges in government bureaucracies, reducing the cost for citizens of tracking bureaucratic performance, or making it easier to detect corrupt practices in public-sector salary distribution)?
And, does participation in mobile communication spill into more traditional forms of political participation?
If you could make one recommendation to those developing new civic tech, and wanting to see real impact from it, what would it be?
Adopt a user perspective — make sure that the suggested application/platform addresses a real need that could not be addressed using a low-tech solution.
You won’t want to miss what Guy has to say at TICTeC, so make sure you book your tickets now.
In my last post I described how we’re taking stock of where and how we’re delivering against our theory of change to give greater influence to citizens over those with power.
Since starting at mySociety I’ve spent my time meeting lots of lovely people, getting to know the team, our funders, partners and peers and finding out how mySociety does what it does.
One thing I have learned is that despite our British roots, the majority of our work is now international, and we work with wonderful partners in over 35 countries around the world, from Ukraine to South Africa, Liberia to Norway. In each case they tend to be activists, journalists and NGOs who are passionate about better government, citizen empowerment, and fighting corruption.
Our success is defined by our partners’ success – so in order to best support our partners I’d like to talk a little bit about some of the practical steps we’re taking to consolidate what we already have and scale up what works.
Four Simple Goals
The core mission of mySociety remains the same: to invent and popularise websites and apps that enable citizens around the world to exert power over institutions and decision makers.
We see the need to both ‘invent’ and ‘popularise’ digital tools as equally important – digital tools can be useful in developing new approaches to difficult problems, but we must ensure they are both widely used and actually enable citizens to be capable of demanding better.
In order to best help our partners and to better understand the impact of our work we have four really simple goals that will direct our efforts over the next few years:
1. Encourage more people
to use our websites and apps
in more countries
2. Work with more partners
to help them get better at
using digital tools
3. Prove what works and
feed those learnings back into
the wider community
4. Take a lead role in
making technology more useful
to civil society
Planning For Success
In addition to running our successful UK sites TheyWorkForYou.com, WhatDoTheyKnow.com, WriteToThem.com and FixMyStreet.com, we’ll continue to work with our partners to improve our existing services, making them easier to deploy and better integrated together.
We’ve recently established a quarterly call for new proposals for potential new partners who wish to set up new sites of their own from our roster of services (FOI, Parliaments and Elections, FixMyStreet). This helps inform potential partners of what’s involved before getting started, and helps us better target our resources and plan for success upfront.
We’re also putting more effort into increasing the impact and usage of our existing sites and services, by providing targeted development support, training, direct funding and additional technical development. Helping to sustain each site through the difficult first year or two should be a major marker of success.
Proving What Works
One major thing that will change is putting our research much more front and centre to our work, in order to create a greater evidence base for the impacts of civic technology and ensuring we are able to talk about this widely and publicly.
You’ll see us carry out much more inclusive and comparative impact research on the use of civic technology encompassing individual, socio-political and sector-specific factors.
If you haven’t already read our latest research paper ‘Who Benefits From Civic Technology?’ then please do have a look. This is an important first step in laying down the case for impact, being honest about where more work is required and focusing our efforts to create a greater evidence base for civic technology as a whole.
Our long-term aim is to establish a global hub for impact research, and assist more civic tech organisations to assess and improve the impact of their own work. To this end we’ll be hosting our next TICTeC – The Impacts of Civic Technology Conference, in Barcelona on the 27th and 28th of April next year.
This will be an important opportunity to share and discuss research findings and key challenges from across the sector and we hope to see many of you there in person.
Where We Go Next
Over the past decade, through a process of experimentation, consultation and measurement, mySociety have created a portfolio of popular, proven online services, used by over 10 million people each year.
This is an amazing legacy to take on.
Over the next decade I hope that we’ll continue this work, and seek to further establish mySociety as one of the leading international civic technology institutions, providing much-needed global leadership and inspiration in our sector – if we could come to be seen as having a similar impact to that of an Article 19 or Human Rights Watch in our own field, then I think that will be a pretty good measure of success.
For the moment we’ll continue to focus on the practical steps we need to take in order to improve and build upon what we already have, but I’m excited about the plans we have for the future and I’ll share more details on what we have in store in the weeks to come.
How do you pin down the intangible?
More specifically, how do we understand something as nebulous as why people visit our websites? For better or worse, Google Analytics can’t provide brain scans to show our users’ motivations, so the only solution is to ask them.
Between May and August of this year, if you visited TheyWorkForYou, you might have seen a pop-up inviting you to answer a few questions. Some users also gained new site features. It was all part of a concerted drive to understand more about why you use the site, and part of our wider research programme funded by The William and Flora Hewlett Foundation.
Many thanks if you were one of those who responded. In all, the survey generated a couple of a million data points, which has certainly given our resident data-cruncher, Nick, plenty to wade through.
Why do people use TheyWorkForYou?
We know why we run TheyWorkForYou: it aims to make Parliament—often perceived as ‘not for the likes of me’—more accessible for everyone. But what we don’t know is whether or not we are achieving that aim.
You see, with our other sites, WhatDoTheyKnow, FixMyStreet or WriteToThem, there is a set path that indicates a successful visit. The user lodges an FOI request, reports a problem, or writes to their representatives.
TheyWorkForYou, on the other hand, does not promote an action. There are actions you can take, like signing up for email alerts, or adding an annotation to a debate—but these are secondary to the site’s primary function, which is simply to present information. There’s no measurable metric that can tell us how informed people are after they’ve read one of our pages, nor what they do with that information. Hence the survey.
People hate pop-ups
One of our first findings came as no surprise: yes, we hate pop-ups too*! And yet, we wanted to keep users on the site while they answered our questions, so that’s the interface we used.
75.7% of people dismissed the pop-up when it appeared. We’d probably have done the same. We’d like to meet the remaining 24.3% of people and shake their hands.
The pop-up asked users what they would go on to do with the information they’d found on TheyWorkForYou. And most people gave the same answer they would if a shop assistant had asked: in fact, 75.2% said they were ‘just browsing’, with no further plans.
That makes us really curious as to what provoked that browsing: are they regular visitors, or were they sent to TheyWorkForYou via a tweet or Facebook reminder? Maybe one of our email alerts was the trigger. Google Analytics will allow us to see which site referred visitors to us, but of course we can’t match that data with individual motivations.
A further 10.9% of users had come with the intention of contacting their MP: TheyWorkForYou’s MP pages feature a prominent button inviting you to do that, and clicking on it takes you to our site WriteToThem.
6.6% of people were using the site for work purposes. Only 3.1% of people had plans to share content via social media, a statistic we’d like to increase if we can.
During the research period, some users (not necessarily those who had seen the pop-up) saw an addition to person pages and debate pages: a “where next” button, which led to some suggestions on how to use the information they’d found on the site.
This button was not widely used: just 0.1% of people clicked it. As so often with research findings, this result leads to more questions: did people already know what they wanted to do? Did they not want to do anything? Were they, perhaps, annoyed at the suggestion they might need guidance?
Or was the button just poorly positioned? We may run some further testing to find out if a more prominent placing leads to more clicks.
Of those few people who did click through for advice, the most popular options were to use WriteToThem to contact their MP about what they’d just read, or to contact the Parliamentary Ombudsman—it seems that on the few occasions when people do welcome a next step, they do want to take action.
Follow the news
Meanwhile, we were also taking a concentrated look at our site analytics. The test period coincided with the Labour party leadership contest and David Cameron’s cabinet announcements, and we saw very clearly how current affairs have an effect on which pages people visit.
John Whittingdale (the new Culture Secretary)’s voting record and Jeremy Corbyn’s profile page were far and away the most visited pages by several thousand (they’re responsible for the spike on the left of the graph that you see below). Also popular were the profile pages of the other Labour candidates, and Corbyn’s voting record.
But the other pages visited were very diverse: an extreme example of the long tail effect.
We reckon it takes more than a few seconds to take in a parliamentary debate, so one of the things we measure on Google Analytics is how many people stick around for more than 7 seconds, and again, for more than ten seconds.
Actually, we get pretty good results for the 7-second cut-off. Recent work we did on the design of the site’s debate pages had an effect that was a real cause for celebration: a massive 46% drop in the bounce rate (ie people who leave the site after viewing just one page)
But only 38% then stick around long enough for the second trigger, which doesn’t indicate as much involvement as we might have hoped for.
What have we learned?
At the end of this experiment, we reckon we’ve learned four main things:
- We need to carry on thinking about how to encourage more engagement with TheyWorkForYou’s content
- Current affairs are a huge driver to the site, and we can build on this via social media—and especially by encouraging our users to share the content they’ve found
- We need to conduct more experiments to see whether people genuinely don’t want advice about how to use the information they’ve found, or whether they would take it if it was more prominently flagged up
- There’s also a motivation to dig in deeper to motivations: what made people come to the site, if they are ‘just browsing’? What exactly are they doing ‘for work’? And if they want to contact their MP, what was the trigger for that?
If you have thoughts about any of these, as regards your own usage of TheyWorkForYou, please do feel free to comment below. And if you’d like to find out more about our research programme there’s plenty of information here, along with news of our annual research conference, TICTeC.
*And we know who to blame.
Image: The U.S. Printing Co., Russell-Morgan Print, Cincinnati & New York [Public domain] via Wikimedia Commons.
This post refers to the statistics we published for 2014. If you would like to see the latest responsiveness figures on WriteToThem, please visit www.writetothem.com/stats/.
Of course, there are many factors that you’ll consider before you cast your vote in the general election. But we think that one important quality in an MP is that they respond to their constituents.
So you may wish to check your own MP’s performance on the latest WriteToThem responsiveness league table. Just put in your postcode and you can see how they did in 2014.
Where the data comes from
When you send a message to your MP using our site WriteToThem, you’ll receive an automated email two weeks later, asking whether or not you received a response. Every year, we take the data from these surveys and use it to assemble our responsiveness rankings.
You might think that MPs would be doing the best they can this year, in the run-up to the election. Sadly, that doesn’t seem to be the case: overall, responsiveness has fallen a percentage point since last year, with 46% of emails receiving no reply.
You can find all our data and methodology on the league table page.
We know that messages sent to WriteToThem may not reflect all messages sent to an MP; we also know that not every message will require an answer. However, we think that, taken overall, our sample size of over 36,000 interactions can be seen as indicative.
We’re expecting 109 people from 26 different countries and 69 different organisations – all with a common interest in discussing and understanding more about the impact of civic tech.
You can see the full agenda here, and don’t worry if you didn’t manage to get a ticket: we’ll be documenting everything in full.
- For the as-it-happens picture, keep an eye on the Lanyrd page throughout tomorrow.
- We’ll be following up with summaries, podcasts, photos and videos right here on the mySociety blog.
- Be sure to tag your social media with #tictec and we’ll also document the best of that.
See you tomorrow!
Hurry: today’s the last day to book your place at TICTeC, our conference on the Impacts of Civic Technology, if you want to take advantage of the early bird pricing.
You have until midnight tonight to save yourself £100 on your ticket price. Here’s where to book.
We’re still firming up the final schedule and session titles, but let us whet your appetite by listing some of the speakers.
Here are some of the other speakers who’ll be helping to shape the agenda at TICTeC:
We’re really delighted to be presenting such a diverse group of speakers bringing insights from so many parts of the world… and we can hardly wait to hear what they all have to share.
If you feel the same, well, now’s the time to book your ticket.
In a recent blog post, we summarised the research we commissioned from the University of Manchester’s Rachel Gibson, Marta Cantijoch and Silvia Galandini, on whether or not our core UK websites have an impact.
The full research paper is now available, and you can download it here .
Professor Rachel Gibson says: “This research presents a unique and valuable insight into the users of online resources such as FixMyStreet and WhatDoTheyKnow.
“Through applying a highly original methodology that combines quantitative and in-depth qualitative data about people’s experience of mySociety sites over time, we provide a picture of how eDemocracy tools are contributing to activism at the local level.
“We thank all those that contributed to this important study and mySociety for their co-operation in developing this highly rewarding and academically rigorous project.”
Our thanks to Rachel, Marta and Silvia for conducting this research, which utilised methods not previously used in the civic tech field. We hope that it will prove a useful foundation to our own further research, and that of others.
We at mySociety build and popularise digital tools worldwide that help citizens exert power over institutions and decision-makers. Or do we?
Wanting to know whether our well-meaning civic tech is actually making a difference, mySociety recently created the post of Head of Research. My name is Dr Rebecca Rumbul, and I have now been installed in that role for about 6 weeks. I want to know if civic tech like ours is having an impact on citizens and governments, and how such sites operate and negotiate issues not just in the UK, but in the 50 or so countries that we know have digital democracy websites operating in them.
There is enormous scope for interesting and important research to be conducted using sites such as the ones that mySociety and our partners operate. The digital nature of our focus means that we can collect large volumes of data online at a low cost.
That said, there is nothing quite like making connections on the ground or meeting people face to face. mySociety is a small NGO, and does not have the capacity to conduct all of the research activities it would like on its own.
Therefore, we are actively seeking to work with academic partners on both qualitative and quantitative research focusing on the impact of civic tech.
We are planning to conduct research in the following countries. If you are an academic based in one of these countries and interested in our research agenda, please get in touch. We will be very happy to hear from you. Contact me via email@example.com
- South Africa
We conduct and disseminate research regularly. If you would like to hear more about our activities and events, sign up for our newsletter.
Image credit: Into the Unknown by Gary Gao (angrytoast), CC BY-NC 2.0