TICTeC is our annual conference on the impacts of civic technologies.
This year, we’ll be in Barcelona, Spain, with a diverse programme of speakers from all over the world.
Guy Grossman, assistant professor at the University of Pennsylvania’s political science department, will deliver one of two keynotes. See details of our other keynote speaker, Helen Milner OBE, here.
Guy has had a long history in civic tech research, with a special focus on Uganda and Sub-Saharan Africa, and we are delighted that he will be sharing his insights to our audience of practitioners and researchers.
Hi Guy! Tell us what you’ll be covering in your TICTeC keynote.
The state of academic knowledge about the promises and pitfalls of ICT applications designed to increase voice and accountability in low-income countries. Specifically:
- Under what conditions are citizens more likely to communicate with government officials using mobile-platforms? If we “build them – will they come”?
- Do mobile platforms exacerbate inequalities in political access, privileging groups (men, urban, better-off) that are heavier users of ICTs?
- What can be done to help increase the participation rates of marginalised populations (such as women and the poor)?
Low-income countries have leapfrogged past the landline era and directly into the mobile era. Citizens can now connect with each other and with their public officials in unprecedented ways.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited.
The massive penetration of mobile technology, even to remote areas of the least developed countries, has great potential to reshape both the social and political landscapes.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited. My goal at TICTeC is to help take stock of what we know and don’t know in this emerging field.
What are you hoping to get out of TICTeC?
I’m hoping to interact with practitioners, policy makers, technologists and fellow academics to figure out where the overlap is between practical needs and academic research.
I’m also hoping to get exposed to promising and innovative applications that I am unaware of.
Your civic tech research focuses especially on Uganda and Sub-Saharan Africa. What led you down this path?
I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries.
I care deeply about poverty and social and gender inequality, so working and studying in Africa has been a natural choice.
How did I get involved with civic tech? In the past decade I have witnessed an increasing number of mobile-based applications designed to enhance such things as agriculture productivity and financial transactions or match between buyers and sellers.
On the other hand, I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries. I came to believe that this was a great void that needed to be filled.
In fact, unfortunately, we are still quite far from saturation in the development of governance-related ICT applications.
What do you think are the big unanswered questions when it comes to civic tech?
Does participation in mobile communication spill into more traditional forms of political participation?
Here’s a very partial list of some of the answered questions when it comes to civic tech in the global south:
Can simple innovations in mobile technology be used to facilitate new meaningful forms of political participation?
If so, what types of mobile-based political communication are most likely to be adopted by voters?
What is the potential of ICTs to flatten political access to marginalised populations?
What are some of the ways to increase the usage of mobile-based political communication in the face of clear collective action problems?
How can ICT be used to overcome political economy constraints to growth (e.g. by resolving monitoring challenges in government bureaucracies, reducing the cost for citizens of tracking bureaucratic performance, or making it easier to detect corrupt practices in public-sector salary distribution)?
And, does participation in mobile communication spill into more traditional forms of political participation?
If you could make one recommendation to those developing new civic tech, and wanting to see real impact from it, what would it be?
Adopt a user perspective — make sure that the suggested application/platform addresses a real need that could not be addressed using a low-tech solution.
You won’t want to miss what Guy has to say at TICTeC, so make sure you book your tickets now.
In my last post I described how we’re taking stock of where and how we’re delivering against our theory of change to give greater influence to citizens over those with power.
Since starting at mySociety I’ve spent my time meeting lots of lovely people, getting to know the team, our funders, partners and peers and finding out how mySociety does what it does.
One thing I have learned is that despite our British roots, the majority of our work is now international, and we work with wonderful partners in over 35 countries around the world, from Ukraine to South Africa, Liberia to Norway. In each case they tend to be activists, journalists and NGOs who are passionate about better government, citizen empowerment, and fighting corruption.
Our success is defined by our partners’ success – so in order to best support our partners I’d like to talk a little bit about some of the practical steps we’re taking to consolidate what we already have and scale up what works.
Four Simple Goals
The core mission of mySociety remains the same: to invent and popularise websites and apps that enable citizens around the world to exert power over institutions and decision makers.
We see the need to both ‘invent’ and ‘popularise’ digital tools as equally important – digital tools can be useful in developing new approaches to difficult problems, but we must ensure they are both widely used and actually enable citizens to be capable of demanding better.
In order to best help our partners and to better understand the impact of our work we have four really simple goals that will direct our efforts over the next few years:
1. Encourage more people
to use our websites and apps
in more countries
2. Work with more partners
to help them get better at
using digital tools
3. Prove what works and
feed those learnings back into
the wider community
4. Take a lead role in
making technology more useful
to civil society
Planning For Success
In addition to running our successful UK sites TheyWorkForYou.com, WhatDoTheyKnow.com, WriteToThem.com and FixMyStreet.com, we’ll continue to work with our partners to improve our existing services, making them easier to deploy and better integrated together.
We’ve recently established a quarterly call for new proposals for potential new partners who wish to set up new sites of their own from our roster of services (FOI, Parliaments and Elections, FixMyStreet). This helps inform potential partners of what’s involved before getting started, and helps us better target our resources and plan for success upfront.
We’re also putting more effort into increasing the impact and usage of our existing sites and services, by providing targeted development support, training, direct funding and additional technical development. Helping to sustain each site through the difficult first year or two should be a major marker of success.
Proving What Works
One major thing that will change is putting our research much more front and centre to our work, in order to create a greater evidence base for the impacts of civic technology and ensuring we are able to talk about this widely and publicly.
You’ll see us carry out much more inclusive and comparative impact research on the use of civic technology encompassing individual, socio-political and sector-specific factors.
If you haven’t already read our latest research paper ‘Who Benefits From Civic Technology?’ then please do have a look. This is an important first step in laying down the case for impact, being honest about where more work is required and focusing our efforts to create a greater evidence base for civic technology as a whole.
Our long-term aim is to establish a global hub for impact research, and assist more civic tech organisations to assess and improve the impact of their own work. To this end we’ll be hosting our next TICTeC – The Impacts of Civic Technology Conference, in Barcelona on the 27th and 28th of April next year.
This will be an important opportunity to share and discuss research findings and key challenges from across the sector and we hope to see many of you there in person.
Where We Go Next
Over the past decade, through a process of experimentation, consultation and measurement, mySociety have created a portfolio of popular, proven online services, used by over 10 million people each year.
This is an amazing legacy to take on.
Over the next decade I hope that we’ll continue this work, and seek to further establish mySociety as one of the leading international civic technology institutions, providing much-needed global leadership and inspiration in our sector – if we could come to be seen as having a similar impact to that of an Article 19 or Human Rights Watch in our own field, then I think that will be a pretty good measure of success.
For the moment we’ll continue to focus on the practical steps we need to take in order to improve and build upon what we already have, but I’m excited about the plans we have for the future and I’ll share more details on what we have in store in the weeks to come.
How do you pin down the intangible?
More specifically, how do we understand something as nebulous as why people visit our websites? For better or worse, Google Analytics can’t provide brain scans to show our users’ motivations, so the only solution is to ask them.
Between May and August of this year, if you visited TheyWorkForYou, you might have seen a pop-up inviting you to answer a few questions. Some users also gained new site features. It was all part of a concerted drive to understand more about why you use the site, and part of our wider research programme funded by The William and Flora Hewlett Foundation.
Many thanks if you were one of those who responded. In all, the survey generated a couple of a million data points, which has certainly given our resident data-cruncher, Nick, plenty to wade through.
Why do people use TheyWorkForYou?
We know why we run TheyWorkForYou: it aims to make Parliament—often perceived as ‘not for the likes of me’—more accessible for everyone. But what we don’t know is whether or not we are achieving that aim.
You see, with our other sites, WhatDoTheyKnow, FixMyStreet or WriteToThem, there is a set path that indicates a successful visit. The user lodges an FOI request, reports a problem, or writes to their representatives.
TheyWorkForYou, on the other hand, does not promote an action. There are actions you can take, like signing up for email alerts, or adding an annotation to a debate—but these are secondary to the site’s primary function, which is simply to present information. There’s no measurable metric that can tell us how informed people are after they’ve read one of our pages, nor what they do with that information. Hence the survey.
People hate pop-ups
One of our first findings came as no surprise: yes, we hate pop-ups too*! And yet, we wanted to keep users on the site while they answered our questions, so that’s the interface we used.
75.7% of people dismissed the pop-up when it appeared. We’d probably have done the same. We’d like to meet the remaining 24.3% of people and shake their hands.
The pop-up asked users what they would go on to do with the information they’d found on TheyWorkForYou. And most people gave the same answer they would if a shop assistant had asked: in fact, 75.2% said they were ‘just browsing’, with no further plans.
That makes us really curious as to what provoked that browsing: are they regular visitors, or were they sent to TheyWorkForYou via a tweet or Facebook reminder? Maybe one of our email alerts was the trigger. Google Analytics will allow us to see which site referred visitors to us, but of course we can’t match that data with individual motivations.
A further 10.9% of users had come with the intention of contacting their MP: TheyWorkForYou’s MP pages feature a prominent button inviting you to do that, and clicking on it takes you to our site WriteToThem.
6.6% of people were using the site for work purposes. Only 3.1% of people had plans to share content via social media, a statistic we’d like to increase if we can.
During the research period, some users (not necessarily those who had seen the pop-up) saw an addition to person pages and debate pages: a “where next” button, which led to some suggestions on how to use the information they’d found on the site.
This button was not widely used: just 0.1% of people clicked it. As so often with research findings, this result leads to more questions: did people already know what they wanted to do? Did they not want to do anything? Were they, perhaps, annoyed at the suggestion they might need guidance?
Or was the button just poorly positioned? We may run some further testing to find out if a more prominent placing leads to more clicks.
Of those few people who did click through for advice, the most popular options were to use WriteToThem to contact their MP about what they’d just read, or to contact the Parliamentary Ombudsman—it seems that on the few occasions when people do welcome a next step, they do want to take action.
Follow the news
Meanwhile, we were also taking a concentrated look at our site analytics. The test period coincided with the Labour party leadership contest and David Cameron’s cabinet announcements, and we saw very clearly how current affairs have an effect on which pages people visit.
John Whittingdale (the new Culture Secretary)’s voting record and Jeremy Corbyn’s profile page were far and away the most visited pages by several thousand (they’re responsible for the spike on the left of the graph that you see below). Also popular were the profile pages of the other Labour candidates, and Corbyn’s voting record.
But the other pages visited were very diverse: an extreme example of the long tail effect.
We reckon it takes more than a few seconds to take in a parliamentary debate, so one of the things we measure on Google Analytics is how many people stick around for more than 7 seconds, and again, for more than ten seconds.
Actually, we get pretty good results for the 7-second cut-off. Recent work we did on the design of the site’s debate pages had an effect that was a real cause for celebration: a massive 46% drop in the bounce rate (ie people who leave the site after viewing just one page)
But only 38% then stick around long enough for the second trigger, which doesn’t indicate as much involvement as we might have hoped for.
What have we learned?
At the end of this experiment, we reckon we’ve learned four main things:
- We need to carry on thinking about how to encourage more engagement with TheyWorkForYou’s content
- Current affairs are a huge driver to the site, and we can build on this via social media—and especially by encouraging our users to share the content they’ve found
- We need to conduct more experiments to see whether people genuinely don’t want advice about how to use the information they’ve found, or whether they would take it if it was more prominently flagged up
- There’s also a motivation to dig in deeper to motivations: what made people come to the site, if they are ‘just browsing’? What exactly are they doing ‘for work’? And if they want to contact their MP, what was the trigger for that?
If you have thoughts about any of these, as regards your own usage of TheyWorkForYou, please do feel free to comment below. And if you’d like to find out more about our research programme there’s plenty of information here, along with news of our annual research conference, TICTeC.
*And we know who to blame.
Image: The U.S. Printing Co., Russell-Morgan Print, Cincinnati & New York [Public domain] via Wikimedia Commons.
This post refers to the statistics we published for 2014. If you would like to see the latest responsiveness figures on WriteToThem, please visit www.writetothem.com/stats/.
Of course, there are many factors that you’ll consider before you cast your vote in the general election. But we think that one important quality in an MP is that they respond to their constituents.
So you may wish to check your own MP’s performance on the latest WriteToThem responsiveness league table. Just put in your postcode and you can see how they did in 2014.
Where the data comes from
When you send a message to your MP using our site WriteToThem, you’ll receive an automated email two weeks later, asking whether or not you received a response. Every year, we take the data from these surveys and use it to assemble our responsiveness rankings.
You might think that MPs would be doing the best they can this year, in the run-up to the election. Sadly, that doesn’t seem to be the case: overall, responsiveness has fallen a percentage point since last year, with 46% of emails receiving no reply.
You can find all our data and methodology on the league table page.
We know that messages sent to WriteToThem may not reflect all messages sent to an MP; we also know that not every message will require an answer. However, we think that, taken overall, our sample size of over 36,000 interactions can be seen as indicative.
We’re expecting 109 people from 26 different countries and 69 different organisations – all with a common interest in discussing and understanding more about the impact of civic tech.
You can see the full agenda here, and don’t worry if you didn’t manage to get a ticket: we’ll be documenting everything in full.
- For the as-it-happens picture, keep an eye on the Lanyrd page throughout tomorrow.
- We’ll be following up with summaries, podcasts, photos and videos right here on the mySociety blog.
- Be sure to tag your social media with #tictec and we’ll also document the best of that.
See you tomorrow!
Hurry: today’s the last day to book your place at TICTeC, our conference on the Impacts of Civic Technology, if you want to take advantage of the early bird pricing.
You have until midnight tonight to save yourself £100 on your ticket price. Here’s where to book.
We’re still firming up the final schedule and session titles, but let us whet your appetite by listing some of the speakers.
Here are some of the other speakers who’ll be helping to shape the agenda at TICTeC:
We’re really delighted to be presenting such a diverse group of speakers bringing insights from so many parts of the world… and we can hardly wait to hear what they all have to share.
If you feel the same, well, now’s the time to book your ticket.
In a recent blog post, we summarised the research we commissioned from the University of Manchester’s Rachel Gibson, Marta Cantijoch and Silvia Galandini, on whether or not our core UK websites have an impact.
The full research paper is now available, and you can download it here .
Professor Rachel Gibson says: “This research presents a unique and valuable insight into the users of online resources such as FixMyStreet and WhatDoTheyKnow.
“Through applying a highly original methodology that combines quantitative and in-depth qualitative data about people’s experience of mySociety sites over time, we provide a picture of how eDemocracy tools are contributing to activism at the local level.
“We thank all those that contributed to this important study and mySociety for their co-operation in developing this highly rewarding and academically rigorous project.”
Our thanks to Rachel, Marta and Silvia for conducting this research, which utilised methods not previously used in the civic tech field. We hope that it will prove a useful foundation to our own further research, and that of others.
We at mySociety build and popularise digital tools worldwide that help citizens exert power over institutions and decision-makers. Or do we?
Wanting to know whether our well-meaning civic tech is actually making a difference, mySociety recently created the post of Head of Research. My name is Dr Rebecca Rumbul, and I have now been installed in that role for about 6 weeks. I want to know if civic tech like ours is having an impact on citizens and governments, and how such sites operate and negotiate issues not just in the UK, but in the 50 or so countries that we know have digital democracy websites operating in them.
There is enormous scope for interesting and important research to be conducted using sites such as the ones that mySociety and our partners operate. The digital nature of our focus means that we can collect large volumes of data online at a low cost.
That said, there is nothing quite like making connections on the ground or meeting people face to face. mySociety is a small NGO, and does not have the capacity to conduct all of the research activities it would like on its own.
Therefore, we are actively seeking to work with academic partners on both qualitative and quantitative research focusing on the impact of civic tech.
We are planning to conduct research in the following countries. If you are an academic based in one of these countries and interested in our research agenda, please get in touch. We will be very happy to hear from you. Contact me via email@example.com
- South Africa
We conduct and disseminate research regularly. If you would like to hear more about our activities and events, sign up for our newsletter.
Image credit: Into the Unknown by Gary Gao (angrytoast), CC BY-NC 2.0
Today, we’re sharing research conducted on the impact of online Freedom of Information technology, including our own platform Alaveteli.
Researchers Savita Bailur and Tom Longley spent three months gathering first-person experiences, analysing data and assessing existing literature to answer this question:
“In what circumstances, if any, can the Freedom of Information tools mySociety builds be shown to have measurable impacts on the ability of citizens to exert power over underperforming institutions?”
You can read their findings here:
1. Literature review [PDF]
The research was conducted in three parts: first, Tom and Savita reviewed existing literature on the impact of FOI, particularly FOI online, to form a baseline of existing knowledge in the area.
They went on to interview people who run, or ran, FOI sites in 27 different countries. They used the resulting transcripts for qualitative research, pulling out common themes to help them draw conclusions.
Finally, they were able to use these insights to create a list of critical success factors for those implementing FOI (especially Alaveteli) websites.
Why did we conduct this research, and why now?
Alaveteli has had a period of intense growth over the last three years – but it would be irresponsible of us to continue its promotion without assessing its true worth and impact.
This is best learned from the people who are at the coalface – the implementers (as Tom and Savita mention in the final research, a fuller study would have allowed them to include government workers and the sites’ end users, too, but that’s perhaps something for the future).
Alaveteli was created with the best intentions – to allow anyone, anywhere to put questions to the people and institutions in power – but it is important to assess whether those intentions have been realised.
We need to ensure that we have spent our efforts and our funders’ money responsibly, and that we are not wasting resources by making poor decisions.
mySociety’s Head of Research, Dr Rebecca Rumbul, says, “This report confirms that the basic model does work, with the UK site WhatDoTheyKnow.com operating as a well-used civic resource with thousands of users per month.
“Whilst the research shows that our partners implementing Alaveteli in their own countries are demonstrably up to the technical challenge of running these sites, it identifies the importance of governmental relations and receiving the right support in the early stages of implementation.
“We now hope to build on this research to better understand how to maximise the use and effectiveness of our platforms around the world in empowering citizens to engage with governments and decision-makers.”
The research was made possible by a grant from the Open Society Foundations.
As the literature review confirms, this type of study has never been done before – and with practitioners speaking to the researchers from within many different cultural backgrounds and political regimes (they interviewed implementers of 20 Alaveteli instances, from Australia to Uruguay), we are in a unique position to take a global view on the subject. For a fully-rounded picture, the study also spoke to implementers of seven sites running non-Alaveteli FOI software.
Of the experience, Savita and Tom say “We were so impressed by the dedication and determination of all the implementers in wanting to raise awareness of FOI and seeing Alaveteli as the platform to do this (even taking into account constructive criticism). The research experience was also great.”
The end result? Take a look for yourself – if you have the slightest interest in online democratic technologies or government-to-citizen information sharing on an international basis, it’s compelling reading.
What we’ll take away
There are learnings for us here, although it was great to hear such consistent praise for the Alaveteli platform and the community that has been created around it.
mySociety’s Director Tom Steinberg said, “We will certainly be looking carefully at the recommendations that have come from this report.
“This will include decisions about how to share best practice across the Alaveteli community, and not just in the technical areas.
“We’ll also be looking hard at the issue of how to ensure consistency in the analytics that are collected by different sites. And we very much hope to return to the subject in a couple of years’ time, when today’s new sites have become established, in order to conduct a follow-up piece of research.”
What impact do mySociety sites actually have? We could lose a lot of sleep over this important question – or we could do something concrete, like conducting academic research to nail the answers down for once and for all.
As slumber enthusiasts, we went for the research option – and, to help us with this commitment we’ve recently taken on a new Head of Research, Rebecca Rumbul. Watch this space as she probes more deeply into whether our tools are making a difference, both in the UK and abroad.
Even before Rebecca came on board, though, we had set a couple of research projects in motion. One of those was in partnership with the University of Manchester, funded by the ESRC, which sought to understand what impact our core UK sites (FixMyStreet, WriteToThem, TheyWorkForYou and WhatDoTheyKnow) have on their users, and specifically on their level of political engagement.
Gateways to participation
It’s perhaps worth mentioning that, while our sites appear, on the face of it, to be nothing more than a handy set of tools for the general citizen, they were built with another purpose in mind. Simply put, each site aims to show people how easy it is to participate in democracy, to contact the people who make decisions on our behalf, and to make changes at the local and national levels.
Like any other online endeavour, we measure user numbers and transaction completions and time spent on site – all of that stuff. But one of the metrics we pay most attention to is whether users say they are contacting their council, their MP or a public body for the first time. Keeping track of this number ensures that we’re doing something to open democratic avenues up to people that haven’t used them before.
But there are plenty more questions we can ask about the impact we’re having. The University of Manchester study looked into one of them, by attempting to track whether there was a measurable change in people’s political activity and engagement after they’ve used one of our sites. On Monday, researchers Rachel Gibson, Marta Cantijoch and Silvia Galandini presented their findings to an attentive audience at King’s College London.
The project has taken a multi-pronged approach, asking our users to complete questionnaires, participate in online discussions, or keep a 12-week diary about political and community engagement (thanks very much to you, if you were one of the participants in this!). The result was a bunch of both qualitative and quantitative data which we’ll be able to come back to and slice multiple ways in the future – Gibson says that they haven’t as yet managed to analyse all of the free text diaries yet, for example.
In itself this study was interesting, because not much research has previously been conducted into the impact of digital civic tools – and yet, as we know from our own international activities, people (not least ourselves) are launching sites all over the world based on the premise that they work.
Some top-level conclusions
The research will be published in full at a future date, and it’s too complex to cover all of it within the confines of a short blog post, but here are just a few of the takeaway findings:
- A small but quantifiable uplift in ‘civic participation’ was noticed in the period after people had used our sites. This could include anything from working with others in the local community to make improvements, to volunteering for a charity.
- No change was found in the level of political influence or understanding that people judged themselves to have. This was a surprise to the researchers, who had thought that users would feel more empowered and knowledgeable after contacting those in power, or checking up on their parliamentary activity.
- As with our research back in 2011, the ‘average’ user of mySociety sites was found to be white, above middle-aged, and educated to at least degree level. Clearly this is a userbase which we desperately need to expand, and we’ll be looking carefully – with more research and some concentrated outreach efforts – at how we can do that.
- Users tended to identify themselves as people who already had an interest in politics. Again, here is an area in which we can improve. Of course, we’re happy to serve such users, but we also want to be accessible to those who have less of a baseline interest.
- Many users spoke of community action as bringing great satisfaction. In some cases, that was getting together in real life to make improvements, but others saw something as simple as reporting graffiti on FixMyStreet as an action that improved the local area for everyone.
Thanks to the University of Manchester researchers for these insights and for presenting them so engagingly. We’ll update when the full research is available.