Suppose we sent an automated tweet every time someone made a successful Freedom of Information request on WhatDotheyKnow — would it bring more visitors to the site?

And, if you get a response to your first FOI request, does it mean you are more likely to make a second one?

These, and many more, are the kind of questions that emerge as we refine the advice that we’re offering partner organisations.

Our Freedom of Information platform Alaveteli underpins Freedom of Information sites all around the world. When we first launched it, our only priorities were to make the code work, and to make that code as easy as possible to implement. But, as a community emerged around Alaveteli, we realised that we’d all be better off if we shared advice, successes and ideas.

And that’s where we began to encounter questions.

Some of them, like how to get more users, or how to understand where users come from, are common to anyone running a website.

Others are unique to our partner structure, in which effectively anyone in any part of the world may pick up the Alaveteli code and start their own site. In theory, we might know very little more than that a site is running, although we’ll always try to make contact and let the implementers know what help we can offer them.

There were so many questions that we soon saw the need to keep them all in one place. At mySociety, we’re accustomed to using Github for anything resembling a to-do list (as well as for its primary purposes; Github was designed to store code, allow multiple people to work on that code, and to suggest or review issues with it), and so we created a slightly unusual repo, Alaveteli-experiments.

This approach also gives us the benefit of transparency. Anyone can visit that repo and see what questions we are asking, how we intend to find the answers, and the results as they come in. What’s more, anyone who has (or opens) a Github account will also be able to add their own comments.

Have a browse and you’ll come across experiments like this one and this one, which attempt to answer the questions with which we opened this post.

Some of the experiments, like this one to analyse whether people click the ‘similar requests’ links in the sidebar, we’re running on our own site, WhatDoTheyKnow. Others, such as this one about the successful requests listed on every Alaveteli site’s homepage, are being conducted on our partners’ sites.

Our aims are to find out more about how to bring more users to all Alaveteli sites, how to encourage browsing visitors to become people who make requests, and how to turn one-off requesters into people who come back and make another — and then pass all that on to our partners.

We hope you’ll find plenty of interest on there. We reckon it’s all relevant, especially to anyone running an FOI website, but in many cases to anyone wondering how best to improve a site’s effectiveness. And we’re very happy to hear your ideas, too: if we’ve missed some obvious experiment, or you’ve thought of something that would be really interesting to know through the application of this kind of research, you’re  welcome to let us know.

You can open your own ticket on the repo, suggest it in the Alaveteli community mailing list, or email Alaveteli Partnerships Manager Gemma.

—
Image: Sandia Labs (CC by-nc-nd/2.0)

Researching in an unstable environment

It’s been nearly two years since the InfoLib Liberia project with iLab Liberia started. In that time the project has faced many hurdles, some predicted, and some completely unforeseen.

The iLab team have seen their country devastated by Ebola, only 11 years after the end of their second civil war, bringing tragedy and instability along with it. As you can probably imagine, the impact of curfews, fear and death in communities has made it difficult for people to continue with their daily lives. The social impact of such a disease is wide-reaching. Distrust, marginalisation and exclusion can be directed at those who show symptoms, or even who suffered and survived.

These are challenges that our local partners have had to contend with every day, both when holding training sessions and more crucially when researching the impact of the project on people’s lives.

However, by far the largest hurdle for this particular project has been a mixture of low internet penetration and lack of government will to release information. The team on the ground have been working tirelessly to create an ecosystem of requesting and training Public Information Officers (PIOs) to reply – even providing them with tablets to scan documents without needing electricity, let alone a computer. But if those officers have no access to the information that has been requested, their jobs become virtually impossible.

The project is now drawing to a close and we’re undertaking our final research survey. It seemed like a good time to take a look at what we’ve learnt about the impact of our joint Freedom of Information project in Liberia.

Results

When designing the project we decided that impact could best be measured in terms of whether or not the project increased confidence in government transparency.

We carried out surveys in January 2016 and April 2016, to provide a baseline picture and then an assessment of impact at midline. The final survey is being conducted in August 2016 just as the project ends.

The first survey – the baseline – was carried out mainly in the rural areas. iLab Liberia teamed up with LFIC to survey 152 participants who had been involved in the FOI workshops that LFIC had held in the counties.

We had to attempt the second survey twice, as it turned out to be more challenging than we’d expected. We needed the participants from the first questionnaire to answer the same questions we’d asked them initially, in order to measure change — but it proved hard to locate all of them.

There were many factors which caused this, but the main one was economic drivers, forcing people to move to where the opportunities are. It’s a problem many researchers must run into working in the field.

Carter, the project lead at iLab Liberia told us:

“There are several reasons why this happens […]. People migrate a lot between markets, farms. Several persons who participated in the baseline could not be reached as they [had] travelled to other cities/counties. [Or] the job that allowed them to reside in that city/county is no longer available so they might have left seeking after another job.”

Our second attempt was more successful. We managed to contact a large percentage of the original participants in the survey: 112 of the 152.

We’ve found out some interesting things from doing this research. We saw that 74% of people who use the internet daily say it’s their main source of information, though it is still only a small percentage of the population who have access to the internet.

So the next biggest source of information? Radio! 85% of people with with no access to the internet give radio as their main source of information. Thinking of the migration of workers between cities and counties – you suddenly appreciate why Radio is such an important medium for getting hold of information. Thankfully, as you’ll remember from our original blog post, we’re covering both of these media in the InfoLib project.

In the months since we began studying the impact of this project we also learned that fear of making a request has dropped by 5% in the individuals surveyed . The amount of people who reported that they didn’t know how to ask for information dropped from 24% to 21%. This is pretty great news to us as it shows that our training and our encouragement is working – albeit slowly.

Finally we saw the percentage of people who believe government would be more transparent if citizens could see the information they hold rise by 3% to 93% of the surveyed respondents. Even if this figure hadn’t risen, this demonstrates a clear existing demand from the citizens of Liberia for the Government to release more information about its activities which is great news overall!

Challenges

No project is without its challenges, and as you’ve seen above one of the big ones is ensuring that the same people respond from survey to survey. Not being able to pin down precisely the same set of people means that we can’t say with 100% certainty that we have a true measure in the difference in attitude.

As a result of the economic and social drivers mentioned above, the workforce in Liberia is very transient. This makes disseminating information through radio and internet mediums even more important. This research has shown that these are the primary sources of news and official information for the majority of Liberians, and continuing to improve knowledge about, and access to, information via these sources will empower the population further.

Finally, it can be challenging to demonstrate impact in projects like these, simply because research is not the main focus for our local partners. We partner with local groups because they are passionate, capable, and able to engage and mobilise citizens around a certain issue. We cannot expect small grassroots groups to have the resources or experience to conduct academic surveying, sampling or interviewing that could detect and definitively isolate the short term impact of a small project. This piece of research has provided some encouraging interim results, but most of all, it has provided valuable lessons to us at mySociety in trying to conduct this kind of impact research remotely and in partnership.

While we wait for the outcome of the final survey we can feel cautiously hopeful that this project has caused a small change in the way access to information operates in Liberia. infoLib will continue to run after the project officially ends, and mySociety will continue to support the work that iLab does in this area . However it may take longer than we had expected or hoped, to see the governmental shift towards releasing information.

Videos from TICTeC — our Impacts of Civic Technologies Conference — are now available for viewing.

So, whether you were there and you want to experience it all over again, or you just want to see what you missed out on, settle down and enjoy accounts from some of the most interesting civic tech projects around the world right now.

What is TICTeC?

Here’s mySociety CEO Mark Cridge explaining what TICTeC is, and why we run it:

And if that’s left you hungry for more, make sure you visit our YouTube channel to watch the other TICTeC videos.

You might also enjoy the slides, photos and other media that have all been collected together on the TICTeC 2016 page.

TICTeC is our annual conference on the impacts of civic technologies.

This year, we’ll be in Barcelona, Spain, with a diverse programme of speakers from all over the world.

Guy Grossman, assistant professor at the University of Pennsylvania’s political science department, will deliver one of two keynotes. See details of our other keynote speaker, Helen Milner OBE, here.

Guy has had a long history in civic tech research, with a special focus on Uganda and Sub-Saharan Africa, and we are delighted that he will be sharing his insights to our audience of practitioners and researchers.

Hi Guy! Tell us what you’ll be covering in your TICTeC keynote.

The state of academic knowledge about the promises and pitfalls of ICT applications designed to increase voice and accountability in low-income countries. Specifically:

• Under what conditions are citizens more likely to communicate with government officials using mobile-platforms? If we “build them – will they come”?
• Do mobile platforms exacerbate inequalities in political access, privileging groups (men, urban, better-off) that are heavier users of ICTs?
• What can be done to help increase the participation rates of marginalised populations (such as women and the poor)?

Low-income countries have leapfrogged past the landline era and directly into the mobile era. Citizens can now connect with each other and with their public officials in unprecedented ways.

Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited.

The massive penetration of mobile technology, even to remote areas of the least developed countries, has great potential to reshape both the social and political landscapes.

Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited. My goal at TICTeC is to help take stock of what we know and don’t know in this emerging field.

What are you hoping to get out of TICTeC?

I’m hoping to interact with practitioners, policy makers, technologists and fellow academics to figure out where the overlap is between practical needs and academic research.

I’m also hoping to get exposed to promising and innovative applications that I am unaware of.

Your civic tech research focuses especially on Uganda and Sub-Saharan Africa. What led you down this path?

I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries.

I care deeply about poverty and social and gender inequality, so working and studying in Africa has been a natural choice.

How did I get involved with civic tech? In the past decade I have witnessed an increasing number of mobile-based applications designed to enhance such things as agriculture productivity and financial transactions or match between buyers and sellers.

On the other hand, I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries. I came to believe that this was a great void that needed to be filled.

In fact, unfortunately, we are still quite far from saturation in the development of governance-related ICT applications.

What do you think are the big unanswered questions when it comes to civic tech?

Does participation in mobile communication spill into more traditional forms of political participation?

Here’s a very partial list of some of the answered questions when it comes to civic tech in the global south:

Can simple innovations in mobile technology be used to facilitate new meaningful forms of political participation?

If so, what types of mobile-based political communication are most likely to be adopted by voters?

What is the potential of ICTs to flatten political access to marginalised populations?

What are some of the ways to increase the usage of mobile-based political communication in the face of clear collective action problems?

How can ICT be used to overcome political economy constraints to growth (e.g. by resolving monitoring challenges in government bureaucracies, reducing the cost for citizens of tracking bureaucratic performance, or making it easier to detect corrupt practices in public-sector salary distribution)?

And, does participation in mobile communication spill into more traditional forms of political participation?

If you could make one recommendation to those developing new civic tech, and wanting to see real impact from it, what would it be?

Adopt a user perspective — make sure that the suggested application/platform addresses a real need that could not be addressed using a low-tech solution.

You won’t want to miss what Guy has to say at TICTeC, so make sure you book your tickets now.

In my last post I described how we’re taking stock of where and how we’re delivering against our theory of change to give greater influence to citizens over those with power.

Since starting at mySociety I’ve spent my time meeting lots of lovely people, getting to know the team, our funders, partners and peers and finding out how mySociety does what it does.

One thing I have learned is that despite our British roots, the majority of our work is now international, and we work with wonderful partners in over 35 countries around the world, from Ukraine to South Africa, Liberia to Norway. In each case they tend to be activists, journalists and NGOs who are passionate about better government, citizen empowerment, and fighting corruption.

Our success is defined by our partners’ success – so in order to best support our partners I’d like to talk a little bit about some of the practical steps we’re taking to consolidate what we already have and scale up what works.

Four Simple Goals

The core mission of mySociety remains the same: to invent and popularise websites and apps that enable citizens around the world to exert power over institutions and decision makers.

We see the need to both ‘invent’ and ‘popularise’ digital tools as equally important – digital tools can be useful in developing new approaches to difficult problems, but we must ensure they are both widely used and actually enable citizens to be capable of demanding better.

In order to best help our partners and to better understand the impact of our work we have four really simple goals that will direct our efforts over the next few years:

Planning For Success

In addition to running our successful UK sites TheyWorkForYou.com, WhatDoTheyKnow.com, WriteToThem.com and FixMyStreet.com, we’ll continue to work with our partners to improve our existing services, making them easier to deploy and better integrated together.

We’ve recently established a quarterly call for new proposals for potential new partners who wish to set up new sites of their own from our roster of services (FOI, Parliaments and Elections, FixMyStreet). This helps inform potential partners of what’s involved before getting started, and helps us better target our resources and plan for success upfront.

We’re also putting more effort into increasing the impact and usage of our existing sites and services, by providing targeted development support, training, direct funding and additional technical development. Helping to sustain each site through the difficult first year or two should be a major marker of success.

Proving What Works

One major thing that will change is putting our research much more front and centre to our work, in order to create a greater evidence base for the impacts of civic technology and ensuring we are able to talk about this widely and publicly.

You’ll see us carry out much more inclusive and comparative impact research on the use of civic technology encompassing individual, socio-political and sector-specific factors.

If you haven’t already read our latest research paper ‘Who Benefits From Civic Technology?’ then please do have a look. This is an important first step in laying down the case for impact, being honest about where more work is required and focusing our efforts to create a greater evidence base for civic technology as a whole.

Our long-term aim is to establish a global hub for impact research, and assist more civic tech organisations to assess and improve the impact of their own work. To this end we’ll be hosting our next TICTeC – The Impacts of Civic Technology Conference, in Barcelona on the 27th and 28th of April next year.

This will be an important opportunity to share and discuss research findings and key challenges from across the sector and we hope to see many of you there in person.

Where We Go Next

Over the past decade, through a process of experimentation, consultation and measurement, mySociety have created a portfolio of popular, proven online services, used by over 10 million people each year.

This is an amazing legacy to take on.

Over the next decade I hope that we’ll continue this work, and seek to further establish mySociety as one of the leading international civic technology institutions, providing much-needed global leadership and inspiration in our sector – if we could come to be seen as having a similar impact to that of an Article 19 or Human Rights Watch in our own field, then I think that will be a pretty good measure of success.

For the moment we’ll continue to focus on the practical steps we need to take in order to improve and build upon what we already have, but I’m excited about the plans we have for the future and I’ll share more details on what we have in store in the weeks to come.

—

Image: Rachel Pasch (CC)

How do you pin down the intangible?

More specifically, how do we understand something as nebulous as why people visit our websites? For better or worse, Google Analytics can’t provide brain scans to show our users’ motivations, so the only solution is to ask them.

Between May and August of this year, if you visited TheyWorkForYou, you might have seen a pop-up inviting you to answer a few questions. Some users also gained new site features. It was all part of a concerted drive to understand more about why you use the site, and part of our wider research programme funded by The William and Flora Hewlett Foundation.

Many thanks if you were one of those who responded. In all, the survey generated a couple of a million data points, which has certainly given our resident data-cruncher, Nick, plenty to wade through.

Why do people use TheyWorkForYou?

We know why we run TheyWorkForYou: it aims to make Parliament—often perceived as ‘not for the likes of me’—more accessible for everyone. But what we don’t know is whether or not we are achieving that aim.

You see, with our other sites, WhatDoTheyKnow, FixMyStreet or WriteToThem, there is a set path that indicates a successful visit. The user lodges an FOI request, reports a problem, or writes to their representatives.

TheyWorkForYou, on the other hand, does not promote an action. There are actions you can take, like signing up for email alerts, or adding an annotation to a debate—but these are secondary to the site’s primary function, which is simply to present information. There’s no measurable metric that can tell us how informed people are after they’ve read one of our pages, nor what they do with that information. Hence the survey.

People hate pop-ups

One of our first findings came as no surprise: yes, we hate pop-ups too*! And yet, we wanted to keep users on the site while they answered our questions, so that’s the interface we used.

75.7% of people dismissed the pop-up when it appeared. We’d probably have done the same. We’d like to meet the remaining 24.3% of people and shake their hands.

Just browsing

The pop-up asked users what they would go on to do with the information they’d found on TheyWorkForYou. And most people gave the same answer they would if a shop assistant had asked: in fact, 75.2% said they were ‘just browsing’, with no further plans.

That makes us really curious as to what provoked that browsing: are they regular visitors, or were they sent to TheyWorkForYou via a tweet or Facebook reminder? Maybe one of our email alerts was the trigger. Google Analytics will allow us to see which site referred visitors to us, but of course we can’t match that data with individual motivations.

A further 10.9% of users had come with the intention of contacting their MP: TheyWorkForYou’s MP pages feature a prominent button inviting you to do that, and clicking on it takes you to our site WriteToThem.

6.6% of people were using the site for work purposes. Only 3.1% of people had plans to share content via social media, a statistic we’d like to increase if we can.

Need suggestions?

During the research period, some users (not necessarily those who had seen the pop-up) saw an addition to person pages and debate pages: a “where next” button, which led to some suggestions on how to use the information they’d found on the site.

This button was not widely used: just 0.1% of people clicked it. As so often with research findings, this result leads to more questions: did people already know what they wanted to do? Did they not want to do anything? Were they, perhaps, annoyed at the suggestion they might need guidance?

Or was the button just poorly positioned? We may run some further testing to find out if a more prominent placing leads to more clicks.

Of those few people who did click through for advice, the most popular options were to use WriteToThem to contact their MP about what they’d just read, or to contact the Parliamentary Ombudsman—it seems that on the few occasions when people do welcome a next step, they do want to take action.

Follow the news

Meanwhile, we were also taking a concentrated look at our site analytics. The test period coincided with the Labour party leadership contest and David Cameron’s cabinet announcements, and we saw very clearly how current affairs have an effect on which pages people visit.

John Whittingdale (the new Culture Secretary)’s voting record and Jeremy Corbyn’s profile page were far and away the most visited pages by several thousand (they’re responsible for the spike on the left of the graph that you see below). Also popular were the profile pages of the other Labour candidates, and Corbyn’s voting record.

But the other pages visited were very diverse: an extreme example of the long tail effect.

Most people don’t hang around

We reckon it takes more than a few seconds to take in a parliamentary debate, so one of the things we measure on Google Analytics is how many people stick around for more than 7 seconds, and again, for more than ten seconds.

Actually, we get pretty good results for the 7-second cut-off. Recent work we did on the design of the site’s debate pages had an effect that was a real cause for celebration: a massive 46% drop in the bounce rate (ie people who leave the site after viewing just one page)

But only 38% then stick around long enough for the second trigger, which doesn’t indicate as much involvement as we might have hoped for.

What have we learned?

At the end of this experiment, we reckon we’ve learned four main things:

• We need to carry on thinking about how to encourage more engagement with TheyWorkForYou’s content
• Current affairs are a huge driver to the site, and we can build on this via social media—and especially by encouraging our users to share the content they’ve found
• We need to conduct more experiments to see whether people genuinely don’t want advice about how to use the information they’ve found, or whether they would take it if it was more prominently flagged up
• There’s also a motivation to dig in deeper to motivations: what made people come to the site, if they are ‘just browsing’? What exactly are they doing ‘for work’? And if they want to contact their MP, what was the trigger for that?

If you have thoughts about any of these, as regards your own usage of TheyWorkForYou, please do feel free to comment below. And if you’d like to find out more about our research programme there’s plenty of information here, along with news of our annual research conference, TICTeC.

—

*And we know who to blame.

Image: The U.S. Printing Co., Russell-Morgan Print, Cincinnati & New York [Public domain] via Wikimedia Commons.

This post refers to the statistics we published for 2014. If you would like to see the latest responsiveness figures on WriteToThem, please visit www.writetothem.com/stats/.

—

Of course, there are many factors that you’ll consider before you cast your vote in the general election. But we think that one important quality in an MP is that they respond to their constituents.

So you may wish to check your own MP’s performance on the latest WriteToThem responsiveness league table. Just put in your postcode and you can see how they did in 2014.

Where the data comes from

When you send a message to your MP using our site WriteToThem, you’ll receive an automated email two weeks later, asking whether or not you received a response. Every year, we take the data from these surveys and use it to assemble our responsiveness rankings.

A downturn

You might think that MPs would be doing the best they can this year, in the run-up to the election. Sadly, that doesn’t seem to be the case: overall, responsiveness has fallen a percentage point since last year, with 46% of emails receiving no reply.

Some caveats

You can find all our data and methodology on the league table page.

We know that messages sent to WriteToThem may not reflect all messages sent to an MP; we also know that not every message will require an answer. However, we think that, taken overall, our sample size of over 36,000 interactions can be seen as indicative.

—

Image: Michael Scott (CC)

Woah: TICTeC, the Impacts of Civic Technology Conference, is tomorrow. Tomorrow! That came quickly.

We’re expecting 109 people from 26 different countries and 69 different organisations – all with a common interest in discussing and understanding more about the impact of civic tech.

You can see the full agenda here, and don’t worry if you didn’t manage to get a ticket: we’ll be documenting everything in full.

• For the as-it-happens picture, keep an eye on the Lanyrd page throughout tomorrow.
• We’ll be following up with summaries, podcasts, photos and videos right here on the mySociety blog.
• Be sure to tag your social media with #tictec and we’ll also document the best of that.

See you tomorrow!

Hurry: today’s the last day to book your place at TICTeC, our conference on the Impacts of Civic Technology, if you want to take advantage of the early bird pricing.

You have until midnight tonight to save yourself £100 on your ticket price. Here’s where to book.

Speakers

We’re still firming up the final schedule and session titles, but let us whet your appetite by listing some of the speakers.

We’ve already introduced our two keynotes, Dr Shelley Boulianne and Ethan Zuckerman.

Here are some of the other speakers who’ll be helping to shape the agenda at TICTeC:

Luke Bacon of Open Australia Foundation, Sydney

Jonathan Bright of Oxford Internet Institute, UK

Tim Davies of Practical Participation, London

Kerry Brennan of Reboot, New York

Blair Glencorse of Accountability Lab, Washington DC

Nanjira Sambuli of ihub research, Nairobi

Linda Sandvik of the Guardian, London

Sandy Schuman of New College, Oxford University

Martin Szyszlican of Congreso Interactivo, Buenos Aires

Dr Nick Taylor of University of Dundee

Dr Loren Treisman of Indigo Trust, London

Gail Ramster of The Helen Hamlyn Centre for Design, London

Jonathan Mellon of the World Bank/Nuffield College, Baltimore

Jean Brice Tetka of Transparency International, Berlin

We’re really delighted to be presenting such a diverse group of speakers bringing insights from so many parts of the world… and we can hardly wait to hear what they all have to share.

If you feel the same, well, now’s the time to book your ticket.

In a recent blog post, we summarised the research we commissioned from the University of Manchester’s Rachel Gibson, Marta Cantijoch and Silvia Galandini, on whether or not our core UK websites have an impact.

The full research paper is now available, and you can download it here .

Professor Rachel Gibson says: “This research presents a unique and valuable insight into the users of online resources such as FixMyStreet and WhatDoTheyKnow.

“Through applying a highly original methodology that combines quantitative and in-depth qualitative data about people’s experience of mySociety sites over time, we provide a picture of how eDemocracy tools are contributing to activism at the local level.

“We thank all those that contributed to this important study and mySociety for their co-operation in developing this highly rewarding and academically rigorous project.”

Our thanks to Rachel, Marta and Silvia for conducting this research, which utilised methods not previously used in the civic tech field. We hope that it will prove a useful foundation to our own further research, and that of others.

Image: Nick Southall (CC)