What impact do mySociety sites actually have? We could lose a lot of sleep over this important question – or we could do something concrete, like conducting academic research to nail the answers down for once and for all.

As slumber enthusiasts, we went for the research option – and, to help us with this commitment we’ve recently taken on a new Head of Research, Rebecca Rumbul. Watch this space as she probes more deeply into whether our tools are making a difference, both in the UK and abroad.

Even before Rebecca came on board, though, we had set a couple of research projects in motion. One of those was in partnership with the University of Manchester, funded by the ESRC, which sought to understand what impact our core UK sites (FixMyStreet, WriteToThem, TheyWorkForYou and WhatDoTheyKnow) have on their users, and specifically on their level of political engagement.

Gateways to participation

It’s perhaps worth mentioning that, while our sites appear, on the face of it, to be nothing more than a handy set of tools for the general citizen, they were built with another purpose in mind. Simply put, each site aims to show people how easy it is to participate in democracy, to contact the people who make decisions on our behalf, and to make changes at the local and national levels.

Like any other online endeavour, we measure user numbers and transaction completions and time spent on site – all of that stuff. But one of the metrics we pay most attention to is whether users say they are contacting their council, their MP or a public body for the first time. Keeping track of this number ensures that we’re doing something to open democratic avenues up to people that haven’t used them before.

Questioning impact

But there are plenty more questions we can ask about the impact we’re having. The University of Manchester study looked into one of them, by attempting to track whether there was a measurable change in people’s political activity and engagement after they’ve used one of our sites. On Monday, researchers Rachel Gibson, Marta Cantijoch and Silvia Galandini presented their findings to an attentive audience at King’s College London.

The project has taken a multi-pronged approach, asking our users to complete questionnaires, participate in online discussions, or keep a 12-week diary about political and community engagement (thanks very much to you, if you were one of the participants in this!). The result was a bunch of both qualitative and quantitative data which we’ll be able to come back to and slice multiple ways in the future – Gibson says that they haven’t as yet managed to analyse all of the free text diaries yet, for example.

In itself this study was interesting, because not much research has previously been conducted into the impact of digital civic tools – and yet, as we know from our own international activities, people (not least ourselves) are launching sites all over the world based on the premise that they work.

Some top-level conclusions

The research will be published in full at a future date, and it’s too complex to cover all of it within the confines of a short blog post, but here are just a few of the takeaway findings:

• A small but quantifiable uplift in ‘civic participation’ was noticed in the period after people had used our sites. This could include anything from working with others in the local community to make improvements, to volunteering for a charity.
• No change was found in the level of political influence or understanding that people judged themselves to have. This was a surprise to the researchers, who had thought that users would feel more empowered and knowledgeable after contacting those in power, or checking up on their parliamentary activity.
• As with our research back in 2011, the ‘average’ user of mySociety sites was found to be white, above middle-aged, and educated to at least degree level. Clearly this is a userbase which we desperately need to expand, and we’ll be looking carefully – with more research and some concentrated outreach efforts – at how we can do that.
• Users tended to identify themselves as people who already had an interest in politics. Again, here is an area in which we can improve. Of course, we’re happy to serve such users, but we also want to be accessible to those who have less of a baseline interest.
• Many users spoke of community action as bringing great satisfaction. In some cases, that was getting together in real life to make improvements, but others saw something as simple as reporting graffiti on FixMyStreet as an action that improved the local area for everyone.

Thanks to the University of Manchester researchers for these insights and for presenting them so engagingly. We’ll update when the full research is available.

 Image: Phil Richards (CC)

The Written Answer is a noble parliamentary tradition, dating back almost 300 years. MPs and peers use them to hold the government to account, getting facts and figures on the record.

But wriggling out of answering them is also a recognised Parliamentary skill – and one that, while often applied with dexterity, can impede the process of democracy.

That’s the primary reason that, beside each Written Answer on TheyWorkForYou, we poll our users on a single point:

“Does this answer the above question?”

Last month marked the tenth birthday of TheyWorkForYou, and over that time, this unassuming poll has amassed more than 275,000 of these yes or no responses on a total of around 130,000 written answers.

That’s a substantial sample for us to analyse. Running that data through a few tickertape machines and putting the results in order means that we can now see just how many written answers actually address the question in hand – and which government departments are the best and worst at giving a straight answer.

Is the current administration more slippery?

It seems that ministers are getting worse at returning a straightforward answer.

In the previous government: 47% of written answers that were voted on got more ‘yes’ answers than ‘no’s from our users.

In the current administration: That figure has dropped to 45%. Even within the current term, the figure has been falling year on year, with a 49% ‘yes’ rate in 2010 comparing to a 42% rate in 2013.

Best and worst departments for a straight answer

Breaking down the data by department is also eye-opening – some departments are decidedly more likely to be judged as prevaricators by TheyWorkForYou’s users.

Accolade for ‘most improved’ goes to the Wales office, who managed an 86% ‘yes’ rate in the current government, against 48% in the last. Worst of the bunch – as perceived by TheyWorkForYou’s users – is the Department of Work and Pensions, with just 31% in this administration.

We’ve put the full rankings below, for those of you who would like to delve deeper into these figures.

(more…)

In a break from tradition, I’m going to start this blog with an appeal.

We on the international team at mySociety are trying to improve the install process and documentation for all of our internationalised websites. Since we built the original sites, we’re not the best people to ask on what needs to be improved, as I’m sure you understand. If you’re interested in helping us out doing this I’ve created two surveys, you’ll find them at the end of this post! Or email me at hello@mysociety.org so I can ask you a few questions. On to other exciting things…

In site news we are working on Alaveteli sites for Uganda and Italy. Both of these should be finished and ready for launch soon, thanks to our developers and of course our partners for showing interest.

We’ve also been helping set up a FixMyStreet site in Cape Verde and a demo FixMyStreet site for Whypoll in India. While these two sites are being installed on mySociety’s servers, three people from Singapore and two people from South Africa are also working on FixMyStreet for their countries, as self installs.

And in Pombola news we are helping with websites in South Africa, Zimbabwe and are hoping to work with a team in Malawi.

But these are just the most recent sites! People are working on sites in Uruguay, Bosnia, Croatia, Italy and a number of other countries. Follow our twitter @mysocietyintl to find out more.

We’d love to help you set up your own site, or just give you advice on why sites like these can be useful. Send me an email at hello@mysociety.org to find out how!

Finally, we’re going to be attending a few conferences and we’d love to meet up with you to chat and get to know you. You can find us at:

15th to 19th September – OKCon, Geneva (Jen and Dave)

27th to 28th September – OverTheAir, Bletchley Park (Dave)

30th Sept to 3rd October – African Entrepreneurship Summit, Mauritius (Paul)

25th to 27th October – Mozfest, London (Dave)

30th October to 1st November – OGP London (Paul and Jen)

27th to 29th November – World Forum for Democracy, Strasbourg  (Jen)

Please do drop by and say hello!

By the way, if you are hosting a conference and want us to come along and speak (for free! We don’t charge, and a lot of the time we try to pay our own way!) please drop a note to hello@mysociety.org . We love to connect with new people and would be delighted to be involved!

As promised, here are the survey links. If you have ever installed or had us create one of our websites for you please take a look at them and fill them in.
FixMyStreet Survey
Alaveteli Survey

One more thing, as a p.s. Hopefully these “What we’ve been up to” updates will soon come to you in video format! Be kind to me if the first one is awkward!

Hand photo by Alban Gonzalez | Android photo by Tiago A Pereira | Bike photo by Raul Lieberwirth | Thank you for making your content creative commons distribution.

Thanks to the work of thousands of volunteers across the country, we’ve now launched our survey of candidates to be your MP.

View the survey results

It tells you the views of candidates on a range of national and local issues. What’s particular exciting is that this is individual views – we separately surveyed all the candidates.

About 1/3rd of them have replied. The survey has a tool to let you ask the other candidates in your constituency to respond. Please give it a go, as we’d like to the survey to get as complete as possible over the weekend, to be most use to people in the last days leading up to the election.

Competition! Have you found an inventive way to ask your candidates to respond to the survey? Maybe you doorstepped them, or sent them a cake. Post your ideas and things you’ve done in the comments below.

As you may know, TheyWorkForYou are conducting a survey of candidates for Parliament.

Quite a few people have been asking how we worked out the questions. There are two parts to this, one local and one national.

Local questions

We used the power of volunteers.

Thousands of DemocracyClub members were asked to suggest local issues in there area. These were then edited by other volunteers, to have consistent grammar, and be worded as statements to agree/disagree with, and filtered to remove national issues. The full criteria and examples are available.

You can view the issues for any constituency on the DemocracyClub site. They are in the “local questions” tab.

We’ve ended up with local issues for about 85% of constituencies. They’re really interesting and high quality, and quite unique for a national survey.

Thank you to all the volunteers who helped make this happen!

National questions

This was hard, because we felt that asking more than 15 questions would make the survey too long. We also wanted to be sure it was non-partisan.

We convened a panel of judges, either from mySociety/Democracy Club or with professional experience in policy, and from across the political spectrum. They were:

• James Crabtree, chair of judges, trustee of mySociety, journalist for Prospect magazine
• Tim Green, Democracy club developer, Physics student, Cambridge University.
• Michael Hallsworth, senior researcher, Institute for Government.
• Will Davies, sociologist at University of Oxford, has worked for left of centre policy think tanks such as IPPR and Demos.
• Andrew Tucker, researcher at Birkbeck, worked for Liberal Democrats from 1996-2000.
• Robert McIlveen, research fellow, Environment and Energy unit at Policy Exchange, did PhD on Conservative party election strategy.

They met at the offices of the Institute for Government, and had a 3 hour judging session on 29th March 2010. They were asked to think of 8-15 questions, with multiple choice answers, which could usefully be answered both by members of the public and prospective candidates for national office.

To ensure maximum transparency, the discussions of the judges were recorded. You can download the recordings in two parts: part 1, part 2 (2 hours, 20 mins total).

Details of the broad framework the judges operated under are given by the chair of judges, James Crabtree, a trustee of mySociety, in the opening to the recordings.

Please do ask any questions in the comments below.

The following is a message that we’d like to see emailed around within political parties of all stripes. If you work for a party, or know anyone who does, please send it along:

———-

Hi there,

TheyWorkForYou.com has sent online surveys to nearly 3000 candidates across the UK, including most of your party’s candidates. If you don’t know it, TheyWorkForYou is probably the largest politician transparency website in the UK, with about 3m visitors last year.

The survey we’ve sent is a rigorously neutral attempt to clarify candidates positions on many of the biggest issues at the election. It is also a long-term document – the data that comes from candidate responses will be viewed millions of times between now and the general election after this one. It also contains both local and national questions.

There are 6000+ volunteers now nagging non-responsive candidates. You can help your party improve its responsiveness rating, here, by passing on the word that TheyWorkForYou’s survey is not push-polling, not single issue, not short-termist.

Please help us by passing on the message that TheyWorkForYou will be one of the main ways that new MPs from all parties (and none) will be scrutinised and neither we nor new MPs want to start our relationship with a “refused to go on the record” badge on their pages.

If you are a candidate, and you want to do the survey, check your email for TheyWorkForYou (no spaces). If you don’t have it, drop a mail to developers@democracyclub.org.uk and it’ll be sent along shortly.

many thanks,

The staff and volunteers at TheyWorkForYou and Democracy Club

In January last year, at our yearly staff and volunteers retreat, we decided that TheyWorkForYou should do something special for the general election. We decided that we wanted to gather information on where every candidate in every seat stood on what most people would think were the biggest issues, not just nationally but locally too.

Our reasons for setting this ambitious goal were two fold. First, we thought that pinning people down to a survey that didn’t reward rhetorical flourishes would help the electorate cut through the spin that accompanies all elections. But even more important was to increase our ability to hold new MPs to account: we want users of TheyWorkForYou in the future to be able to see how Parliamentary voting records align with campaign statements.

This meant doing quite a lot of quite difficult things:

1. Working out who all the candidates are (thousands of them)
2. Working out how to contact them.
3. Gathering thousands of local issues from every corner of the country, and quality assuring them.
4. Developing a balanced set of national issues.
5. Sending the candidates surveys,  and chasing them up.

The Volunteer Army

This has turned out to be a massive operation, requiring  the creation of the independent Democracy Club set up by the amazing new volunteers Seb Bacon and Tim Green,  and an entire candidate database site YourNextMP, built by another new volunteer Edmund von der Burg.  Eventually we managed to get at least one local issue in over 80% of constituencies, aided by nearly 6000 new volunteers spread from Lands End to John O’Groats. There’s at least one volunteer in every constituency in Great Britain, and in all but three in Northern Ireland. Volunteers have done more than just submit issues, they’ve played our duck house game  to help gather thousands of email addresses, phone numbers, and postal addresses.

The Survey

What we ended up with is a candidate survey that is different for every constituency – 650 different surveys, in short. The survey always contains the same 15 national issues (chosen by a politically balanced panel held at the Institute for Government)  and then anything between zero and ten local issues. We’ve seen everything from cockle protection to subsidies for ferries raised – over 3000 local issues were submitted, before being painstakingly moderated, twice, by uber-volunteers checking for for spelling, grammar, obvious bias and straightforward interestingness (it isn’t really worth asking candidates if they are in favour of Good Things and against Bad Things).

In the last couple of days we’ve started to send out the first surveys – we’ve just passed 1000 emails, and there are at least 2000 still to be sent.

The Output

We’re aiming to release the data we are gathering on candidates positions on 30th April. We’ll build a nice interface to explore it, but we also hope that others will do something with what we are expecting to be quite a valuable dataset.

The Pressure

Candidates are busy people, so how do we get their attention? Happily, some candidates are choosing to answer the survey just because TheyWorkForYou has a well know brand in the political world, but this has limits.

The answer is that we are going to ask Democracy Club, and it’s army of volunteers to help. We’ll shortly roll out a tool that will tell volunteers which of their candidates haven’t taken the opportunity to go on the record , and provide a range of ways for them to push for their candidates to fill it in.

It would be a lie to say we’re confident we’ll get every last candidate. But we are confident we can make sure that no candidate can claim they didn’t see, or didn’t know it was important to their constituents. And every extra voice we have makes that more likely.

Join Democracy Club today

I’ve been working on PledgeBank quite a bit recently. As well as adding survey emails asking whether signers have done their pledge, and a feature for people to contact a pledge’s creator, I’ve been fixing numerous bugs that have sprung up along the way. For starters, people on the Isle of Man and the Channel Islands now get a much more helpful error if they try and enter their postcode anywhere on the site, rather than the confusing postcode not recognised they were getting previously.

Other errors I found turned out not to be with our code. The PledgeBank test suite (that we run before deploying the site to check it all still works) was throwing lots of warnings about “Parsing of undecoded UTF-8 will give garbage” when it got to the testing of our other language pages. Our code wasn’t doing anything special, and there were multiple places the warning came from – upgrading our libwww-perl removed one, and I’ve submitted bug reports to CPAN for the rest (having patched our copies locally – hooray for open source).

The Perl warnings were at least understandable, though. While tracking down why the site was having trouble sending a couple of emails, I discovered that we had a helper function splitting very long words up to help with word-wrapping – which when applied to some Chinese text was cutting a UTF-8 multibyte character in two and invalidating the text. No problem, I think, I simply have to add the “/u” modifier to PHP’s regular expression so that it matches characters and not bytes. This didn’t work, and after much playing had to submit a bug report to PHP – apparently in PHP “non-space character followed by non-space character” isn’t the same as “two non-space characters in a row”…

I’m still busy beavering away at the Facebook / PledgeBank integration. It all works now, but will take a bit more polishing to get just right. Matthew is, I think adding surveys to PledgeBank. So it finds out later if people have or have not done their pledge. Or is he updating to a new version of BoundaryLine at the moment, so our postcode lookup on WriteToThem and everywhere else gets better? Hard to keep track when he does so much at once.

Keith is upgrading our internal documentation, so new people at mySociety can learn how to keep things going. Heather is stalking all of America, finding people to use and promote PledgeBank. Tom is on a much deserved holiday, after seemingly a zillion meetings per day for months.

There’s lots of ongoing maintenance for all our sites. We’re lucky that large chunks of our customer support email are done by volunteers (thanks Anna, Louise, Tim and Tomski/James) and by Debbi (yay Debbi!). Much of this is routine – changing pledge text, updating council email addresses, giving MPs posting links for HearFromYourMP, putting new MP photos up on TheyWorkForYou etc. A lot of it is unique – handling new translations, answering questions from MPs and Lords about their voting record. I’ll let the others give some more examples of the kind of thing we answer.

Speaking of which, do you know any good web developers who would like to work for mySociety? If so, put them in touch.