Trying to Practice What We Preach: mySociety Evaluation Reports Published

Transparent screen by Kathy McLeod

Transparent screen, by Kathy McLeod, used under the Creative Commons licence, with thanks.

At mySociety we like transparency – it’s baked into most of our projects.

TheyWorkForYou attempts to make it easier to find out what your MP has been doing in Parliament. WhatDoTheyKnow tries to make it easier to find out what’s going on inside other public bodies. FixMyStreet and the upcoming FixMyTransport also use transparency to help get problems resolved.

We think transparency is a good thing for many reasons, but one of its rarely mentioned virtues is how valuable transparency can be for the people within the organisations which are transparent.

Transparency can be useful because it means people outside an organisation can make critical, constructive suggestions about how you can improve, and it lowers the odds that people in one part of your own organisation will be ignorant of the activities of people in other parts.

Research

To that end we commissioned Tobias Escher of the Oxford Internet Institute to conduct detailed analysis into two of our major websites – TheyWorkForYou and WriteToThem.

We were not highly prescriptive in our instructions, and we certainly didn’t ask Tobias to ‘discover’ pre-determined findings. All we did was ask Tobias to find out who was coming to the sites, what they were doing, and whether or not the sites could be considered to be succeeding.  We didn’t do it for a PR stunt:  we did it so we could learn from our mistakes, and so that we could share those learnings with others who might benefit.

His detailed, quantitative analysis holds the sites up to mySociety’s own stated aims, for the first time. And we’ve published both documents, in full, below.

Swings and Roundabouts

It was great to discover that we have, indeed, attained some of our goals by running these sites. For example, one of the reasons we set up TheyWorkForYou and WriteToThem was to make representatives accessible to people who were newcomers to the democratic process. It was therefore heartening to read that 60% of visitors to TheyWorkForYou had never previously looked up who represents them, and two in five users of WriteToThem have never before contacted one of their political representatives.

But, as you would expect with any properly neutral evaluation, it’s not all good news. Our sites aim to reach a wide range of people, but compared to the average  British internet user, WriteToThem users are twice as likely to have a higher degree and a higher income. It also seems that users are disproportionately male, white, and over 35. These figures and many more are available within these highly readable papers – Tobias did a terrific job in gathering and analysing a huge amount of data, and then making it easy to understand.

Stories

These reports are rich with data, from how visitor numbers boomed during the MPs’ expenses scandal to which MPs most people sign up to receive alerts about. You can also read how a budget airline almost brought a site to its knees in 2007; what part Joanna Lumley plays in our history; and how many visits to TheyWorkForYou actually come from within Parliament itself.

TheyWorkForYou and WriteToThem have inspired many people around the world to set up similar (and not so similar) sites inspired by the vision of using the Internet to lower barriers to democracy. However, until now we’ve never seen a really clear-eyed assessment of what seems to work, and what doesn’t.

If you’re at all interested in using the Internet to engage people with democratic systems, Tobias Escher’s excellent research papers will make a compelling read. Thank you Toby!

The reports

Download Tobias Escher’s research report on WriteToThem.compdf

or TheyWorkForYou research report by Tobias Escherpdf

…and do come back and tell us what you found interesting.

PS

We hope to publish two evaluation reports like this at the start of each new year from now on. Next year’s sites will probably be FixMyStreet and WhatDoTheyKnow. Do get in touch if you’d like to input!

8 Comments

  1. I think Write to Them is mega, I would never have bothered writing to my MP before. I also like They work For You, because it sends me updates when my MP speaks, so I know he is addressing matters of concern to us and not just paying lip service to enquiries. Well done to all who have been involved in making the internet useful for eGov and opendata teams
    chris

  2. Good on you for commissioning the evaluations and especially for publishing them. There is nothing like drinking your own bathwater!
    Keep up the great work.

  3. Very comprehensive…more please (WDTK next!).

    ..but what we really want is a full analysis of the No.10 ePetition system ;)

  4. I think as good as you are, it’s very hypcoritcal of you not to allow someone to change their name on your site/s, it can take advantage of people who don’t understand libel law and other law and may be very harmed by using their own name. You either support freedom and anonymity (democracy and privacy) or you exploit vulnerable people who can’t protect themselves from the powers-that-be.

  5. Amazing.

    Definitely keen to see an evaluation of WDTK… feels like it’s missing one or two features (ability four journos to post up FOIs and data after they’ve filed their story, ability to co-create FOI requests with advice from others etc.) before it truly sings.

    rich

  6. great stuff, could you release the data from the users survey? I’m very interested in the fix my street data