If you’ve visited our parliamentary site TheyWorkForYou.com, you’ll have noticed that on each MP’s page there is a short summary of his or her voting record on various key issues.

These issues have always been carefully chosen to give a simple but neutral top-line view of each MP’s voting activity. Judging by Twitter, they’re a fairly popular part of the site, too.

There’s way, way more tedious complexity behind producing these little summaries than you might think, and due to a lack of appropriately skilled people in our team over the last year we had let our vote analyses get a bit behind the times. If you’re really interested you can read about why authoring these things in such a scrupulously balanced way is so time consuming here.

We’re posting today to tell you that we have recruited a pair of excellent new part-time voting analysts, David and Ambreen, and they have recently produced the first of a new generation of voting summaries.

The first shows how each MP has voted on increasing the rate of VAT, and second on the recent changes to university tuition fees. We have also increased the number of votes which feed into the EU integration policy to bring it more up to date.

To see this new data, just pop along to TheyWorkForYou’s home page, stick in your postcode, and check out your own MPs’ page. Then, if you want to be made aware as soon as we’ve published the next analyses, please follow our new TheyWorkForYou Twitter account.

Lastly, I just want to say thank you to the vote analysts Ambreen and David, to senior developer Matthew and to uber-volunteer Richard Taylor for kicking this vital part of TheyWorkForYou back into top gear.

Image by European Parliament.

As well as council elections and the referendum, the Scottish Parliament, Welsh Assembly, and Northern Ireland Assembly are holding elections this May. In Scotland and Northern Ireland, there are accompanying boundary changes, meaning this year you might be voting in a different constituency from last time.

To help people, as we’ve again had a few requests, our service from the 2010 general election is back, at http://www.theyworkforyou.com/boundaries/, just for the Scottish Parliament and Northern Ireland Assembly. Our generic lookup service MaPit also provides programmatic access to these results (technical footnote).

Alongside this service, we have refreshed our Scotland and Northern Ireland front pages, to slightly better display and access the wide array of information TheyWorkForYou holds for those devolved legislatures.

Sadly the Scottish Parliament changed the format of their Official Report in mid January and we haven’t been able to parse the debates from then until its dissolution this March – hopefully we’ll be able to fix that at some point, and apologies for the inconvenience in the meantime.

There don’t appear to be any central official lists of candidates in these elections. Amnesty.org.uk has a PDF of all candidates in Northern Ireland; David Boothroyd has a list of Scottish Parliament candidates. CAMRA appears to have lists for both Scotland and Wales. Those were simply found while searching for candidate lists, we obviously hold no position on those organisations 🙂

Technical footnote: To look up the new Scottish Parliament boundaries using MaPit, provide a URL query parameter of “generation=15” to the postcode lookup call. The Northern Ireland Assembly boundaries are aligning with the Parliamentary boundaries, so you can just perform a normal lookup and use the “WMC” result for the new boundary.

The two days leading up to election day are a hugely important time for less politically-obsessive voters. The parties know that a lot of people are only starting to seriously think how to vote today and tomorrow, and TheyWorkForYou saw its biggest spike ever the day before the election, way back in 2005.

This means it’s a super-important time to get trustworthy, non-partisan information in front of as many people as possible. And you can help by doing the following simple things:

1. Go to your constituency page on the TheyWorkForYou Election Quiz and take a good look at the answers. Is there anything surprising in the answers? Has anyone failed to respond who really shouldn’t? Is there anything funny in the responses? Make a couple of notes about what you think are the most interesting findings.

2. If you know the name of your local papers or radio stations, try to Google for the email or phone number of the news desk. If you don’t know the names, try sticking the name of your nearest town into a media database like this, to get a phone number or email address.

3. If possible, you should start your pitch by phoning rather than emailing. If you get a phone number for a news desk, give them a bell and say that you’re a volunteer from “The country’s largest non-partisan election information project”, and ask for the email of a specific person who might be interested in a story about what local candidates are saying.

4. Once you have an email address of a specific journalist, compose a locally specific email for them, along the following lines:

“Hi X,

I’m a resident of Z constituency, and this election I’ve been one of 6000 volunteers helping  to build an unprecedented project to get candidates across the country to go on the record, in conjunction with the website TheyWorkForYou.com. It’s a strictly non-partisan project, aimed at giving voters a really clear, spin-free view of what their candidates stand for. I’d really appreciate it if you could give it some coverage before election day.

In my constituency, N candidates have completed our survey. From this we can see some quite interesting things, namely:

* Candidate A thinks…

* Candidate B thinks…

Would you be so kind as to print a story encouraging people to check our their candidates via TheyWorkForYou.com, and mentioning some of the highlights I’ve included?

all the best,

Your name, email, phone”

5. An hour after you send the email through, give the journalist a call back to see if they need any more help.

6. If you do this, please leave us a comment on this post so we know who’s had a go!

Thank you for helping spread some non-partisan information this election time, and enjoy the election…

mySociety is lucky enough to have a number of small donors who give us monthly donations, normally ranging from £5-£20 (if you like our work and want to support us, please do join them!). Today we’re announcing a change to TheyWorkForYou which is supported by these donations.

One of the most popular features on TheyWorkForYou is the vote analysis – the bit that tells you that your MP “voted strongly against introducing a smoking ban” and so on. These voting analyses cut through a massive wall of parliamentary opacity whilst still allowing visitors to examine the details first hand. Despite each analysis resulting in just a single line on TheyWorkForYou, each one is rather time-consuming to construct, and TheyWorkForYou has not updated them as much as our users deserve.

Thanks to our small donors we’ve now been able to commission two part time researchers, Marcus Fergusson and Stephen Young, to help add new vote analyses more regularly. We’re pleased to say that we’ve just rolled out the first new policies, covering issues relating to schools, inquests and the House of Lords. We aim to add a couple of new vote analyses a month for the foreseeable future.

We take the business of authoring analyses that are scrupulously fair and neutrally worded extremely seriously. To this end we have replaced our previously ad-hoc approach with a newly instituted process designed to ensure the maximum rigour and balance, and to ensure we focus on issues which MPs thought were important even if they were not so well covered by the media.

TheyWorkForYou’s analysis of MP’s voting positions relies on The Public Whip, a project run by Julian Todd which tracks which way MPs vote.

The new process for analysing MP’s positions works like this:

1. A list of votes in the current Parliament, ordered with the highest turnout at the top, is taken as a starting point. The turnout figure used is corrected to account for party abstentions.
2. mySociety’s researchers work down that list writing explanations in easy to read terms describing what the vote was about; they also identify other related votes on the same issue and research those too.
3. A “policy position on the issue” is then chosen against which MP’s votes are compared to determine to what extent they agree or disagree with it. Policy positions are written to be intelligible and interesting to a wide range of users and in such a way that votes to change the status quo are ultimately described on TheyWorkForYou MP pages as votes “for” that change.

We hope you find these analyses useful. Thanks to Richard Taylor for his divisions list and help with this post.

Despite being strictly non-partisan, mySociety is still indisputibly a political entity that seeks to encourage new and better sorts of things to happen in our democracy. But ‘new’ and ‘better’ are vague words without a clear understanding of what democracy is, or what it could become.

Consequently, mySociety needs a theory of British democracy of the same robustness and quality as the code written by our developers. I am not as good a theorist as my team are programmers, but I will do my best to spell out where we see the weaknesses and opportunities for improvement in the current system.

What I am publishing here is explicity my view, not mySociety’s, and it is most definitely a first draft. I would be very grateful if you would be kind enough to leave comments containing your thoughts and suggestions, so that eventually (perhaps) it can be published as a proper essay or pamphlet and, with luck, enjoy the endorsement of the mySociety developers, trustees and volunteers.

Part 1. Democracy is about far more than voting

Once every four years a majority of all those elegible to vote in the UK make their way to schools and community centres to cast their ballots at the general election. These voters, picking from a range of options usually somewhat smaller than their supermarket choice of canned tuna, put their cross next to their preferred candidate and then make their way home.

The crosses that the voters leave on their ballot papers are painstakingly aggregated to determine the next occupents of 646 prized jobs; few of the jubilant winners understanding at the time what a seismic career mistake they have just made. But the individual constituency results are only the side-show to the main attraction. The real purpose of spending all that time and money on holding an election is to send two bits of information back to Parliament, the public and the Queen: who gets to govern, and how easily. When written down, the main output of the huge logistical exercise of a general election is astonishingly small: Lab 343, Con 196, Lib 63. Just 24 characters to dictate who gets to run the country, and how.

This seemingly esoteric observation highlights of the most remarkable things about democracies: despite the tiny amount of formal instruction given by the public to the political classes, democracies tend to be very different places compared to non-democracies. Clearly these 24 characters (bytes) are very important. With few exceptions democratic states come with a host of other institutions which are rare or unheard of in countries without them: fair courts, freedom of speech, freedom of association and so on.

These institutions haven’t just been created from scratch by democratically formed laws: they’re also the foundations upon which democracies have grown – they’re both the chicken and the egg. But why mention them here?

The answer is that these institutions attempt to provide guidance as to what government should actually do between elections. There are many millions of political decisions that need to be made in between our widely spaced elections, and the 24 bytes of instruction data generated at a general election just doesn’t contain enough detail. The non-electoral institutions of a democratic society are history’s solution to getting as many of those millions of decisions as possible right.

And what a bewildering array there are: from Quangos, to newspapers, to community groups, to industry lobbyists, to PTAs and NGOs to blogs. All ceasely pour out decisions that are vary from pure punditry to powerful statutory judgement.

Collectively these institutions and the decisions they make create a democratic culture which actually shapes our lives more frequently than bellweather issues at elections. Whether a school accepts our child, whether we get to add a skylight to our roof, and whether or not we feel afraid on our streets are determined by organisations that have almost never been subject to elections. And yet these institutions and the decisions they make are normally thought of as ‘democractic’, even though most of them are not formally elected. A community group of a dozen pensioners somehow seems far more democratic than Pravda, despite the latter being read by millions, and neither being elected.

The culture of democratic states isn’t some wishy washy backdrop against which real power politics happens, either. The institutions that shape our democratic culture often act as some of the hardest checks and balances against which politicians bang their heads. Just speak to a politician who has ever been foiled by a concerted media campaign to understand the power of democratic culture.

Where does mySociety fit into this? Well, we build tools that enable people to get better decisions made between elections. Whether that means writing direct to your politicians, or forming a campaign, or keeping politicians on their toes through publishing data on what they say and do, most of the things we build are about enabling our users to get the decisions made that they want to be made.

However, we’re far from alone in the arena of decision-influencing. The next section looks at how it has happened in the past, and how it may change in the future.

The meeting day voting application (vote often!) that we’ve been mentioning everywhere all week is a new departure for mySociety. In a frantic bid to catch up with the cool kids, it’s our first deployed Ruby on Rails application. This happened because Louise Crow, who kindly volunteered to make it (thanks Louise!), felt like learning Rails. We used to have a policy of using any language, as long as it was open source and began with the letter P (Python/Perl/PHP…). This has now been extended to the letter R!

You can browse the source code in our CVS repository. One interesting thing about Rails applications is that they are structured things, a deployable directory tree. So are mySociety applications.

For example, take a look at PledgeBank’s directory. It’s a mini, well defined filesystem – the ‘web’ directory is the meat of the stuff, but note also ‘web-admin’ for the administrator tools. Include files are tucked away in ‘perllib’ and ‘phplib’, while script files nestle under ‘bin’. We keep configuration files (analogous to the Windows Registry, or /etc on Unix) under ‘conf’. Database schema files live in ‘db’.

And a rails application is much the same. But much much much more detailed. Some of those are extra directories which we also have, but only when we deploy, not in CVS (for example, log files). All in all they are surprisingly similar structures, which shows we’re either both on the right lines, or both on the wrong false trail.

Like making Frankenstein’s monster, poor Louise and I had to graft these two beasts together just to deploy this small application. For example, we have a standard configuration file format which we read from Perl, Python and PHP. The deploy system does useful things with it like check all entries are present, and generate the file for any sandbox from a template. To get round this, there’s an evil script, possibly the first time PHP has been used to make YAML. (And please don’t look at the thing that makes symlinks.)

We could have extended Rails to be able to read its configuration from our file format, but that would be a lot more work. And we could have discovered how to hack its log file system to write to the mySociety log file directory. But everything is so coupled, it doesn’t ever seem worth it. Any Rails apps we deploy will just have to be an even more confusing mass of directories, application trees inside application trees.

Not content with building surprise election hit NotApathetic, mySociety’s keen band of volunteers have been busy recycling the code to produce IVotedForYouBecause.com. Just as NotApathetic was a platform for non-voters to explain their decision, IVotedForYouBecause.com is a place to explain the reasons why you voted for a particular candidate, and a place to make it clear what you do and don’t expect from them. In a few weeks we’ll coalate the responses by constituency, and use WriteToThem.com to mail them to the new MPs.

This morning I woke up quite late because I didn’t sleep well (and had been building Public Whip’s How To Vote). Today I’m not really working for mySociety, as I’m going to London this afternoon, but Tom had a couple of important bits for me to do. There was a bug in the admin page, and a missing link on the confirmation page. I fixed these up, improved the test suite a bit, and deployed to the main site.

PledgeBank is now ready for people to start using it in earnest. We’re still in testing, as we’re sure there’ll be lots of changes needed to it as it is used in the real world. But all the basic features, and fancier ones such as SMS and the auto-generated flyers, work. Tom’s just been on the radio, and he’s making lots of specific example pledges like this one about Shropshire. So, the hunt is now on.

My next jobs are to tidy up outstanding tickets which we have already fixed, and fix any bugs in there. The next feature we’re adding to the site is comments, so people can discuss the pledge.

Just as I was getting on with something else, I am called hither to write a blog post. So this’ll be a short one, I’m afraid. And instead of talking about what I was doing today (fixing bugs, modifying database schemas, and other dull interludes in the programming life) I’ll draw your attention to a couple of NotApathetic things.

NotApathetic has (notwithstanding various teething troubles) been going pretty well, and there are some more cool ideas in the pipeline. For those of you who are enjoying the latest web-logger fad, “tags“, Matthew Somerville has implemented this experimental page generated from users’ confessions reasons for not voting. It’s rather fun; since, unlike the “‘blogosphere”, mySociety isn’t about throwing away all the lessons of the last forty years of research in information retrieval, we generate tags automatically rather than expecting users to annotate their posts with the most meaningful set we can manage. (They wouldn’t.) Hopefully this or something like it will become part of the site’s front page soon.

And something which I want to implement — later this week, hopefully — is a voting system to collect two types of data: firstly, which posts people think are particularly interesting, so that we can give them more prominence on the front page; and secondly, given two example posts, how similar (in some general sense) the two reasons are. The point of the second one is that, given a set of similarity data, we should be able to cluster and categorise the country’s apathy and better understand it (and, of course, draw beautiful pictures of it in gnuplot, too). You might not think that sounds like fun, but I do.

So, we’ve launched Not Apathetic. If you aren’t voting, here’s your chance to say why. It’s a grouphug.us for political confessions. Much kudos to Sam, Matthew and Matthew who did all the work on the site.