Thanks to the work of thousands of volunteers across the country, we’ve now launched our survey of candidates to be your MP.

View the survey results

It tells you the views of candidates on a range of national and local issues. What’s particular exciting is that this is individual views – we separately surveyed all the candidates.

About 1/3rd of them have replied. The survey has a tool to let you ask the other candidates in your constituency to respond. Please give it a go, as we’d like to the survey to get as complete as possible over the weekend, to be most use to people in the last days leading up to the election.

Competition! Have you found an inventive way to ask your candidates to respond to the survey? Maybe you doorstepped them, or sent them a cake. Post your ideas and things you’ve done in the comments below.

As you may know, TheyWorkForYou are conducting a survey of candidates for Parliament.

Quite a few people have been asking how we worked out the questions. There are two parts to this, one local and one national.

Local questions

We used the power of volunteers.

Thousands of DemocracyClub members were asked to suggest local issues in there area. These were then edited by other volunteers, to have consistent grammar, and be worded as statements to agree/disagree with, and filtered to remove national issues. The full criteria and examples are available.

You can view the issues for any constituency on the DemocracyClub site. They are in the “local questions” tab.

We’ve ended up with local issues for about 85% of constituencies. They’re really interesting and high quality, and quite unique for a national survey.

Thank you to all the volunteers who helped make this happen!

National questions

This was hard, because we felt that asking more than 15 questions would make the survey too long. We also wanted to be sure it was non-partisan.

We convened a panel of judges, either from mySociety/Democracy Club or with professional experience in policy, and from across the political spectrum. They were:

• James Crabtree, chair of judges, trustee of mySociety, journalist for Prospect magazine
• Tim Green, Democracy club developer, Physics student, Cambridge University.
• Michael Hallsworth, senior researcher, Institute for Government.
• Will Davies, sociologist at University of Oxford, has worked for left of centre policy think tanks such as IPPR and Demos.
• Andrew Tucker, researcher at Birkbeck, worked for Liberal Democrats from 1996-2000.
• Robert McIlveen, research fellow, Environment and Energy unit at Policy Exchange, did PhD on Conservative party election strategy.

They met at the offices of the Institute for Government, and had a 3 hour judging session on 29th March 2010. They were asked to think of 8-15 questions, with multiple choice answers, which could usefully be answered both by members of the public and prospective candidates for national office.

To ensure maximum transparency, the discussions of the judges were recorded. You can download the recordings in two parts: part 1, part 2 (2 hours, 20 mins total).

Details of the broad framework the judges operated under are given by the chair of judges, James Crabtree, a trustee of mySociety, in the opening to the recordings.

Please do ask any questions in the comments below.

In January last year, at our yearly staff and volunteers retreat, we decided that TheyWorkForYou should do something special for the general election. We decided that we wanted to gather information on where every candidate in every seat stood on what most people would think were the biggest issues, not just nationally but locally too.

Our reasons for setting this ambitious goal were two fold. First, we thought that pinning people down to a survey that didn’t reward rhetorical flourishes would help the electorate cut through the spin that accompanies all elections. But even more important was to increase our ability to hold new MPs to account: we want users of TheyWorkForYou in the future to be able to see how Parliamentary voting records align with campaign statements.

This meant doing quite a lot of quite difficult things:

1. Working out who all the candidates are (thousands of them)
2. Working out how to contact them.
3. Gathering thousands of local issues from every corner of the country, and quality assuring them.
4. Developing a balanced set of national issues.
5. Sending the candidates surveys,  and chasing them up.

The Volunteer Army

This has turned out to be a massive operation, requiring  the creation of the independent Democracy Club set up by the amazing new volunteers Seb Bacon and Tim Green,  and an entire candidate database site YourNextMP, built by another new volunteer Edmund von der Burg.  Eventually we managed to get at least one local issue in over 80% of constituencies, aided by nearly 6000 new volunteers spread from Lands End to John O’Groats. There’s at least one volunteer in every constituency in Great Britain, and in all but three in Northern Ireland. Volunteers have done more than just submit issues, they’ve played our duck house game  to help gather thousands of email addresses, phone numbers, and postal addresses.

The Survey

What we ended up with is a candidate survey that is different for every constituency – 650 different surveys, in short. The survey always contains the same 15 national issues (chosen by a politically balanced panel held at the Institute for Government)  and then anything between zero and ten local issues. We’ve seen everything from cockle protection to subsidies for ferries raised – over 3000 local issues were submitted, before being painstakingly moderated, twice, by uber-volunteers checking for for spelling, grammar, obvious bias and straightforward interestingness (it isn’t really worth asking candidates if they are in favour of Good Things and against Bad Things).

In the last couple of days we’ve started to send out the first surveys – we’ve just passed 1000 emails, and there are at least 2000 still to be sent.

The Output

We’re aiming to release the data we are gathering on candidates positions on 30th April. We’ll build a nice interface to explore it, but we also hope that others will do something with what we are expecting to be quite a valuable dataset.

The Pressure

Candidates are busy people, so how do we get their attention? Happily, some candidates are choosing to answer the survey just because TheyWorkForYou has a well know brand in the political world, but this has limits.

The answer is that we are going to ask Democracy Club, and it’s army of volunteers to help. We’ll shortly roll out a tool that will tell volunteers which of their candidates haven’t taken the opportunity to go on the record , and provide a range of ways for them to push for their candidates to fill it in.

It would be a lie to say we’re confident we’ll get every last candidate. But we are confident we can make sure that no candidate can claim they didn’t see, or didn’t know it was important to their constituents. And every extra voice we have makes that more likely.

Join Democracy Club today

On Monday the 12th of April 2010 Parliament was formally dissolved in advance of the forthcoming general election. This has prompted some interesting, and rather bizarre, responses to the Freedom of Information requests which have been made to the House of Commons and House of Lords since dissolution. Each such request made via mySociety’s Freedom of Information website WhatDoTheyKnow.com has received a standard reply stating:

When Parliament has been dissolved there is no ‘House of Commons/Lords’ for the purposes of Part 1 of Schedule 1 to the 2000 Act, and there is therefore no ‘public authority’ to which the 20 day deadline under section 10 of the 2000 Act is capable of applying. The time limits do not, therefore, apply during the period of Dissolution.

The effect of the 2000 Act, including its time limits, resumes when the new House of Commons/Lords first meets.

Requests which were due to receive a response during the period parliament is dissolved have had their “clocks” stopped, with notices saying:

As your request was received before the House dissolved, the 20 working day time limit of your request will be split, ceasing on 12 April 2010 and resuming on 18 May 2010 when the new Parliament first meets.

As mySociety’s Freedom of Information website WhatDoTheyKnow automatically publishes requests, correspondence and responses online it’s not just the requestors who can see those responses, anyone can.

During the 2005 election according to the UK FOI blog Parliament placed a notice on its website saying it had consulted with the Information Commissioner and agreed the procedure for extending the time limit for a response.

Clearly the Houses of Parliament still have staff employed and people are still acknowledging the FOI requests. While both houses have stopped meeting, the institutions behind them must surely still be operating, and to claim they have ceased to exist is bordering on the utterly ludicrous.

Are the staff who would otherwise be in a position to respond to requests for things like viewer statistics for the Parliament Live TV stream, content of the Commons’ Intranet or cost of the Parliament Education Service not at their desks at the moment? If they are who’s employing them? Who’s paying them? While it is presumably a busy time for those staff preparing for a new intake of MPs; you might think that without MPs and Lords around it may be a quite time for many staff who might want to use the opportunity to catch up with correspondence like FOI requests. Perhaps in the midst of all this rather British oddness we should be happy that at least the parliament website hasn’t been turned off at this time of peak interest in the nation about parliament and our democratic system.

WhatDoTheyKnow already has to be aware of public holidays and follows some rather complex rules when it comes to calculating deadlines for responses however we have decided against updating the system to deal with this new and unexpected situation. We don’t think it is right that the institution of Parliament should consider itsself not to exist during an election period. We still be marking FOI requests as “late” when the twenty working day statutory time limit has expired regardless of the odd stance being taken by Parliamentary officials.

mySociety is lucky enough to have a number of small donors who give us monthly donations, normally ranging from £5-£20 (if you like our work and want to support us, please do join them!). Today we’re announcing a change to TheyWorkForYou which is supported by these donations.

One of the most popular features on TheyWorkForYou is the vote analysis – the bit that tells you that your MP “voted strongly against introducing a smoking ban” and so on. These voting analyses cut through a massive wall of parliamentary opacity whilst still allowing visitors to examine the details first hand. Despite each analysis resulting in just a single line on TheyWorkForYou, each one is rather time-consuming to construct, and TheyWorkForYou has not updated them as much as our users deserve.

Thanks to our small donors we’ve now been able to commission two part time researchers, Marcus Fergusson and Stephen Young, to help add new vote analyses more regularly. We’re pleased to say that we’ve just rolled out the first new policies, covering issues relating to schools, inquests and the House of Lords. We aim to add a couple of new vote analyses a month for the foreseeable future.

We take the business of authoring analyses that are scrupulously fair and neutrally worded extremely seriously. To this end we have replaced our previously ad-hoc approach with a newly instituted process designed to ensure the maximum rigour and balance, and to ensure we focus on issues which MPs thought were important even if they were not so well covered by the media.

TheyWorkForYou’s analysis of MP’s voting positions relies on The Public Whip, a project run by Julian Todd which tracks which way MPs vote.

The new process for analysing MP’s positions works like this:

1. A list of votes in the current Parliament, ordered with the highest turnout at the top, is taken as a starting point. The turnout figure used is corrected to account for party abstentions.
2. mySociety’s researchers work down that list writing explanations in easy to read terms describing what the vote was about; they also identify other related votes on the same issue and research those too.
3. A “policy position on the issue” is then chosen against which MP’s votes are compared to determine to what extent they agree or disagree with it. Policy positions are written to be intelligible and interesting to a wide range of users and in such a way that votes to change the status quo are ultimately described on TheyWorkForYou MP pages as votes “for” that change.

We hope you find these analyses useful. Thanks to Richard Taylor for his divisions list and help with this post.

In my last blog post, I explained the new service TheyWorkForYou offers to show you what constituency you will be in at the next general election. Now I’m going to show you why you shouldn’t use anything else.

The defintions of the boundaries for the forthcoming constituencies in England were originally published in The Parliamentary Constituencies (England) Order 2007 (SI 2007/1681), based on ward boundaries as they were on 12th April 2005. However, due to some local government changes since that date, The Parliamentary Constituencies (England) (Amendment) Order 2009 (SI 2009/698) was published changing the boundaries for four constituencies – Daventry, South Northamptonshire, Wells, and Somerton & Frome – to be based on the new council wards as they were on 3rd May 2007.

The forthcoming constituencies in Northern Ireland were defined in The Parliamentary Constituencies (Northern Ireland) Order 2008 (SI 2008/1486). In this, Derryaghy ward was split between two constituencies – Belfast West is given “that part of Derryaghy ward lying to the north of the Derryaghy and Lagmore townland boundary.”

All of which means that other sites that try to tell you what constituency you will be in at the election invariably get it wrong.

Both Labour and the Conservatives say that BA6 8NJ is in Wells at the next election, when it will be in Somerton & Frome. Both say that NN12 8NF will be in Daventry, when it will be in South Northamptonshire. I assume that both sites are using boundary data predating the Amendment Order from March 2009. The Conservatives also say that BT17 0XD will be in Lagan Valley when it will be in Belfast West; Labour simply say “Northern Ireland” for any Northern Irish postcode you provide.

The Liberal Democrats site currently returns no results for any postcode, which I assume is a bug 🙂

The /current/ boundary between Belfast West and Lagan Valley. (Image produced from the Ordnance Survey electionmap service. Image reproduced with permission of Ordnance Survey and Land and Property Services)

The official election-maps.co.uk service (from where TheyWorkForYou gets its boundary maps) returns the correct results for BA6 8NJ and NN12 8NF, but doesn’t have future boundaries for Northern Ireland. It’s not clear that it doesn’t, as searching for Lagan Valley with “future boundaries” selected returns a result, but that result is the current boundary. This can be seen from the picture on the right – as is clear from the quote I gave above, everything within Derryaghy ward north of the Lagmore/Derryaghy boundary will be in Belfast West at the next election. Plus the site doesn’t work without JavaScript.

TheyWorkForYou’s “constituency at the next election” service gives BA6 8NJ in Somerton & Frome, NN12 8NF in South Northamptonshire, and BT17 0XD in Belfast West. There is enough confusion with the changes to boundaries for everywhere except Scotland, that it is somewhat frustrating to have it compounded by sites giving incorrect information. The lack of any official service also doesn’t help.

Over the last weekend of November 2009 a group of 21 mySociety staff, volunteers and trustees went to a house outside of Bristol to wrestle with the question of what mySociety should build over the next 12 months. This was the fourth time we’ve done it, and these meetings have become a crucial part of our planning.  This year, we were talking not just about what new features to add to our current sites, but also about the possibility of building an entirely new website for the first time in a couple of years. The discussions were lively and passionate because we know we have a lot to live up to: not only is our last major new site (WhatDoTheyKnow) likely to cross the 1 million unique visitors threshold this year, but we understood that there were people and organisations who weren’t there who would be counting on us to set the bar high.

A chunk of the weekend involved vetting the 227 project ideas that were proposed via our Call for Proposals. I’m going to write a separate post on our thoughts about that process, but if you look at the list below you may spot things that were submitted in that call.

One nice innovation that helped us whittle down our ideas from unmanageable to manageable numbers was a pairwise comparison game to help us prioritise ideas, build custom for the occasion by the wonderful and statistically talented Mark Longair.  In other words, we used the technique that powers  KittenWar.com to help decide our key strategic priorities for the next year: after all , if we don’t, who will?

Screenshot from the pairwise comparison game that Mark Longair coded

By the end of the weekend we had not battened everything down – there are too many uncertainties around how much time we will have, and some key ideas that need more speccing.  However, we were able to put various things into different buckets, marked according to size and degree of certainty.  So here goes:

1. Things which were decided at the last retreat, which we are definitely building,  and which (mostly) need doing before next year’s stuff starts getting built

• A top level page for each bill on TheyWorkForYou
• Future business (ie the calendar) for events in the House of Commons, including a full set of alerting options.
• Video clips on MP pages on TheyWorkForYou
• Epicly ambitious election data gathering and quiz building with the lovely volunteers at DemocracyClub

2. Small new things that we are very probably doing because there was lots of consensus

• Publish a standard that councils can use to post problems like potholes in their databases to FixMyStreet and other similiar sites.
• Template requests in WhatDoTheyKnow so that users are strongly encouraged to put in requests that are well structured.
• After the next general election, email new MPs with various bits of info of interest to them including their new login to HearFromYourMP, their page on TheyWorkForYou, explanation of how WriteToThem protects them from spam and abuse, a double check that their contact details are correct, and a introduction to the fact that we record their correspondance responsiveness and voting records.
• Add to WhatDoTheyKnow descriptions about what kind of public authority a specific entity is (ie ‘school’, ‘council’) and the information they are likely to hold if FOIed.
• Show divisions (parliamentary votes) properly on debate pages on TheyWorkForYou, ie show the results of a vote on the same page as the debate where the issue was discussed, with full party breakdowns on each division.
• Add “How to benefit from this site” page on TheyWorkForYou, inspired by OpenCongress.org
• Help Google index TheyWorkForYou faster by creating a sitemap.xml file that is dynamically updated.
• Using the data we expect to have from DemocracyClub’s volunteers, send a press release about every new MP and to all relevent local newspapers
• Incorporate a council GeoRSS problem feed into FMS

3. Slighty more time consuming things we are very probably doing because there was lots of consensus

• 1 day per month developer time that customer support guru Debbie Kerr gets to allocate as she see fit.
• Premium account feature on WhatDoTheyKnow to hide requests so that journalists and bloggers can still get scoops and then share their correspondance later.
• Add Select Committees to TheyWorkForYou, including email alerts on calls for evidence.
• Take professional advice on how to handle PR around the election

4. Much more time consuming things and things around which there is less consensus. NB – We do not currently have the resources to do everything on this list next year – it is an ambitious target list.

• Primary New site: TBA in a new post
• Add a new queue feature to WhatDoTheyKnow so that users can write requests, then table them for comments from other users and expert volunteers before they are sent to the public authority
• Relaunch our Volunteer tasks page on our sites, keep it populated with new tasks, specifically allocate resources to handhold potential volunteers. Allocate time to see if any of the ideas that we didn’t build could be parcelled into volunteer tasks.
• Secondary New site (if we have a lot more time than we expect): Exploit extraordinary richness of Audit Comission local government target data in a TheyWorkForYou-like fashion.
• FixMyStreet to become international with  a) maps for most of the world b)  easy to follow instructions explaining how to supply mySociety with the required data to us to enable us to turn on FixMyStreet in non UK countries or areas. This data would includ  ie gettext powered text translation files,  shapefiles of administrative boundaries, and lists of contact data.
• Add votes and proceedings to TheyWorkForYou (where they reveal statutory instrument titles that are not debated but where the law gets changed anyway)
• Carry out usability testing on TheyWorkForYou with then help of volunteer Joe Lanman – then implement changes recommended during a development process taking up to 10 days.
• Add to TheyWorkForYou questions that have been tabled in the house of commons but which haven’t been answered yet.
• Add a new interface for just councils so that they can say if a problem on FixMyStreet has changed status.

Phew. And that’s not even counting the projects we hope to help with in Central and Eastern Europe, our substantial commercial work, or the primary new site idea, which will be blogged in Part 2.

Many MPs and Lords use the Freedom of Information Act to obtain information from public authorities despite the fact they are able to table parliamentary questions. Occasionally they make their requests via mySociety’s freedom of information website WhatDoTheyKnow.com which ensures both the request, and its response, are freely available online. Surprisingly the freedom of information route can result in the release of more, and better quality, information than a written Parliamentary Question.

For example on the 12th of November 2009 Eleanor Laing the Conservative Shadow Minister for Justice submitted the following written Parliamentary Question:

To ask the Secretary of State for Health how many staff in his Department were employed on the management of freedom of information requests submitted to his Department in each year since 2005; and how much his Department spent on the management of such requests in each such year.

The response contained the number of staff per year as requested but with respect to the spending the parliamentary response stated: “The information requested on expenditure could be provided only at disproportionate cost.”

A very similar request for information had been made many months previously, in July, by WhatDoTheyKnow.com user and FOI campaigner Heather Brooke. The response to the FOI request contained more information, and more precise information, than Eleanor Laing had obtained via her parliamentary question. When the request was made via WhatDoTheyKnow how much staff substantially involved in answering requests were paid was disclosed, in detail.

While the costs of complying with a particular request are capped by regulations under the Freedom of Information Act, data on total costs of FOI compliance such as that released by this request allows the average costs of dealing with a request to be calculated.

MPs using WhatDoTheyKnow

• Stephen Crabb MP
• Lindsay Roy MP
• Tom Watson MP

Do let us know in the comments if you’ve spotted any more!

Parliamentary boundary changes appear to be a source of confusion to many people and organisations. The facts are quite simple – parliamentary boundary changes, proposed by the various Boundary Commissions, do not take effect until the next general election. Until then, your MP remains whoever they have been, no matter what literature you may get through your letter box, or what anyone may tell you.

As one example, take Birmingham City Council. Their page on constituencies and wards correctly states that Birmingham is divided into eleven parliamentary constituencies, but then goes on to list only ten – they are listing the new constituencies which do not yet exist, as Birmingham is losing one constituency at the next election. It appears that they have organised themselves along the new boundaries in advance – which is fine, but this doesn’t affect current Parliamentary representation, and so they should explain this clearly, as otherwise members of the public get confused (and blame us for giving them the “wrong” MP, when we haven’t done so). As you can see from the maps above (which highlight Birmingham, Hall Green), the constituencies will be changing their boundaries quite a bit, and we have had reports of people receiving letters from candidates in the next election who are MPs of different neighbouring constituencies, simply referring to themselves as an MP, which is a great source of confusion.

An inhabitant of St Josephs Avenue, Birmingham (behind the Royal Orthopaedic Hospital), which is currently within the Selly Oak parliamentary constituency (red), and the Northfield ward of Birmingham City Council (green), would, on looking at Birmingham City Council’s website, assume they’re in a parliamentary constituency called Northfield. Northfield is currently the constituency to the west of Selly Oak; at the next election, its boundary with Selly Oak will change to the blue line, at which point St Josephs Avenue will be in the Northfield constituency. But not until then.

As another example (chosen purely as it has come up in user support), the Labour candidate’s website for Streatham had a page about the constituency – obviously you would expect a candidate to be talking about the future constituency, but would it hurt to add some explanation that Streatham is currently a slightly different shape?

Boundaries of different things are all independent – if a ward boundary moves due to some local issue, the corresponding Parliamentary boundary does not necessarily change with it (probably not, in fact). So when Birmingham changed its ward boundaries back in 2003, they became out of sync with the Parliamentary constituencies. From the next election, things will be more in sync as the new Parliamentary boundaries are based on more recent ward boundaries, but this will again separate over time. All we can do is always clearly explain the current situation, and ask that others do the same.

Hansard's Parliamentary Debates by hugovk (cc)

The House of Commons debate coverage on TheyWorkForYou has recently extended back from the 2001 general election to the 1935 general election, and our knowledge of MPs now extends back to the start of the 19th century. This means TheyWorkForYou now includes things such as Anthony Eden on the Suez Canal in 1956, saying “there was not foreknowledge that Israel would attack Egypt“; the debate the day after Bloody Sunday in 1972; Geoffrey Howe’s resignation statement in 1990; Neville Chamberlain on the eve and start of the second World War in 1939; and Winston Churchill‘s speeches to the House, such as We shall fight on the beaches and This was their finest hour in 1940. This and much, much more are available and searchable using our new improved advanced search, which allows you to filter by e.g. date range or person. We hope people enjoy researching this huge wealth of information (I certainly do), and add useful annotations to the text to help other people.

This would not have been possible without the original project by Parliament to digitise historical copies of Hansard and make them available, nor the internal Parliamentary project to clean up the data, match up speaker names, and so on. The project was kindly funded by the Ministry of Justice’s Innovation Fund, which also supported the creation of FixMyStreet and GroupsNearYou.