Scottish Parliamentary proceedings are now back on TheyWorkForYou.
Back in August 2014, the Scottish Parliament changed the way it published the Official Report of its debates.
TheyWorkForYou works by fetching data from various parliamentary sources—and in this case, unfortunately, the change at the Scottish Parliament end meant that our code no longer worked. We replaced our ‘debates’ section with an apologetic note.
Well, thanks to the Scottish Parliament kindly republishing the data in almost the format we used to use, we’ve managed to make some small tweaks and restore that content—including debates from the previously missing period. If you’re subscribed to alerts, you should have received an email digest with links to the backdated content (always supposing there was any that matched your chosen keywords).
And if you’re not subscribed to alerts? Now is a great time to rectify that. We’ll send you an email every time your chosen word or phrase is mentioned in Parliament, or every time your chosen representative speaks.
While we were doing this work, we also modified TheyWorkForYou so that it now pulls in ministerial data from the Scottish Parliament API. This is a welcome time-saver for us: previously we were creating a list manually from the official PDFs, while we can now automatically fetch it and reformat it into Popolo JSON, meaning it’s consistent with all our other data.
Thanks for your patience; we know that many people were awaiting this repair, and for longer than we would have liked. Enjoy!
Back in November 2013, we asked you what improvements you’d like to see on TheyWorkForYou.
One answer dominated: you wanted more information about how MPs vote.
Adding information on voting has been the single biggest project on the site since its launch, and has required several different phases of development. We announced each of these as it happened, but now that we’re at the end of this large piece of work, it seems like a good time for a complete overview.
So let’s take a look at exactly what it has involved—and, more importantly, what it means for you.
We’ll start with a rundown of features, then go into more detail about how they are created at the end of the post, for those who are interested.
What vote information means for you
1. You can easily see how your MP voted
Just how much do you know about how your MP voted on the stuff that matters? Most of us would have a hard time keeping up with every vote, simply because it isn’t information that’s widely publicised.
On TheyWorkForYou, you can see a run-down of how any MP has voted on key policies, by visiting their page on the site and clicking the ‘voting record’ tab (see image, above). We’ve created summaries of their stance on all kinds of matters, including the EU, same-sex marriage, NHS reform and a lot more.
Each of these summaries is compiled from every vote the MP has made on a motion that impacts on that policy.
You can click ‘show votes’ (see image above) to see the specific votes that go to make up any particular stance, and we’ve laid them all out in plain English so that it’s easy to grasp exactly what the issue is.
And from there you can click through to the website Public Whip, where you can explore votes in more detail, including lists of who voted for or against any given motion.
2. You can find out how strongly your MP feels
When we first presented voting information, we said that an MP had voted ‘strongly for’ or ‘moderately against’ certain policies, which led to quite a large postbag from people asking, “How can you vote strongly, surely you either vote for or against?”.
We wrote in the second half of this blog post about the wording changes we made to clarify the fact that these stances are calculated from a number of votes.
3. You can assess if your MP is a sheep or a lone wolf
We’ve pulled out all the votes which differ substantially from the way that the majority of each MP’s party voted. If your MP has voted against the flow, you’ll see something like this on their page:
Why do we highlight this type of vote? Because we think they’re a really good indication of where an MP feels strongly enough about something to risk sticking their neck out. It’s also a great way to check the truth when people say, “MPs? They’re all the same”.
4. You can understand the background to the votes
Generally speaking, there’s a debate before any vote takes place in Parliament, covering all the matters which may be topmost in MPs’ minds before they cast their lot.
Clicking on the ‘show full debate’ link from the topic pages (see image above) will give you the full context.
How we compile vote information
If that all seems nice and simple, well, great! That was our aim.
Putting it all together definitely wasn’t so simple, though. Voting information has never been previously presented all in one place in quite this way before—on TheyWorkForYou or anywhere else, to the best of our knowledge—so we had to figure out how to import the data and how best to display it.
As with much of our work, it’s a mixture of manual graft and automating whatever we can. Some things, like rewriting votes so that everyone can understand them, can’t be done by a computer. But many of our users are surprised to learn just how much of what we publish out is untouched by human hand.
Our Developer Struan, who did the most recent round of work on the voting records, said:
We get all our voting data from PublicWhip, a site set up by Francis Irving (once of mySociety) and Julian Todd. Public Whip takes the data we [TheyWorkForYou] produce from Hansard and extracts only the information on votes (or divisions in Parliamentary jargon) that take place in Parliament. It then allows you to look up how an MP or a Lord voted.
Let’s just think about that for a moment. We’re looking at a process where Parliament publishes Hansard, TheyWorkForYou scrapes the data and re-presents it, Public Whip extracts the voting information and presents that, and TheyWorkForYou takes that voting information back for its own voting pages. Simple…
One of the first things we did was to ‘translate’ the votes into plain English, so that it was very clear what was being voted for or against— and if you want to read more about that process, we talked about it in a blog post back in July 2014.
That allowed us to move to the next phase, as Struan explains:
Public Whip groups related votes together into policies, e.g renewing Trident, so you can see how an MP voted on the policy as a whole.
It does this by saying which way an MP would have to vote each of the divisions in the policy if they agreed with the policy. It then takes the MP’s votes on each division in the policy and assigns a score to it based on how they voted. These scores are then added up and compared to the score they would get if they always voted in agreement with the policy. The closer the MP’s score is to the score of an MP who always voted in agreement with the policy, the more they agree with the policy.
Thanks to Public Whip’s grouping, we were able to start compiling our MPs’ voting records along those same policy lines.
One of the most fiddly parts of the process was figuring out how to ensure that the information we present is a true, non-biased representation of the MP’s intentions. You might think that a vote is quite a simple matter – it’s either a yes or a no for a particular motion. But as soon as we started displaying votes within a policy, things got a bit trickier.
Some divisions in a policy can be marked as important and voting with the policy in those divisions is worth more points. This is to prevent voting in agreement on a set of minor votes, e.g “Parliament will commission a report on the future of Trident”, outweighing voting against something important, e.g. “Renew Trident”. It also reflects the way Parliament works, often with several smaller votes on parts of a bill and then a vote on the bill as a whole.
For clarity I should point out here that sometimes voting no in a division is a vote for the policy, e.g voting no in a “This house believes Trident should not be renewed” division would clearly be a vote for our example “Renew Trident” policy.
This approach also helps where one vote straddles several topics: for example, consider a vote against the Budget when the Budget contains many proposals including, say, the capping of VAT. It’s quite possible that an MP may be for the capping of VAT but broadly against several other motions covered by the Budget, and so decides to vote against it on balance. So long as we mark the Budget vote as a weak vote for the capping of VAT, its significance should be properly accounted for.
Where we don’t have enough information to show a stance, for example where an MP never voted on the topic, is too new to have had a chance to vote on the topic, or all their votes on the topic have been labelled as ‘weak’, we say so:
A final little subtlety is the difference between “Never voted” on a policy and votes where the MP was absent. If it says an MP has never voted on a policy that means they were elected after all the divisions in the policy took place so did not have a chance to vote on them. Absent means they could have voted in the divisions but did not.
Absent votes count towards your score but at half the rate of voting in agreement with the policy. This is so that an MP who votes in agreement with the policy in one division and then misses all the other divisions shows as agreeing with the policy rather than against as it would if no score was assigned to absent votes. That does currently mean that if they were always absent it shows, slightly unhelpfully, as “a mixture of for and against”.
It’s not an ideal system as it does produce some odd results occasionally but it mostly works.
To show where an MP has voted against the majority of their party, we have to figure out a similar score across the party as a whole.
This is exactly the same process, only we add up all the votes by all the MPs but the maths is pretty much the same.
All in a day’s work
As mentioned at the top of this post, vote information was our most-requested addition. And rightly so! Our MPs represent us, so naturally we want to see their track records, quickly and easily.
If you’re not an expert, you might not have known how to find this information before. And that’s essentially what TheyWorkForYou aims to do: make the workings of Parliament more accessible for everyone.
Parliamentary copyright images are reproduced with the permission of Parliament.
For a while now, TheyWorkForYou has shown how your MP voted on key topics.
What it hasn’t done, until this week, is give a crucial piece of context. That is, how do your MP’s votes differ from those of their colleagues in the same party?
We all know that, on many issues, the whip ensures that MPs vote according to the party line rather than their own convictions. So in theory, by examining the votes which diverge from the majority party vote, we might get the clearest picture of what an MP truly cares about.
And now, we’ve added a small piece of code to the site, which allows us to do just that. At the top of your MP’s page, you’ll now see text along these lines:
If your MP never disagrees with their party, you’ll just see the top line followed by a random selection of votes.
The importance of wording
The screenshot above shows another small change we’ve made to TheyWorkForYou: just a matter of wording, this time.
When we first started displaying how MPs had voted, we used terms such as “voted strongly for”, “voted moderately against”, etc. This was to allow us to represent a range of positions along a spectrum for each topic.
For every topic, such as EU Integration, or smoking bans, several different votes are analysed. The ‘show votes’ button, as seen above, takes you to a page where these are listed.
However, we received a steady stream of emails, tweets and Facebook messages asking how an MP can vote ‘strongly’ or ‘moderately’ for something. To a fly-by reader, it seemed nonsensical, because of course they were thinking of that fact that MPs vote for or against a single motion.
To counteract this, we’ve used words which we hope encapsulate the concept of a series of votes over time – words like ‘consistently’, ‘occasionally’ and ‘never’.
Choosing these words proved to be harder than we’d anticipated, and, after a long heated discussion between colleagues, resulted in a straw poll asking anyone we could find to arrange pieces of paper in a line to indicate how they perceived their strength.
We finally came up with an answer that the majority agreed on—and we haven’t had any mail on the subject since then. Let’s cautiously call that a win for careful wording.
Visit TheyWorkForYou’s homepage today, and you’ll see big changes.
These pages being the most visited, it made sense – but it did leave us with a shop window that did no justice to the goods within.
High time for a new front page
TheyWorkForYou’s homepage had not changed all that much since the site began in 2004: as new content such as committees or the devolved parliaments were added in, they simply got squeezed in wherever they would fit.
This is the first time in a good few years that we’ve taken a step back, started again with a blank canvas and prioritised what’s important.
Simpler, leaner, better-looking
It tells you what the site is for
Consistently, a good 60% of our users are first-time visitors to the site, so we need to make it very clear exactly what we do, and why they should care.
It helps you find your MP’s information
78% of the UK population don’t know the name of their MP (presumably that’ll be even higher for a while, post-election!). That’s why a postcode box, which matches you to your MP—and not a search box—is the most prominent input on the page.
It highlights current affairs
We know from experience that a large proportion of our users’ searches are based around issues that have just hit the headlines, whether that’s the latest budget, a ding-dong at Prime Minister’s questions, or a big news story.
It’s not always obvious to a casual observer where to find the relevant debates: for example, in Hansard (the official record of Parliament, which is where our content comes from) the budget is called the ‘Autumn Statement’, while Prime Minister’s Questions is labelled ‘Engagements’.
So now we have space to signpost the content that most visitors are likely to be looking for.
It encourages you to subscribe to activity
In a secondary but still prominent position, we signal that you can sign up for email alerts whenever your chosen MP speaks or your chosen keyword is mentioned. We hope that this will encourage still more people to engage on the things that matter to them.
It offers other paths in to content
If you’re just browsing, there’s still plenty of chance to see what’s new. Recent activity from Parliament is showcased on the lower half of the page, or you can riffle through all the different parliaments and types of debate in the top menu.
A fitting gateway
TheyWorkForYou still has all the same content, but now it has a homepage to be proud of, too.
That homepage is still a gateway into rich data: an archive of searchable, shareable, readable debates going back to the 1930s, profiles and voting records for MPs and Lords both current and historic, and the calendar of upcoming events.
With this new design, though, it should all be much easier to find. We hope you like it.
How many people visit mySociety’s websites?
That’s a question we don’t ask ourselves as much as many other organisations. Much of our current funding is dependent on transactions (that is, the number of people using the site to complete an action such as making an FOI request, writing to a politician, or signing up to receive emails when their MP speaks), and rightly so, since that is a better measure of the sites’ actual effectiveness.
All the same, visitor numbers* do tell us about things like how much public awareness there is of what we do, and which of our sites is more visible than the others, so it’s good to take a proper look now and again.
Which of our UK sites is most visited?
By far our most popular site in terms of visitor numbers is our Freedom of Information site WhatDoTheyKnow. With over 4.5 million visitors 2014-15, it’s had three times more users than its closest competitor, TheyWorkForYou.
As well as allowing users to submit FOI requests, WhatDoTheyKnow also puts the responses into the public domain, so that the information becomes openly available. Every request receives, on average, twenty readers, meaning that transactions do not show the whole picture for this site.
WhatDoTheyKnow’s user numbers are also rising steadily. It’s up 8% on last year, and March 2015 was its highest month for unique users since its launch in 2008, at 470,509.
Which is least visited?
This dubious honour goes to WriteToThem, which nonetheless welcomed 457,209 visitors during the year, either helping them to write to their representatives, or simply showing them who those representatives were.
This was still a decent 11% rise on the previous year, despite a real rollercoaster where some months dipped substantially from the previous year.
Which made the most gains in the last year?
FixMyStreet saw the biggest percentage change, with a 21% rise in visitor numbers compared to the previous year; we talked a bit more about that in a recent blog post. WhatDoTheyKnow had the highest rise in actual visitor numbers: over 360,000 up on 2013-14.
Which fell by the most in the last year?
TheyWorkForYou saw a 12% drop in visitor numbers year on year (and also the biggest drop in real terms)—disappointing, but something we hope to rectify with the new voting pages, an ongoing process of rolling redesign, and some grassroots outreach.
How much effect do external events have on visitor numbers?
We already know that, as you’d expect, when Parliament is on holiday, MPs, debates and legislation aren’t in the news, and TheyWorkForYou visitor numbers fall. There’s also a weekly pattern for all our sites, where far fewer people use them at the weekends, presumably indicating that lots of our users access them from work.
It’s too early to say exactly what effect the election has had on our sites: as I write, people are eagerly checking out the voting records of newly-appointed cabinet ministers on TheyWorkForYou.
One thing we know for sure is that fewer people will have been using WriteToThem, because there have been no MPs to write to for the last few weeks. We’ve removed the “write to your MP” links from TheyWorkForYou, which always drove a good deal of WriteToThem’s traffic.
FixMyStreet enjoyed a boost back in June, when it was featured on the Channel 4 programme ‘The Complainers’—and the nice thing is, user numbers never receded back to their previous levels after the programme was over. Maybe people just need to use FixMyStreet to see how useful it is.
How many people visit mySociety’s UK websites in total?
This is a difficult figure for us to produce with accuracy, because we don’t trace whether you’re the same person visiting a number of our different sites.
However, the aggregate total of visitors to all our UK sites (WriteToThem, TheyWorkForYou, FixMyStreet and WhatDoTheyKnow) for 2014-15 is 6,983,028. Thanks very much if you were one of them 🙂
How can I help?
Glad you asked! If you find mySociety sites useful, you can help us spread the word by telling friends, sharing the URLs with any groups you are a member of, posting on Facebook or Twitter, or writing to your local paper.
We have a number of materials for FixMyStreet which can be found here; we hope to create similar materials for our other sites too, and we’ll make sure we announce it on here when we do.
* Note: all references to ‘users’ refer to unique users within the period discussed. So, users in a year means individual people who may have visited any number of times over that year, but are only counted once; same with monthly users.
So, the results are in. Some of us have a brand new MP. Others will see the same familiar face returning to the benches of Westminster.
Either way, the important questions remain the same:
- What will your MP do in Parliament?
- Will they speak about the things that matter to you?
- How will they vote in your name?
The easy way to keep up
TheyWorkForYou.com makes it very easy to keep check: you can even sign up to receive an email whenever your MP speaks. These are in the form of a daily digest, and we only send them on days when your MP has actually contributed to a debate.
It’s the low-effort way to see exactly what your MP is getting up to, with no spin, just the facts. Click here for our easy sign-up.
If you already receive alerts, but your prior MP has lost their seat, be sure to set up an alert for the new one now. We’ll be sending reminders to all current subscribers.
There’s no need to cancel the previous alert, however: if your old MP isn’t in Parliament, we simply won’t be sending any more emails about them.
Spare a thought for us over the night of May 7th – for, when the nation wakes up to the General Election results, we’ll have been up all night updating TheyWorkForYou.
As you might imagine, elections are bitter-sweet times for us here at mySociety. On the one hand, swingometers, marginals and ballot boxes are about as exciting as life gets for a bunch of political geeks. On the other, we have only a short window of time in which to ensure our parliamentary websites reflect the new administration.
In previous years, this has meant manually updating an XML file and running an import script 650 times – slightly arduous, even for the most dedicated civic coder. This year, we’re taking advantage of the fact that YourNextMP exists and several of us will be staying up anyway to see the results, and hoping to do things a little differently.
As each result is announced (or potentially even earlier, if it’s clear that there’s only one possible winner), site administrators will be logged in to YourNextMP, where they’ll have access to a “this person won!” button. We’ll be on a rota throughout the night, sharing duties with the equally dedicated Democracy Club volunteer team.
When that button is clicked, YourNextMP will update, and TheyWorkForYou will notice and automatically update its underlying JSON data.
This is the data we match you with when you input your postcode on the homepage, meaning that TheyWorkForYou should be a great place to find out who your next MP is as soon as you wake up (assuming the results are in) on 8th May.
UPDATE: If you are interested in the technical aspects of the YourNextMP and TheyWorkForYou updates, you may like to read more about it in this thread.
A blank canvas
Note that new MPs will not have a great deal on their pages yet: TheyWorkForYou’s MP pages are built up of voting and debating activity, past positions and expenses, etc, and of course, totally new MPs will have none of that. But there’s one important feature that you should take advantage of on Friday—the ‘subscribe’ button.
Sign up, and we’ll send you an email every time your new MP speaks in Parliament, so you can keep track of exactly what he or she is saying in your name. If you were previously following an MP who has resigned or lost their seat, don’t forget to follow the new one! We’ll be sending out a message straight after the election to remind you.
Another website which will require a lot of attention post-election is WriteToThem, which matches you with your local and national politicians so that you can contact them.
Unfortunately, WriteToThem takes a little longer to update, as we rely on data, including email addresses, from external sources. We’ll be updating as soon as we can. Meanwhile, if you have an urgent message for your MP or councillors, you may find that you can locate direct email addresses on the official Parliament and council websites.
Has he or she reflected your interests? One key way of checking that is to look at their voting record.
We’d like everyone to know exactly how their MP voted over the last parliament, so we’ve made some changes to TheyWorkForYou that make votes easier to understand.
See an example here, or read on to find out how to check your own MP’s voting record.
A complex matter
TheyWorkForYou publishes activity from Parliament each day.
This content includes parliamentary votes, along with the debates that they are part of. But it’s not always obvious to the lay reader exactly what’s being voted on.
Take a look at this debate, for example, on exemptions for smoke-free premises. By the time you’ve waded through the first clause,
“The appropriate national authority may make regulations providing for specified descriptions of premises, or specified areas within specified descriptions of premises, not to be smoke-free despite section 2”
– you may well be lost. And who would blame you?
Making it nice and simple
We don’t think you should have to be an expert to check your own MP’s voting record, and our new pages for each voting stance are here to help.
For some time now we’ve given you summaries of how your MP voted on certain topics, with a link to the votes that helped us understand each MP’s position on that stance.
Now we’ve created a page for each stance, and worded it in plain English so that anyone can understand exactly what it means.
See for yourself
Here’s how to see how your own MP voted (or we should say ‘previous MP’, since until the General Election, no MPs are now in office):
Go to TheyWorkForYou.com and input your postcode on the homepage.
You’ll be taken to the page of your (former) MP. Click on the ‘voting record’ tab.
Choose a topic you’re interested in, and click the ‘Details’ link on the far right.
You’ll see a plain English description of the stance, followed by descriptions of all the votes that were considered to contribute to it.
Want to see the context? Click on ‘show full debate’ and you’ll be taken to the full record of that vote.
Let us know what you think
These pages are still a work in progress, so we’ve included a feedback box at the top of each voting stance page. Do be sure to let us know if there’s anything else you’d like to see on them.
If you have feedback about how your MP has voted, mind you, that’s another matter… one you might want to reflect at the ballot box.
In a recent blog post, we summarised the research we commissioned from the University of Manchester’s Rachel Gibson, Marta Cantijoch and Silvia Galandini, on whether or not our core UK websites have an impact.
The full research paper is now available, and you can download it here .
Professor Rachel Gibson says: “This research presents a unique and valuable insight into the users of online resources such as FixMyStreet and WhatDoTheyKnow.
“Through applying a highly original methodology that combines quantitative and in-depth qualitative data about people’s experience of mySociety sites over time, we provide a picture of how eDemocracy tools are contributing to activism at the local level.
“We thank all those that contributed to this important study and mySociety for their co-operation in developing this highly rewarding and academically rigorous project.”
Our thanks to Rachel, Marta and Silvia for conducting this research, which utilised methods not previously used in the civic tech field. We hope that it will prove a useful foundation to our own further research, and that of others.
Here, in both video and quotes, are a few selected highlights from our speakers; including some of their lovely remarks about the work mySociety does.
Speaking about The Content Mine
mySociety is one of the most wonderful things to have come out of the bottom-up democratic movement in the UK and the UK is a shining light for the rest of the world. I’ve used WriteToThem on many occasions…. It just makes the whole business of contacting your representative so much easier. And I’ve also used a lot of WhatDoTheyKnow FOI requests and again it’s absolutely brilliant. It makes the difference between doing it and not doing it.
We’re going to liberate one hundred million facts per year from the scientific literature and we’re going to put them in Wikipedia or rather WikiData and we’re working closely with WikiData.
What’s happened this year is the UK Government has pushed through copyright reform and it has given exemptions to copyright … We’ve got the law. The law hasn’t been tested. I am allowed to do it according to the law for non-commercial purposes. Elsevier says I can’t because they can stop me doing it under the law and we had a big public fight in London.
Speaking about TheyWorkForYou.com
The thing I work on particularly on TheyWorkForYou is the statements we write on each MP’s page on how they voted. … This will be the first time we’re going into a general election in this country where the sitting MPs’ voting records are comprehensively easily accessible to the electorate.
It’s really important to us that we’re impartial and non-partisan. So one of the things we had to think about when we were doing this was how do we even decide what topics to cover because we could be accused of being partisan just by what we decide to draw attention to. … Not all MPs attend all votes by any-means so we can use MPs’ own attendance at votes to give them some kind of ranking of importance.
Everything that I do is available under an open licence so as long as you attribute where it has come from you can use it and do what you like with it. And hopefully people will do stuff with it as we run into the election.
Mike Soper and Hendrik Grothuis
Speaking about Cambridgeshire Insight
If you think about something like FixMyStreet you can see where that application has had a very positive impact on local government, on councils.
The idea is that pressure will come from the great British public at a local level to hold public sector organisations to account. In order to hold people to account you need information.
Professor Shepherd several years ago realised, because he was a medical professor, that he was looking at facial injuries of people who had been injured by having beer glasses shoved in their faces during fights and recording meticulously the detail of these physical and working out that if you change the composition of the beer glass you can drastically reduce the severity of the injury.
We’re getting support at a national level for the sort of work we are doing and the sort of line about trying to encourage openness and promote open data here in Cambridgeshire. We’re getting national support for that.
Videos of full talks, including Q&A:
- Peter Murry-Rust on The Content Mine. (Transcript)
- Richard Taylor on TheyWorkForYou’s voting records.
- Mike Soper and Hendrik Grothuis of Cambridgeshire Insight. (Mike has also published his slides)