As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
Back in November 2013, we asked you what improvements you’d like to see on TheyWorkForYou.
One answer dominated: you wanted more information about how MPs vote.
Adding information on voting has been the single biggest project on the site since its launch, and has required several different phases of development. We announced each of these as it happened, but now that we’re at the end of this large piece of work, it seems like a good time for a complete overview.
So let’s take a look at exactly what it has involved—and, more importantly, what it means for you.
We’ll start with a rundown of features, then go into more detail about how they are created at the end of the post, for those who are interested.
What vote information means for you
1. You can easily see how your MP voted
Just how much do you know about how your MP voted on the stuff that matters? Most of us would have a hard time keeping up with every vote, simply because it isn’t information that’s widely publicised.
On TheyWorkForYou, you can see a run-down of how any MP has voted on key policies, by visiting their page on the site and clicking the ‘voting record’ tab (see image, above). We’ve created summaries of their stance on all kinds of matters, including the EU, same-sex marriage, NHS reform and a lot more.
Each of these summaries is compiled from every vote the MP has made on a motion that impacts on that policy.
You can click ‘show votes’ (see image above) to see the specific votes that go to make up any particular stance, and we’ve laid them all out in plain English so that it’s easy to grasp exactly what the issue is.
And from there you can click through to the website Public Whip, where you can explore votes in more detail, including lists of who voted for or against any given motion.
2. You can find out how strongly your MP feels
When we first presented voting information, we said that an MP had voted ‘strongly for’ or ‘moderately against’ certain policies, which led to quite a large postbag from people asking, “How can you vote strongly, surely you either vote for or against?”.
We wrote in the second half of this blog post about the wording changes we made to clarify the fact that these stances are calculated from a number of votes.
3. You can assess if your MP is a sheep or a lone wolf
We’ve pulled out all the votes which differ substantially from the way that the majority of each MP’s party voted. If your MP has voted against the flow, you’ll see something like this on their page:
Why do we highlight this type of vote? Because we think they’re a really good indication of where an MP feels strongly enough about something to risk sticking their neck out. It’s also a great way to check the truth when people say, “MPs? They’re all the same”.
4. You can understand the background to the votes
Generally speaking, there’s a debate before any vote takes place in Parliament, covering all the matters which may be topmost in MPs’ minds before they cast their lot.
Clicking on the ‘show full debate’ link from the topic pages (see image above) will give you the full context.
How we compile vote information
If that all seems nice and simple, well, great! That was our aim.
Putting it all together definitely wasn’t so simple, though. Voting information has never been previously presented all in one place in quite this way before—on TheyWorkForYou or anywhere else, to the best of our knowledge—so we had to figure out how to import the data and how best to display it.
As with much of our work, it’s a mixture of manual graft and automating whatever we can. Some things, like rewriting votes so that everyone can understand them, can’t be done by a computer. But many of our users are surprised to learn just how much of what we publish out is untouched by human hand.
Our Developer Struan, who did the most recent round of work on the voting records, said:
We get all our voting data from PublicWhip, a site set up by Francis Irving (once of mySociety) and Julian Todd. Public Whip takes the data we [TheyWorkForYou] produce from Hansard and extracts only the information on votes (or divisions in Parliamentary jargon) that take place in Parliament. It then allows you to look up how an MP or a Lord voted.
Let’s just think about that for a moment. We’re looking at a process where Parliament publishes Hansard, TheyWorkForYou scrapes the data and re-presents it, Public Whip extracts the voting information and presents that, and TheyWorkForYou takes that voting information back for its own voting pages. Simple…
One of the first things we did was to ‘translate’ the votes into plain English, so that it was very clear what was being voted for or against— and if you want to read more about that process, we talked about it in a blog post back in July 2014.
That allowed us to move to the next phase, as Struan explains:
Public Whip groups related votes together into policies, e.g renewing Trident, so you can see how an MP voted on the policy as a whole.
It does this by saying which way an MP would have to vote each of the divisions in the policy if they agreed with the policy. It then takes the MP’s votes on each division in the policy and assigns a score to it based on how they voted. These scores are then added up and compared to the score they would get if they always voted in agreement with the policy. The closer the MP’s score is to the score of an MP who always voted in agreement with the policy, the more they agree with the policy.
Thanks to Public Whip’s grouping, we were able to start compiling our MPs’ voting records along those same policy lines.
One of the most fiddly parts of the process was figuring out how to ensure that the information we present is a true, non-biased representation of the MP’s intentions. You might think that a vote is quite a simple matter – it’s either a yes or a no for a particular motion. But as soon as we started displaying votes within a policy, things got a bit trickier.
Some divisions in a policy can be marked as important and voting with the policy in those divisions is worth more points. This is to prevent voting in agreement on a set of minor votes, e.g “Parliament will commission a report on the future of Trident”, outweighing voting against something important, e.g. “Renew Trident”. It also reflects the way Parliament works, often with several smaller votes on parts of a bill and then a vote on the bill as a whole.
For clarity I should point out here that sometimes voting no in a division is a vote for the policy, e.g voting no in a “This house believes Trident should not be renewed” division would clearly be a vote for our example “Renew Trident” policy.
This approach also helps where one vote straddles several topics: for example, consider a vote against the Budget when the Budget contains many proposals including, say, the capping of VAT. It’s quite possible that an MP may be for the capping of VAT but broadly against several other motions covered by the Budget, and so decides to vote against it on balance. So long as we mark the Budget vote as a weak vote for the capping of VAT, its significance should be properly accounted for.
Where we don’t have enough information to show a stance, for example where an MP never voted on the topic, is too new to have had a chance to vote on the topic, or all their votes on the topic have been labelled as ‘weak’, we say so:
A final little subtlety is the difference between “Never voted” on a policy and votes where the MP was absent. If it says an MP has never voted on a policy that means they were elected after all the divisions in the policy took place so did not have a chance to vote on them. Absent means they could have voted in the divisions but did not.
Absent votes count towards your score but at half the rate of voting in agreement with the policy. This is so that an MP who votes in agreement with the policy in one division and then misses all the other divisions shows as agreeing with the policy rather than against as it would if no score was assigned to absent votes. That does currently mean that if they were always absent it shows, slightly unhelpfully, as “a mixture of for and against”.
It’s not an ideal system as it does produce some odd results occasionally but it mostly works.
To show where an MP has voted against the majority of their party, we have to figure out a similar score across the party as a whole.
This is exactly the same process, only we add up all the votes by all the MPs but the maths is pretty much the same.
All in a day’s work
As mentioned at the top of this post, vote information was our most-requested addition. And rightly so! Our MPs represent us, so naturally we want to see their track records, quickly and easily.
If you’re not an expert, you might not have known how to find this information before. And that’s essentially what TheyWorkForYou aims to do: make the workings of Parliament more accessible for everyone.
Parliamentary copyright images are reproduced with the permission of Parliament.
Remember the UK General Election? Yes, we know it’s a distant memory now, and you’ve probably forgotten YourNextMP, too. But the project is far from dormant!
YourNextMP successfully crowd-sourced information on every election candidate, and made it available as open data for anyone who wanted to use it to build useful websites and online tools.
And while here in the UK we won’t have further use for it until 2020, the great news is that the underlying code can be repurposed to work for other elections around the world. Thanks to Yo Quiero Saber, the first of these is now live and collecting data for Argentina at http://investigacion.yoquierosaber.org/, and there are also plans for DataMade Chicago to use it in the USA.
In Argentina, the crowdsourcing component sits as part of a wider voter informing project. Martín Szyszlican from Yo Quiero Saber explains more:
We just launched Yo Quiero Saber and it’s had a great reception. You’re welcome to visit our main site, where we feature the game and full profiles for candidates for presidency and governors of four provinces.
You can also see our YourNextRepresentative instance (we renamed it, since MP is not a relevant term for us) where, in just two weeks, we’ve already had more than 100 registered users, and have also managed to add all the official candidates from DINE (the national elections office).
We’re still missing city-level and provincial-level candidates from the site, but that’s going to be improved before the October general elections.
So far, we’ve had 350,000 unique users and a million page views since launch. That means we are close to reaching 1% of the total number of voters in the country. Neatly, the number of people who have used the site is roughly equivalent to the number of voters a party needs to pass from this election to the next ones.
Media reception has been great with online portals big, small and regional mentioning our site and some of them embedding our game in their articles. We’ve also been kept busy with radio interviews and some tv programmes featuring the game. In Argentina, the media is deeply split down party lines, and we very much like the fact that we’ve surfed that divide, being featured in media from both sides of the political spectrum.
This is just the beginning: we’re working as an alliance of local NGOs, and our bid for a prototype grant from the Knight Foundation has been successful, meaning that we can forge ahead with our plans. We’ve also had support from HacksLabs, a data journalism accelerator. The full list of partners can be found on the footer of both sites.
We’re really glad to hear of this success—it’s great to see the code get another lease of life, which is, of course, what the Poplus project is all about.
Naturally, the YourNextRepresentative codebase also available to other countries who want to help inform their electorates, and what’s more, Martín says they’ll be glad to offer help to anyone who wants it. That goes for us here at mySociety too.
Earlier this year, the AlaveteliCon conference brought together people with an interest in online Freedom of Information technologies.
It was an event quite unlike any other, and left a lasting impression of many dedicated people making good things happen for their communities, in places across the world.
That impression is reflected in these short videos, which came about when we yanked attendees away from their lunches and asked them questions in a darkened room.
Thanks very much to everyone who responded so amiably, as well as giving us such useful insights into what it’s like to run an FOI site in all sorts of circumstances. We’ve named them at the foot of this post, along with links to their sites.
My last blog post ran through the history of our ‘rate the view’ site ScenicOrNot.
I was expecting to wrap up with a final paragraph describing its graceful retirement. But no — it turns out that, even as I wrote, emails were going back and forth to secure a whole new career for ScenicOrNot.
Here’s what its new owners at the Warwick Business School have to say:
Does living in picturesque areas make you feel healthier? Urban planners and think tanks have puzzled over this question for years, but have been held back by a lack of data on the beauty of our environment.
We were immensely excited to discover the data being collected by ScenicOrNot, as it gives us a crucial opportunity to finally get some answers to this age-old question.
Our initial analyses of the ScenicOrNot data suggest that people living in more scenic environments report better health, even when taking variables such as income and greenspace into account. These results suggest that the beauty of our everyday environment might have more practical importance than has previously been realised.
We’ve written a paper describing these analyses, which is currently under review. Keep in touch with us via Twitter (@thoughtsymmetry or @thedatascilab) and we’ll let you know when the paper is published.
We’re very honoured that mySociety are passing the ScenicOrNot site into our care. We’re excited about having the opportunity to customise the site and gather more data for our research, and we’d also love to expand this work to other countries. Stay tuned to hear what comes next!
We’re excited too, of course — and really pleased that ScenicOrNot has been redeployed in such a useful way.
The good news for you is that you can carry on rating photos for scenicness over at the site’s new home, all in the knowledge that you are increasing our understanding about the correlation between health and our environment.
Oh, and meanwhile: how would you rate the view from your window? You might want to talk to your doctor about that.
Take a look out of the window. How would you rate the view, on a scale of one to ten?
Your response can probably tell us a little about the beauty, or otherwise, of the area around you. That’s the premise that ScenicOrNot, one of the mySociety sites that we recently stopped running, was founded on. Happily, ScenicOrNot has now found a home and will continue under new ownership: more about that in a future blog post. Meanwhile, we’d like to celebrate it with a potted history.
An exercise in crowdsourcing, ScenicOrNot served up a series of random images, each representing one square kilometre of Great Britain, and invited users to rate them (the images were sourced from the Geograph project, itself a fascinating open source repository). The results fed into a database of ‘scenicness’.
ScenicOrNot collected that data and also permits anyone to download it, under an Open Data Licence, for their own ends.
What was it for?
To understand why we made ScenicOrNot, you have to go back to the beginnings of our transit-time mapping technology, Mapumental.
Mapumental shows journeys in terms of how long they take, and it was intended to help people make decisions about where to live, work, or go on holiday. We’d figured out how to display bands of public transport journey times, but we knew that those weren’t the only factors that feed into such important life choices.
House prices, average salaries, and, yes, the beauty of the surrounding area all have a part to play. We wanted to be able to add them to Mapumental so that users could get a really rounded picture.
But while there are public databases for house prices and average salaries available, until the creation of ScenicOrNot, there was no such thing for scenicness. There was just one solution: we would have to make our own.
‘Hot or Not’ for scenery
Rather than go and look at every part of the country ourselves, it was time to harness the wisdom of the crowd.
ScenicOrNot, the building of which was managed by the Dextrous Web, launched in 2009. It served users with an unending random series of images showing landscapes from around the country, and was an early foray into both crowdsourcing and gamification for mySociety.
Rating images, as also seen in Kittenwar (and other, less fluffball-centric sites like HotOrNot) is a pleasingly compulsive activity, and within just a few months, every kilometre of the country had been rated at least once.
And as time went by, we reached a critical milestone: the project amassed a minimum of three votes for each image, helping to ensure that the results were less likely to be skewed by eccentric or unusual opinions about what makes a place scenic.
Slotting ScenicOrNot into Mapumental
We now had our ‘scenicness’ data, and house price and salary data from other sources. The decision we made about how to incorporate these data sets was an important one which has worked well for subsequent Mapumental projects like the work we did for the Welsh government, or for the Fire Protection Association.
Effectively, you can think of each data set as a map layer, which may be slotted in our out, as needed. Our showcase site Mapumental Property demonstrates this – it’s effectively the vanilla transit-time Mapumental, with a house price layer (from Zoopla) added in.
A new lease of life
If we hadn’t found a new owner for ScenicOrNot, we’d have shut it down. Happily, though, it’s found a new home and a whole new purpose: we’ll be explaining more about that in our next blog post.
If you want a copper-bottomed example of how Freedom of Information can benefit us all, you might do worse than to watch How the Hammers Struck Gold (broadcast last week, and available via iPlayer until Friday).
This BBC programme examines, in the space of half an hour, the fine detail of the rental agreement which grants West Ham United the use of the Olympic Stadium.
The stadium’s owners, the London Legacy Development Corporation, are a public authority, so they are bound by the Freedom of Information Act. That means that anyone has the right to ask them for information, and if they hold it, they must release it.
WhatDoTheyKnow user Richard Hunt requested the terms of the rental agreement, and—well, you can see the rest for yourself. It’s a must-watch for anyone interested in football, but there’s plenty for the rest of us too.
The investigation speaks more widely of transparency around the use of tax-payers’ money, as well as the multiple revenue streams—some only loosely related to the actual game—which are up for grabs when a team reaches the Premier League.
If you happen to see this post after the programme has been removed from iPlayer, you can also find a good written summary of the findings on the BBC website.
For a while now, TheyWorkForYou has shown how your MP voted on key topics.
What it hasn’t done, until this week, is give a crucial piece of context. That is, how do your MP’s votes differ from those of their colleagues in the same party?
We all know that, on many issues, the whip ensures that MPs vote according to the party line rather than their own convictions. So in theory, by examining the votes which diverge from the majority party vote, we might get the clearest picture of what an MP truly cares about.
And now, we’ve added a small piece of code to the site, which allows us to do just that. At the top of your MP’s page, you’ll now see text along these lines:
If your MP never disagrees with their party, you’ll just see the top line followed by a random selection of votes.
The importance of wording
The screenshot above shows another small change we’ve made to TheyWorkForYou: just a matter of wording, this time.
When we first started displaying how MPs had voted, we used terms such as “voted strongly for”, “voted moderately against”, etc. This was to allow us to represent a range of positions along a spectrum for each topic.
For every topic, such as EU Integration, or smoking bans, several different votes are analysed. The ‘show votes’ button, as seen above, takes you to a page where these are listed.
However, we received a steady stream of emails, tweets and Facebook messages asking how an MP can vote ‘strongly’ or ‘moderately’ for something. To a fly-by reader, it seemed nonsensical, because of course they were thinking of that fact that MPs vote for or against a single motion.
To counteract this, we’ve used words which we hope encapsulate the concept of a series of votes over time – words like ‘consistently’, ‘occasionally’ and ‘never’.
Choosing these words proved to be harder than we’d anticipated, and, after a long heated discussion between colleagues, resulted in a straw poll asking anyone we could find to arrange pieces of paper in a line to indicate how they perceived their strength.
We finally came up with an answer that the majority agreed on—and we haven’t had any mail on the subject since then. Let’s cautiously call that a win for careful wording.
If you need data on the people who make up your parliament, another country’s parliament, or indeed all parliaments, you may be in luck.
What’s more, it’s all provided as Open Data to anyone who would like to use it to power a civic tech project. We’re thinking parliamentary monitoring organisations, journalists, groups who run access-to-democracy sites like our own WriteToThem, and especially researchers who want to do analysis across multiple countries.
But isn’t that data already available?
Yes and no. There’s no doubt that you can find details of most parliaments online, either on official government websites, on Wikipedia, or on a variety of other places online.
But, as you might expect from data that’s coming from hundreds of different sources, it’s in a multitude of different formats. That makes it very hard to work with in any kind of consistent fashion.
Every Politician standardises all of its data into the Popolo standard and then provides it in two simple downloadable formats:
- csv, which contains basic data that’s easy to work with on spreadsheets
- JSON which contains richer data on each person, and is ideal for developers
This standardisation means that it should now be a lot easier to work on projects across multiple countries, or to compare one country’s data with another. It also means that data works well with other Poplus Components.
What can I do with it?
Need a specific example? Yesterday, we introduced Gender Balance, the game that gathers data about women in politics.
As you’ll know if you’ve already given it a try, Gender Balance works by displaying politicians that make up one of the world’s legislatures, one by one.
That data all comes from Every Politician, and it’s meant that the developers have been able to concentrate on making a smooth and functional interface, knowing that the data side of things has already been taken care of.
That’s just one way to use Every Politician data, though. If you’d like to use it in your own site or app, you can find out more here.
We still need more data
As you may have noticed, there are more than 100 parliaments in the world. In fact, despite having reached what feels like a fairly substantial milestone, we’re still barely half way to getting some data for every parliament.
So we could use your help in finding data for the parliaments we don’t yet cover, and historic information for the ones we do. Read more about how you can help out.
From Afghanistan to Zimbabwe, there are over 180 parliaments in the world — but what percentage of their members are female?
The crazy thing is, there’s no definitive figure*.
So we created Gender Balance, an easy game that crowd-sources gender data across every parliament in the world. Try it! We hope you’ll find it fun.
Gender Balance isn’t just an enjoyable way to fill half an hour, though: users will be helping to build up a dataset that will be useful for researchers, campaigners, politicians, and sociologists. As the results emerge, we’ll be making them available in an open format for anyone to use, to answer questions like:
- Which country has the highest proportion of women in parliament?
- Do women vote differently on issues like defence, the environment, or maternity benefits?
- Exactly when did women come into power in different countries, and did their presence change the way the country was run?
Gender Balance’s underlying data comes from another mySociety project—EveryPolitician, a database which aims to collect information on every politician in the world.
And while it’s nailing down those stats on gender balance across every country, Gender Balance also aims to be a showcase of what can be done with the open data from EveryPolitician. That data is free for anyone who wants to build tools like this, and it’s easy to use, too. Find out more about that here.
*While the Inter-Parliamentary Union does collect figures, they are self-reported, often out of date, and only cover its own members.