Even official records aren’t as safe as you might think they are. The archive of a country’s political history might be wiped out in a single conflagration.
Take the example of Burkina Faso, a beautiful West African country that is, sadly, perhaps best known to the rest of the world for its troubled political past.
The uprising in Burkina Faso in 2014 led to a fire in the National Assembly building and archives office. Nearly 90% of the documents were lost. Now the National Assembly is working to reconstruct the list of its parliament’s members before 1992.
This means that the data EveryPolitician has on Burkina Faso has nothing from terms before 1992. We’ve got some data for six of the seven most recent terms from the National Assembly so far, of which five are live on the site. Even though that data is not very rich (there’s little more than names in many cases; and the 6th term was transitional so data on that one’s membership might remain elusive) it’s a beginning.
We know from experience that data-gathering often proceeds piecemeal, and names are always a good place to start.
As Tinto finds new data, whether that’s more information about the politicians already collected or membership lists of the missing terms before 1992, we’ll be adding that to EveryPolitician too.
A vast collection
When people ask what EveryPolitician is, we often say, ‘The clue’s in the name’. EveryPolitician aims to provide data about, well … every politician. In the world.
(We’ve limited our scope — for the time being — to politicians in national-level legislatures).
The project is growing. Since our launch last year, we’ve got data for legislatures in 233 countries. The amount of data we’ve collected currently comprises well over three million items. The number of politicians in our datafiles is now in excess of 70,000.
Seventy thousand is an awful lot of politicians.
In fact, if you think that might be more politicians than the world needs right now, you’re right: as the Burkina Faso example shows, EveryPolitician collects historic data too.
So as well as the people serving in today’s parliaments, our data includes increasing numbers of those from the past. (Obviously, if you have such data for your country’s legislature, we’d love to hear from you!)
More than just today’s data
The Burkina Faso fire is an illustration of the value of collecting and preserving this historic data.
Of course, we’re fully aware of the usefulness of current data, because we believe that by providing it we can seed many other projects — including, but in no way limited to, parliamentary monitoring sites around the world (sites like our own TheyWorkForYou in the UK, or Mzalendo in Kenya, for example).
Nonetheless, we never intended to limit ourselves to the present. By sharing and collating historic records too, we hope to enable researchers, journalists, historians and who-knows-who-else to investigate, model, or reveal connections and trends over time that we haven’t even begun to imagine. We know this data has value; we look forward to discovering just how much value.
But it turns out we’re providing a simpler potential benefit too. EveryPolitician’s core datafiles are an excellent distributed archive.
What Burkina Faso’s misfortune goes to show is that, as historians know only too well, data sources can be surprisingly fragile.
In this case the specific situation involves paper records being destroyed by fire. That is a simple analogue warning to the digital world. Websites and their underlying databases are considerably more volatile than the most flammable of paper archives.
Database-backed sites are often poor catalogues of their pasts. Links, servers and domain registrations all expire. Access to data may be revoked, firewalls can appear.
Digital data doesn’t fade; instead it is so transient that it can simply disappear.
Of course, we cannot ourselves guarantee that our servers will be here forever (we’re not planning on going anywhere, but projects like this have to be realistic about the longer view).
There is an intriguing consequence of us using GitHub as our datastore. The fact is, the EveryPolitician data you can download isn’t coming off our servers at all. Instead, we benefit from GitHub’s industrial-scale infrastructure, as well as the distributed nature of the version control system, git, on which it is based. By its nature, every time someone clones the repository (which is easy to do), they’re securing for themselves a complete copy of all the data.
But the point is not necessarily about data persisting far into the next millennium — that’s a bit presumptuous even for us, frankly — so much as its robustness over the shorter cycles of world events. So, should any nation’s data become inaccessible (who knows? for the length of an interregnum or civil war, a natural disaster, or maybe just a work crew accidentally cutting through the wrong cable outside parliament), we want to know the core data will remain publicly available until it’s back.
Naturally there are other aspects to the EveryPolitician project which are more — as modern language would have it — compelling than collecting old data about old politicians. But the usefulness of the EveryPolitician project as a persistent archive of historical data is one that we have not overlooked.
If you’ve used WriteToThem, you’ll know that two weeks after you submit a message to your MP, we send a follow-up questionnaire to check whether you received a response.
Each year, we collate that data to see how MPs are doing at responding to constituents’ mails*, and we publish the results. (This year, we waited a bit longer than usual so that we could cover a full year since the general election.)
They’re now live, so you can go and check exactly how your own MP did — just enter your postcode.
Some interesting stats
- Because we’ve been running these figures since 2005 (with a gap between 2008-13), we can make some comparisons. We’re disappointed to see that the responsiveness rate of MPs has been steadily declining. In 2005, 63% of respondents indicated that they’d had a reply; this year, that’s down to 50%.
- Before we analysed the data, we thought that new MPs, elected in 2015, would perhaps perform better than the jaded incumbents. Not so: on average ‘old’ MPs responded to 53.07% of constituents’ messages, while the newly-elected managed only 46.10%. One new MP, Marcus Fysh, MP for Yeovil, came in at 635 out of the 642 MPs eligible for inclusion.
- Receiving more mail doesn’t necessarily mean you’ll perform poorly. Notable in this respect is Gerald Kaufman, who managed a 79% responsiveness rate despite having the second largest postbag.
- And being in the public eye doesn’t necessarily impact an MP’s responsiveness: Sadiq Khan and Jeremy Corbyn performed poorly, but have done so in prior years, too. Equally, we suppose it follows that a poor responsiveness level doesn’t necessarily impact on electoral success.
- We were curious to know whether there’s a gender divide when it comes to responsiveness. There is, but it’s very slight: on average male MPs responded to 52% of correspondence; female MPs to 50%.
- And another thing we’ve been asked about, sometimes by MPs themselves. There is no significant relationship between parliamentary constituency size and responsiveness. In other words, having more people in a constituency does not automatically mean that the MP is a poor responder.
Anyway, enough of this — go and check how your MP did, and then tell everyone else to do the same.
*This needs a caveat. Our data only covers messages sent via WriteToThem, and, furthermore, only those messages where users completed the questionnaire. You can see the full methodology on the rankings page.
Well, it certainly all happened over the weekend: the resignation of one Secretary of State on Friday and the quick appointment of another by Saturday.
It all left a lot of people wondering just who this Stephen Crabb fellow was, and what he stood for.
Fortunately, there’s a very handy website where you can look up the details, debates and voting records of every MP — we refer, of course, to our very own TheyWorkForYou. Over the weekend, we saw the link to Crabb’s voting record shared across social media (and even good old traditional media; we were also mentioned on Radio 4’s Any Answers). Naturally, most interest was around Crabb’s voting habits when it comes to welfare and benefits.
The upshot of this was that TheyWorkForYou saw almost three times our normal traffic for a Saturday. Over the weekend, 30% of all page views were for Crabb’s profile or voting records. In contrast, just 1.83% thought to check out his predecessor’s record: yesterday’s news already, it seems.
So Stephen Crabb’s the new guy, and you may want to keep up to date with his contributions to Parliament. Sign up here and we’ll send you an email every time he speaks.
For verified, reliable information, it’s usually best to go to the official source — but here’s an exception.
Checking parliament.go.ke‘s list of MPs against Mzalendo’s, our developers discovered a large number of constituency mismatches. These, explained Jessica Musila from Mzalendo, came about because the official site has not reflected boundary changes made in 2013.
Even more significantly, the official parliament site currently only holds details of 173 of the National Assembly’s 349 MPs.
“The gaps in www.parliament.go.ke validate Mzalendo’s very existence,” said Jessica. We agree: it’s a great example of the sometimes unexpected needs filled by parliamentary monitoring websites.
And of course, through EveryPolitician, we’re working to make sure that every parliamentary monitoring website can access a good, reliable source of data.
A few weeks ago, we highlighted one major difference between the Ghanaian parliament and our own: in Ghana, they register MPs’ attendance.
This week, we received news of another of our partners who are holding their representatives to account on the matter of attendance: People’s Assembly, whose website runs on our Pombola platform. The new page was contributed by Code4SA, who have been doing some really valuable work on the site lately.
According to South Africa’s Daily Maverick, in some cases MPs’ attendance is abysmally low. There’s also a history of those who “arrive, sign the register and leave a short while later”, a practice that may soon be on the decline thanks to People’s Assembly’s inclusion of data on late arrivals and early departures.
With 57 representatives — or about 15% — floundering at a zero rate of attendance, it seems that this simple but powerful display is a much-needed resource for the citizens of South Africa. See it in action here.
We recently added an Environment section to voting pages on TheyWorkForYou, so now you can see exactly how your MP voted on issues like fracking, measures to prevent climate change, and green energy, all in one place, like this:
Votes on environmental issues are clearly a priority for our users. They’ve been one of the most-requested additions in the TheyWorkForYou postbag over the last couple of years, and we’re glad to have fulfilled those requests, even if it took a while.
At the same time, we’ve also made several other additions to existing sections on voting pages, so now you can check how your MP has voted in these areas:
- Assisted dying
- Trade union regulation
- Taxation of banks
- Enforcement of immigration rules
- MPs’ veto over laws only affecting their part of the UK (AKA English votes for English laws)
To check your own MP’s voting record, head over to TheyWorkForYou.com, and input your postcode on the homepage. Then click ‘voting record’ at the top of your MP’s page.
If you have strong opinions about how your MP voted on any issue, don’t forget, you can let them know by clicking on ‘Send a Message’, which will take you over to WriteToThem.com.
It’s around this time of year that we normally publish our responsiveness statistics on WriteToThem. However, if you’ve been looking forward to seeing your MP’s ranking, we’re afraid you’ll have to wait a little longer.
Two weeks after you use WriteToThem to contact a representative, we send you an automated email to check whether or not you received a response. The data gathered by these questionnaires gives us a snapshot of how well the site is working for its users; it also allows us to highlight which MPs, which parties, and which parliamentary bodies do the best and worst at responding to constituents’ messages.
We’ve habitually analysed a calendar year of responses, January to December. Last year, though, was an election year, meaning that several MPs were active up until May, and then several new MPs took their seats in the new Parliament. So we’re going to run the data in June, looking at May 2015 to May 2016, followed by a four-week period to ensure we’ve received all the questionnaires.
Now, in theory, it shouldn’t matter too much, because we rank MPs by the percentage of mail sent through WriteToThem that they respond to (or more accurately, that our users tell us they have responded to). An MP may have responded to 100% of all their mail and then been voted out; their successor may then respond to 10% of their mail: both MPs would be ranked accordingly.
In fact, that’s how we did it for 2005, the first year for which we published WriteToThem rankings, and also an election year*.
But shifting the date like this means that the data will be less confusing. It’ll let us see how every current MP has performed, in terms of responsiveness, across a full year.
Of course, one side effect of this is that if you’re an MP and you want to be top of the pops, you have an extra five months in which to boost your score… so, on your marks, time to get writing!
*2010 fell within a four-year period during which we didn’t publish rankings.
How do you know when Parliament is going to be debating the things you care about? One way is to use TheyWorkForYou—you can set it up to send you handy reminders ahead of time.
On the agenda
The legalisation of cannabis is one of those topics that people have strong opinions about, and we’ve noticed a few tweets where people are saying that they’ve used WriteToThem.com to share their views with their MPs, ahead of today’s Westminster Hall debate on the matter.
Of course, you can share your thoughts on any topic with your MP, at any time. Doing so just before a debate is useful, though, as it means your representative is more likely to take your views into consideration before voting or speaking.
TheyWorkForYou.com, our site that covers the UK’s parliaments, actually makes it pretty easy to time your messages correctly. As well as publishing everything spoken in Parliament, it also shows upcoming business.
More than that, you can subscribe to any key word or phrase within the upcoming business section, and we’ll send you an email whenever it arises. So, whether you care deeply about cannabis, or your interests lie in another topic all together, you’ll know when a debate is scheduled.
And then you can get straight onto WriteToThem to write your message.
Here’s how to set up your ‘future business’ alert:
1. Go to the future business page on TheyWorkForYou.
2. Enter your chosen word or phrase in the search box to the right (titled Search upcoming business, or set up a future business email alert)
3. You’ll be taken to a page showing any future business containing your keyword. On the right of that page you’ll see a box like this:
Notice the small text: (or just forests in Future Business). Click on this if you’d like to receive results only for forthcoming debates.
4. You’ll be asked to confirm what you want:
5. Click ‘subscribe’ and you’ll be asked to input your email address (unless you are already logged in). Check your email to confirm your address and you’re done — all ready to fire off an email to your MP next time something important is on the horizon.
We received a tweet this morning wondering how many emails there had been to MPs on the subject of the ‘Refugees Welcome’ campaign, and whether WriteToThem, our contact-a-politician website, might have some relevant data.
Even if we could, WriteToThem is a completely non-partisan service, and users may be writing on either side of an issue.
We do use Google Analytics, which collects entirely anonymous statistics on how many people visit the site, how long they stay on it, etc. There is one clear indication that the site is being used more than usual: user numbers on Thursday and Friday of last week were about 5 or 6 times higher than the norm. There was a dip at the weekend — there generally is — and numbers have continued to climb on Monday and today.
Google Analytics also allows us to see which websites have referred people to our site. Over this period, it seems it was mostly Facebook and the petitions site Avaaz.
With most websites, you can regard visitor numbers as a pretty good indication of your success — if they’re going up, then at least something’s right.
With WriteToThem though, user numbers regularly fluctuate so wildly that you could be fooled into thinking we’re on the brink of disaster, or the brink of world domination, from one week to the next.
In the normal way of things, there seems to be a baseline at which the UK populace will toddle along. A small percentage of us will write to our politicians whenever we have an opinion that we want to express, but most of us are content with a few acerbic Facebook updates or heated discussions down the pub.
Then, now and then, an issue comes along which grips the nation. This week, that would indeed appear to be the issue of refugees.
Of course, we’re always glad to see the site used, and we hope that people who are referred to it because of an issue they care about will also remember it’s there whenever they need to contact their representatives in the future.
Incidentally, if you are running any kind of campaign and you would like to harness WriteToThem’s functionality on your own site, don’t forget that we’ve written a guide to doing just that.
Back in November 2013, we asked you what improvements you’d like to see on TheyWorkForYou.
One answer dominated: you wanted more information about how MPs vote.
Adding information on voting has been the single biggest project on the site since its launch, and has required several different phases of development. We announced each of these as it happened, but now that we’re at the end of this large piece of work, it seems like a good time for a complete overview.
So let’s take a look at exactly what it has involved—and, more importantly, what it means for you.
We’ll start with a rundown of features, then go into more detail about how they are created at the end of the post, for those who are interested.
What vote information means for you
1. You can easily see how your MP voted
Just how much do you know about how your MP voted on the stuff that matters? Most of us would have a hard time keeping up with every vote, simply because it isn’t information that’s widely publicised.
On TheyWorkForYou, you can see a run-down of how any MP has voted on key policies, by visiting their page on the site and clicking the ‘voting record’ tab (see image, above). We’ve created summaries of their stance on all kinds of matters, including the EU, same-sex marriage, NHS reform and a lot more.
Each of these summaries is compiled from every vote the MP has made on a motion that impacts on that policy.
You can click ‘show votes’ (see image above) to see the specific votes that go to make up any particular stance, and we’ve laid them all out in plain English so that it’s easy to grasp exactly what the issue is.
And from there you can click through to the website Public Whip, where you can explore votes in more detail, including lists of who voted for or against any given motion.
2. You can find out how strongly your MP feels
When we first presented voting information, we said that an MP had voted ‘strongly for’ or ‘moderately against’ certain policies, which led to quite a large postbag from people asking, “How can you vote strongly, surely you either vote for or against?”.
We wrote in the second half of this blog post about the wording changes we made to clarify the fact that these stances are calculated from a number of votes.
3. You can assess if your MP is a sheep or a lone wolf
We’ve pulled out all the votes which differ substantially from the way that the majority of each MP’s party voted. If your MP has voted against the flow, you’ll see something like this on their page:
Why do we highlight this type of vote? Because we think they’re a really good indication of where an MP feels strongly enough about something to risk sticking their neck out. It’s also a great way to check the truth when people say, “MPs? They’re all the same”.
4. You can understand the background to the votes
Generally speaking, there’s a debate before any vote takes place in Parliament, covering all the matters which may be topmost in MPs’ minds before they cast their lot.
Clicking on the ‘show full debate’ link from the topic pages (see image above) will give you the full context.
How we compile vote information
If that all seems nice and simple, well, great! That was our aim.
Putting it all together definitely wasn’t so simple, though. Voting information has never been previously presented all in one place in quite this way before—on TheyWorkForYou or anywhere else, to the best of our knowledge—so we had to figure out how to import the data and how best to display it.
As with much of our work, it’s a mixture of manual graft and automating whatever we can. Some things, like rewriting votes so that everyone can understand them, can’t be done by a computer. But many of our users are surprised to learn just how much of what we publish out is untouched by human hand.
Our Developer Struan, who did the most recent round of work on the voting records, said:
We get all our voting data from PublicWhip, a site set up by Francis Irving (once of mySociety) and Julian Todd. Public Whip takes the data we [TheyWorkForYou] produce from Hansard and extracts only the information on votes (or divisions in Parliamentary jargon) that take place in Parliament. It then allows you to look up how an MP or a Lord voted.
Let’s just think about that for a moment. We’re looking at a process where Parliament publishes Hansard, TheyWorkForYou scrapes the data and re-presents it, Public Whip extracts the voting information and presents that, and TheyWorkForYou takes that voting information back for its own voting pages. Simple…
One of the first things we did was to ‘translate’ the votes into plain English, so that it was very clear what was being voted for or against— and if you want to read more about that process, we talked about it in a blog post back in July 2014.
That allowed us to move to the next phase, as Struan explains:
Public Whip groups related votes together into policies, e.g renewing Trident, so you can see how an MP voted on the policy as a whole.
It does this by saying which way an MP would have to vote each of the divisions in the policy if they agreed with the policy. It then takes the MP’s votes on each division in the policy and assigns a score to it based on how they voted. These scores are then added up and compared to the score they would get if they always voted in agreement with the policy. The closer the MP’s score is to the score of an MP who always voted in agreement with the policy, the more they agree with the policy.
Thanks to Public Whip’s grouping, we were able to start compiling our MPs’ voting records along those same policy lines.
One of the most fiddly parts of the process was figuring out how to ensure that the information we present is a true, non-biased representation of the MP’s intentions. You might think that a vote is quite a simple matter – it’s either a yes or a no for a particular motion. But as soon as we started displaying votes within a policy, things got a bit trickier.
Some divisions in a policy can be marked as important and voting with the policy in those divisions is worth more points. This is to prevent voting in agreement on a set of minor votes, e.g “Parliament will commission a report on the future of Trident”, outweighing voting against something important, e.g. “Renew Trident”. It also reflects the way Parliament works, often with several smaller votes on parts of a bill and then a vote on the bill as a whole.
For clarity I should point out here that sometimes voting no in a division is a vote for the policy, e.g voting no in a “This house believes Trident should not be renewed” division would clearly be a vote for our example “Renew Trident” policy.
This approach also helps where one vote straddles several topics: for example, consider a vote against the Budget when the Budget contains many proposals including, say, the capping of VAT. It’s quite possible that an MP may be for the capping of VAT but broadly against several other motions covered by the Budget, and so decides to vote against it on balance. So long as we mark the Budget vote as a weak vote for the capping of VAT, its significance should be properly accounted for.
Where we don’t have enough information to show a stance, for example where an MP never voted on the topic, is too new to have had a chance to vote on the topic, or all their votes on the topic have been labelled as ‘weak’, we say so:
A final little subtlety is the difference between “Never voted” on a policy and votes where the MP was absent. If it says an MP has never voted on a policy that means they were elected after all the divisions in the policy took place so did not have a chance to vote on them. Absent means they could have voted in the divisions but did not.
Absent votes count towards your score but at half the rate of voting in agreement with the policy. This is so that an MP who votes in agreement with the policy in one division and then misses all the other divisions shows as agreeing with the policy rather than against as it would if no score was assigned to absent votes. That does currently mean that if they were always absent it shows, slightly unhelpfully, as “a mixture of for and against”.
It’s not an ideal system as it does produce some odd results occasionally but it mostly works.
To show where an MP has voted against the majority of their party, we have to figure out a similar score across the party as a whole.
This is exactly the same process, only we add up all the votes by all the MPs but the maths is pretty much the same.
All in a day’s work
As mentioned at the top of this post, vote information was our most-requested addition. And rightly so! Our MPs represent us, so naturally we want to see their track records, quickly and easily.
If you’re not an expert, you might not have known how to find this information before. And that’s essentially what TheyWorkForYou aims to do: make the workings of Parliament more accessible for everyone.
Parliamentary copyright images are reproduced with the permission of Parliament.