Politwoops tracks politicians’ tweets, and reports the ones that are deleted.

Often those tweets are deleted because of a typo: everyone makes simple mistakes with the buttons on their devices, and politicians are no less human than the rest of us.

But Politwoops’s targets are public servants who use Twitter to communicate with that public. And sometimes the contents of the tweets they delete are not simply the result of bad typing. Those tweets can be especially interesting to people whom those politicians are representing: sometimes they may be evidence of a usually-suppressed prejudice, or an attempt to remove evidence of a previously held opinion that is no longer convenient.

In effect, Politwoops is a public archive of direct quotes that would otherwise be lost.

And also… EveryPolitician

Our EveryPolitician project is an ever-growing collection of data on every politician in the world (we’re not there yet, but we’re over 230 countries and 72,700 politicians in, and counting).

Like Politwoops, our data includes politicians’ Twitter handles. But also a lot more besides.

We make that data useful by putting it out in consistent, simple formats — the simplest of which is a comma-separated value (CSV) file for each term of a legislature. In practice, that means if you want a spreadsheet of the current politicians in your country’s parliament, then EveryPolitician is probably the place for you.

Put them together…

Now, Politwoops predates EveryPolitician by several years, and they’ve being doing their thing without needing our data just fine. In fact, Politwoops has been happily politwooping since 2010 (Politwoops is a project of the Open State Foundation, based in the Netherlands).

Behind the scenes, it works pretty much the way you’d expect: with a list of politicians’ Twitter handles for each country where it’s running.

But… who doesn’t want to add something extra for free? Our data also includes Twitter handles (mostly but not entirely from the same public sources Politwoops were using). So that meant they could take our CSVs and match each line—all that extra data!—with the Twitter handle.

Better, for free

So last year, they augmented their data with ours for one very simple win: they get to know party affiliation for the politician associated with each of those twitter handles. Well, actually they get to know lots of other things besides party — gender, date of birth, or… well, all our other data, if they wanted it. But just party? That’s also fine.

This all means that Politwoops now shows the party of each tweet’s deleter, just because they merged our CSV with theirs. Lovely!

A tiny detail… party affiliation (arrowed) added to @deletedbyMPs tweets [screenshot of https://politwoops.co.uk/]

Although party affiliation was the detail Politwoops went for, it turns out the other data from EveryPolitician was a little too tempting for them to ignore… So recently they’ve been doing some playful analysis on their statistics using the gender breakdown that EveryPolitician data makes possible too. You can see more on the Politwoops website.

You can too

To be clear: Politwoops did this, not us. We’re committed to doing the groundwork of finding, collecting and collating the data, and making it available (and, additionally, endlessly checking for updates… if you’re interested in how this all works, you can read our bot’s own blog). We do this so people who want to get on with using the data can do just that. As did, in this case, Breyten and his team at Politwoops.

EveryPolitician’s data is available as plain CSVs for this kind of thing, but we also provide a richer JSON version too if that’s more useful to you. All the files are downloadable from the website. If you’re a coder who wants to dive in, there are libraries to make it even easier for you (the EveryPolitician team works in Ruby, so we wrote the everypolitician gem, but there are also ports to Python and PHP).

For more information see the docs.everypolitian.org.

The EveryPolitician bot wrote its own version of this blog post, which goes into a little more detail of the process.

—
Image: Tom Maladova (CC by-nc-nd/2.0)

We were glad to see this tweet back in July, when @adebradley identified WriteToThem as the place to go for information on how to write to your MP. We do try to make that process as easy as possible, so it was a fair assumption that we’d have such a template¹.

But in fact, it was also a mistaken assumption, although we do have some more general advice in our FAQs. Basically, we offer lots of help on how to use our service, but we assume that the user can manage perfectly well once they’ve got to the ‘compose’ screen.

So I did what I always do when a user points out a ‘nice to have’ feature for one of our sites: I ticketed it on Github, our issue-tracking system. And then, when I got round to it, I sat down and did some thinking, and read some other websites which offer advice on writing to your MP.

And then I created a template to show people how to compose a letter that would be clear, readable, and likely to get a result.

But…

As I was doing so, something felt wrong.

Firstly: who was going to visit this template? Even if we linked to it from the FAQs, would anyone ever find it? We know (without having to check our analytics, merely from the kind of messages we often get in our support mail) that the FAQs are not universally read. They’re more widely read since we moved the ‘Help’ link to the top right of every page, but all the same, it seems many users would rather drop us a line than find the answer on an FAQs page.

Then secondly: my template began to feel very patronising. Here was I, someone whose job is copywriting, handing out tips to… well, who? Presumably, our more educated, literate, eloquent users were not staring at a blank screen wondering how to begin a message to their MPs.

No: the people who need help writing to their MPs are going to be people who find it hard to express themselves in writing, and probably have never contacted their representatives before. And they are also the people least likely to wade through my sanctimonious examples and admonishments about what kind of language to use.

So, what now?

I took the issue to my colleagues, who were very helpful in sorting through this thinking. One of them led me to this link, which underscored the uneasiness over whether anyone ever reads FAQs, with wisdom like:

FAQs are convenient for writers […] But they’re more work for readers.

And between us, we reached the conclusion that the problem of people not knowing what, or how, to write to an MP wasn’t going to be solved by copywriting after all: if it was going to be solved at all, it was going to be with design and development.

If we were really going to help our users, it’d have to be right there on the page, at the moment when they get stuck.

Just as FixMyStreet gives discreet tips about what kind of content is appropriate in a report, WriteToThem might also guide a user to start with a clear statement about what the writer wants or needs, and to follow up with concise details. Or it might detect bad language and alert the user that their message is likely to disappear into an MPs’ anti-abuse filter. Maybe we could have an optional template within the ‘compose’ box which could be toggled on or off.

We haven’t got any further than that yet, and we promise not to build the 21st century equivalent of Clippy — but what started with a tweet may end up as some in-browser guidelines.

Meanwhile, if you’re looking for some help writing to your MP, here are our under-visited FAQs, and here’s an excellent guide to writing to your MP from Open Rights Group.

Footnote

¹ It’s probably worth clarifying that, when we talk about templates for letters to MPs, we are not talking about the sending of identikit messages – rather, we mean guides as to what sort of content to include. We have always, and will always, encourage users to write in their own words, and block mass messages from those who don’t. Here’s why.

—
Image: Grant Hutchinson (cc by-nc-nd/2.0)

All this week, we’ll be celebrating International Right to Know Day and the 250th anniversary of Freedom of Information with some insights from journalists who have used FOI in their work.

Here in the UK, two names are particularly linked to FOI: Professor Heather Brooke, the investigative journalist who is responsible for the publication of MPs’ expenses, and Martin Rosenbaum, the BBC’s FOI correspondent.

Today we hear from Heather about the importance of FOI and how she’s used it, and tomorrow you can read Martin’s views.

Heather Brooke

I took two important FOI cases through the legal appeals process: one seeking the minutes to a BBC Board of Governors Meeting after the Hutton Inquiry¹, and my notable legal victory against the House of Commons for details of MPs’ expenses².

This victory in the UK High Court fundamentally changed law and policy, and for the first time in its history Parliament had to account to an outside body over how MPs’ claimed expenses.  The court ruling and subsequent leak of the data led to a number of high-level political resignations as well as full-scale reform of the parliamentary expense regime and passage of the Recall of MPs Act 2015. A new government was elected in May 2010 on a mandate of transparency in part due to the scandal

I made extensive use of the UK’s Freedom of Information Act, filing about 500 FOIs and writing some 60 newspaper and magazine articles about the law and its impact on democracy from 2005-2010. I used the law to map and monitor public bodies for the first time in a citizen-friendly way in Your Right to Know. Through FOI I was able to flag up current and future problems such as secrecy in food safety regulation, the postcode lottery for criminal justice, the amounts police spend on public liability claims and propaganda.

Freedom of Information, rooted in Enlightenment values, contains within it a key principle of democracy that there must be access to information (and knowledge) for all equally. My approach in my 25-year journalistic career has been to use FOI as a means of testing the promise and practice of democracy.  By their responses to FOI requests, we see how agencies truly think about citizens’ rights to access and participate in the political system.

Read the next installment to learn how Martin Rosenbaum’s use of FOI has underpinned hundreds, if not thousands, of news stories at the BBC.

If you’re a journalist yourself, you might be interested in our latest project.

But don’t forget, FOI isn’t just for journalists: you can make your own requests for information at WhatDoTheyKnow.com.
—

¹Guardian Newspapers Ltd and Heather Brooke v IC and the BBC (2007) EA/2006/0011; EA/2006/0013
²Corporate Officer of the House of Commons v Information Commissioner & Heather Brooke, Ben Leapman, Jonathan Michael Ungoed-Thomas [2008] EWHC 1084 (Admin) (16 May 2008)

Images: Cameramen at the Hutton Inquiry by Ben Sutherland CC BY-2.0; Heather Brooke by Paul Clarke CC BY-SA 3.0 via Wikimedia Commons.

Today we’ve added twelve new topics to the voting pages for every MP on TheyWorkForYou. Covering important areas such as membership of the EU, mass surveillance of data, and military action against Daesh/ISIL, these voting lines will help give an even better at-a-glance picture of what your MP stands for, and how that was reflected in Parliament.

Check your own MP’s votes on these topics by inputting your postcode on the TheyWorkForYou homepage:

and then clicking on the ‘voting record’ tab:

You’ll see all your MP’s voting lines laid out in several topic areas on a single page:

From there, it’s easier to explore further by clicking ‘show votes’, which will take you to a page listing every vote taken into consideration when calculating the MP’s stance.

For a full explanation on the methodology behind these pages — a combination of manual and automated inputs — see our previous blog post.

How do we choose which topics to include? It’s all driven by what MPs have voted on in Parliament, and consequently how much data there is to draw from in any specific subject area (there are also a few ‘topics in waiting’: areas we know we want to cover, but where there need to be some more votes before we have sufficient data to present meaningful stances).

New topics in full

The new voting lines we’ve added are:

• In Constitutional Reform
  • Reducing the voting age
• In Home Affairs
  • Mass surveillance of people’s communications and activities
  • Merging police and fire services under Police and Crime Commissioners
• In Transport (a new topic area)
  • High speed rail network
  • Greater public control of bus services
  • Publicly-owned railways
• In Foreign Policy & Defence
  • UK membership of the EU
  • The right to remain for EU nationals
  • Military action against the group known as ISIS, ISIL or Daesh
• In Taxation & Employment
  • The reduction of Capital Gains Tax
• In Housing (a new topic area)
  • Secure tenancies for life
  • Charging a market rent to high earners renting a council home

We hope this helps you see, even more clearly, the effect your MP has had on a broad range of national and international issues.

Don’t forget that we also offer the chance to discuss these votes (or any other issue) with your MP. Just click the ‘send a message’ button at the top of the page to go to WriteToThem.com, where you can compose a message to your representatives quickly and easily.

—

Image: © UK Parliament/Jessica Taylor

Even official records aren’t as safe as you might think they are. The archive of a country’s political history might be wiped out in a single conflagration.

Take the example of Burkina Faso, a beautiful West African country that is, sadly, perhaps best known to the rest of the world for its troubled political past.

The uprising in Burkina Faso in 2014 led to a fire in the National Assembly building and archives office. Nearly 90% of the documents were lost. Now the National Assembly is working to reconstruct the list of its parliament’s members before 1992.

Teg-Wende Idriss Tinto of Open Knowledge is appealing for anyone with such data to assist.

This means that the data EveryPolitician has on Burkina Faso has nothing from terms before 1992. We’ve got some data for six of the seven most recent terms from the National Assembly so far, of which five are live on the site. Even though that data is not very rich (there’s little more than names in many cases; and the 6th term was transitional so data on that one’s membership might remain elusive) it’s a beginning.

We know from experience that data-gathering often proceeds piecemeal, and names are always a good place to start.

As Tinto finds new data, whether that’s more information about the politicians already collected or membership lists of the missing terms before 1992, we’ll be adding that to EveryPolitician too.

A vast collection

When people ask what EveryPolitician is, we often say, ‘The clue’s in the name’. EveryPolitician aims to provide data about, well … every politician. In the world.

(We’ve limited our scope — for the time being — to politicians in national-level legislatures).

The project is growing. Since our launch last year, we’ve got data for legislatures in 233 countries. The amount of data we’ve collected currently comprises well over three million items. The number of politicians in our datafiles is now in excess of 70,000.

Seventy thousand is an awful lot of politicians.

In fact, if you think that might be more politicians than the world needs right now, you’re right: as the Burkina Faso example shows, EveryPolitician collects historic data too.

Here are two more examples: we’ve got data from Germany’s Bundestag, going back to 1949. Or Turkey’s Grand National Assembly, going back to 1920.

So as well as the people serving in today’s parliaments, our data includes increasing numbers of those from the past. (Obviously, if you have such data for your country’s legislature, we’d love to hear from you!)

More than just today’s data

The Burkina Faso fire is an illustration of the value of collecting and preserving this historic data.

Of course, we’re fully aware of the usefulness of current data, because we believe that by providing it we can seed many other projects — including, but in no way limited to, parliamentary monitoring sites around the world (sites like our own TheyWorkForYou in the UK, or Mzalendo in Kenya, for example).

Nonetheless, we never intended to limit ourselves to the present. By sharing and collating historic records too, we hope to enable researchers, journalists, historians and who-knows-who-else to investigate, model, or reveal connections and trends over time that we haven’t even begun to imagine. We know this data has value; we look forward to discovering just how much value.

But it turns out we’re providing a simpler potential benefit too. EveryPolitician’s core datafiles are an excellent distributed archive.

Future-proofing

What Burkina Faso’s misfortune goes to show is that, as historians know only too well, data sources can be surprisingly fragile.

In this case the specific situation involves paper records being destroyed by fire. That is a simple analogue warning to the digital world. Websites and their underlying databases are considerably more volatile than the most flammable of paper archives.

Database-backed sites are often poor catalogues of their pasts. Links, servers and domain registrations all expire. Access to data may be revoked, firewalls can appear.

Digital data doesn’t fade; instead it is so transient that it can simply disappear.

Of course, we cannot ourselves guarantee that our servers will be here forever (we’re not planning on going anywhere, but projects like this have to be realistic about the longer view).

There is an intriguing consequence of us using GitHub as our datastore. The fact is, the EveryPolitician data you can download isn’t coming off our servers at all. Instead, we benefit from GitHub’s industrial-scale infrastructure, as well as the distributed nature of the version control system, git, on which it is based. By its nature, every time someone clones the repository (which is easy to do), they’re securing for themselves a complete copy of all the data.

But the point is not necessarily about data persisting far into the next millennium — that’s a bit presumptuous even for us, frankly — so much as its robustness over the shorter cycles of world events. So, should any nation’s data become inaccessible (who knows? for the length of an interregnum or civil war, a natural disaster, or maybe just a work crew accidentally cutting through the wrong cable outside parliament), we want to know the core data will remain publicly available until it’s back.

Naturally there are other aspects to the EveryPolitician project which are more — as modern language would have it — compelling than collecting old data about old politicians. But the usefulness of the EveryPolitician project as a persistent archive of historical data is one that we have not overlooked.

Image credit: Gorum Gorum Market in Burkina Faso by Cordelia Persen CC BY-NC 2.0

If you’ve used WriteToThem, you’ll know that two weeks after you submit a message to your MP, we send a follow-up questionnaire to check whether you received a response.

Each year, we collate that data to see how MPs are doing at responding to constituents’ mails*, and we publish the results. (This year, we waited a bit longer than usual so that we could cover a full year since the general election.)

They’re now live, so you can go and check exactly how your own MP did — just enter your postcode.

Some interesting stats

• Because we’ve been running these figures since 2005 (with a gap between 2008-13), we can make some comparisons. We’re disappointed to see that the responsiveness rate of MPs has been steadily declining. In 2005, 63% of respondents indicated that they’d had a reply; this year, that’s down to 50%.
• Before we analysed the data, we thought that new MPs, elected in 2015, would perhaps perform better than the jaded incumbents. Not so: on average ‘old’ MPs responded to 53.07% of constituents’ messages, while the newly-elected managed only 46.10%. One new MP, Marcus Fysh, MP for Yeovil, came in at 635 out of the 642 MPs eligible for inclusion.
• Receiving more mail doesn’t necessarily mean you’ll perform poorly. Notable in this respect is Gerald Kaufman, who managed a 79% responsiveness rate despite having the second largest postbag.
• And being in the public eye doesn’t necessarily impact an MP’s responsiveness: Sadiq Khan and Jeremy Corbyn performed poorly, but have done so in prior years, too. Equally, we suppose it follows that a poor responsiveness level doesn’t necessarily impact on electoral success.
• We were curious to know whether there’s a gender divide when it comes to responsiveness. There is, but it’s very slight: on average male MPs responded to 52% of correspondence; female MPs to 50%.
• And another thing we’ve been asked about, sometimes by MPs themselves. There is no significant relationship between parliamentary constituency size and responsiveness. In other words, having more people in a constituency does not automatically mean that the MP is a poor responder.

Anyway, enough of this — go and check how your MP did, and then tell everyone else to do the same.

—

*This needs a caveat. Our data only covers messages sent via WriteToThem, and, furthermore, only those messages where users completed the questionnaire. You can see the full methodology on the rankings page.

Well, it certainly all happened over the weekend: the resignation of one Secretary of State on Friday and the quick appointment of another by Saturday.

It all left a lot of people wondering just who this Stephen Crabb fellow was, and what he stood for.

Fortunately, there’s a very handy website where you can look up the details, debates and voting records of every MP — we refer, of course, to our very own TheyWorkForYou. Over the weekend, we saw the link to Crabb’s voting record shared across social media (and even good old traditional media; we were also mentioned on Radio 4’s Any Answers). Naturally, most interest was around Crabb’s voting habits when it comes to welfare and benefits.

The upshot of this was that TheyWorkForYou saw almost three times our normal traffic for a Saturday. Over the weekend, 30% of all page views were for Crabb’s profile or voting records. In contrast, just 1.83% thought to check out his predecessor’s record: yesterday’s news already, it seems.

So Stephen Crabb’s the new guy, and you may want to keep up to date with his contributions to Parliament. Sign up here and we’ll send you an email every time he speaks.

—

Image: Number 10 (CC)

For verified, reliable information, it’s usually best to go to the official source — but here’s an exception.

Parliamentary monitoring website Mzalendo, which runs on mySociety’s Pombola platform, carries more accurate MP data than the official Kenyan Parliament site.

Checking parliament.go.ke‘s list of MPs against Mzalendo’s, our developers discovered a large number of constituency mismatches. These, explained Jessica Musila from Mzalendo, came about because the official site has not reflected boundary changes made in 2013.

Even more significantly, the official parliament site currently only holds details of 173 of the National Assembly’s 349 MPs.

“The gaps in www.parliament.go.ke validate Mzalendo’s very existence,” said Jessica. We agree: it’s a great example of the sometimes unexpected needs filled by parliamentary monitoring websites.

And of course, through EveryPolitician, we’re working to make sure that every parliamentary monitoring website can access a good, reliable source of data.

—

Image: Richard Portsmouth (CC)

South African parliamentary monitoring website People’s Assembly have added an Attendance page, allowing citizens to see at a glance what percentage of committee meetings each MP has attended.

A few weeks ago, we highlighted one major difference between the Ghanaian parliament and our own: in Ghana, they register MPs’ attendance.

This week, we received news of another of our partners who are holding their representatives to account on the matter of attendance: People’s Assembly, whose website runs on our Pombola platform. The new page was contributed by Code4SA, who have been doing some really valuable work on the site lately.

According to South Africa’s Daily Maverick, in some cases MPs’ attendance is abysmally low. There’s also a history of those who “arrive, sign the register and leave a short while later”, a practice that may soon be on the decline thanks to People’s Assembly’s inclusion of data on late arrivals and early departures.

With 57 representatives — or about 15% — floundering at a zero rate of attendance, it seems that this simple but powerful display is a much-needed resource for the citizens of South Africa. See it in action here.

—

Top image: GovernmentZA (CC)

We recently added an Environment section to voting pages on TheyWorkForYou, so now you can see exactly how your MP voted on issues like fracking, measures to prevent climate change, and green energy, all in one place, like this:

Votes on environmental issues are clearly a priority for our users. They’ve been one of the most-requested additions in the TheyWorkForYou postbag over the last couple of years, and we’re glad to have fulfilled those requests, even if it took a while.

At the same time, we’ve also made several other additions to existing sections on voting pages, so now you can check how your MP has voted in these areas:

• Assisted dying
• Trade union regulation
• Taxation of banks
• Enforcement of immigration rules
• MPs’ veto over laws only affecting their part of the UK (AKA English votes for English laws)

To check your own MP’s voting record, head over to TheyWorkForYou.com, and input your postcode on the homepage. Then click ‘voting record’ at the top of your MP’s page.

If you have strong opinions about how your MP voted on any issue, don’t forget, you can let them know by clicking on ‘Send a Message’, which will take you over to WriteToThem.com.

—

Image: Paul (CC)