Using EveryPolitician data: Politwoops

Politwoops tracks politicians’ tweets, and reports the ones that are deleted.

Often those tweets are deleted because of a typo: everyone makes simple mistakes with the buttons on their devices, and politicians are no less human than the rest of us.

But Politwoops’s targets are public servants who use Twitter to communicate with that public. And sometimes the contents of the tweets they delete are not simply the result of bad typing. Those tweets can be especially interesting to people whom those politicians are representing: sometimes they may be evidence of a usually-suppressed prejudice, or an attempt to remove evidence of a previously held opinion that is no longer convenient.

In effect, Politwoops is a public archive of direct quotes that would otherwise be lost.

And also… EveryPolitician

Our EveryPolitician project is an ever-growing collection of data on every politician in the world (we’re not there yet, but we’re over 230 countries and 72,700 politicians in, and counting).

Like Politwoops, our data includes politicians’ Twitter handles. But also a lot more besides.

We make that data useful by putting it out in consistent, simple formats — the simplest of which is a comma-separated value (CSV) file for each term of a legislature. In practice, that means if you want a spreadsheet of the current politicians in your country’s parliament, then EveryPolitician is probably the place for you.

Put them together…

Now, Politwoops predates EveryPolitician by several years, and they’ve being doing their thing without needing our data just fine. In fact, Politwoops has been happily politwooping since 2010 (Politwoops is a project of the Open State Foundation, based in the Netherlands).

Behind the scenes, it works pretty much the way you’d expect: with a list of politicians’ Twitter handles for each country where it’s running.

But… who doesn’t want to add something extra for free? Our data also includes Twitter handles (mostly but not entirely from the same public sources Politwoops were using). So that meant they could take our CSVs and match each line—all that extra data!—with the Twitter handle.

Better, for free

So last year, they augmented their data with ours for one very simple win: they get to know party affiliation for the politician associated with each of those twitter handles. Well, actually they get to know lots of other things besides party — gender, date of birth, or… well, all our other data, if they wanted it. But just party? That’s also fine.

This all means that Politwoops now shows the party of each tweet’s deleter, just because they merged our CSV with theirs. Lovely!

A tiny detail...  party affiliation (arrowed) added to @deletedbyMPs tweets [screenshot of https://politwoops.co.uk/]

A tiny detail… party affiliation (arrowed) added to @deletedbyMPs tweets [screenshot of https://politwoops.co.uk/]

Although party affiliation was the detail Politwoops went for, it turns out the other data from EveryPolitician was a little too tempting for them to ignore… So recently they’ve been doing some playful analysis on their statistics using the gender breakdown that EveryPolitician data makes possible too. You can see more on the Politwoops website.

You can too

To be clear: Politwoops did this, not us. We’re committed to doing the groundwork of finding, collecting and collating the data, and making it available (and, additionally, endlessly checking for updates… if you’re interested in how this all works, you can read our bot’s own blog). We do this so people who want to get on with using the data can do just that. As did, in this case, Breyten and his team at Politwoops.

EveryPolitician’s data is available as plain CSVs for this kind of thing, but we also provide a richer JSON version too if that’s more useful to you. All the files are downloadable from the website. If you’re a coder who wants to dive in, there are libraries to make it even easier for you (the EveryPolitician team works in Ruby, so we wrote the everypolitician gem, but there are also ports to Python and PHP).

For more information see the docs.everypolitian.org.

The EveryPolitician bot wrote its own version of this blog post, which goes into a little more detail of the process.


Image: Tom Maladova (CC by-nc-nd/2.0)