My colleague Alex has already written about looking forward from this election, so here I am going to look back at the technical work that was involved for the election, and in getting all the new MPs into TheyWorkForYou.

Boundary changes

This election was the first UK Parliament election with boundary changes since 2010. Due to the long-running nature of TheyWorkForYou, which has been around now for over 20 years, this can throw up some interesting challenges. In this particular case, it turned out we were using two different JSON data lists of constituencies – both containing the same data, but one also included the other Parliaments and Assemblies, whilst the other included alternative names for some constituencies. I took the opportunity presented to merge these together and update the bits of code to use the one consolidated dataset, and then added in the 650 new constituencies to the JSON data.

Loading the new constituency data into TheyWorkForYou then threw up another historical problem – the constituency table was still using the very old Latin-1 character set encoding, rather than a more modern encoding such as UTF-8, that almost everything we have uses. This had been fine until now, with even Ynys Môn covered by that encoding, but the new constituency of Montgomeryshire and Glyndŵr contained a letter that Latin-1 could not cope with, leading to a quick emergency upgrade of the table to UTF-8 (thankfully this is a backwards compatible encoding, so worked without issue).

We had already generated data of the new constituencies and loaded these into our lookup service MapIt before Christmas. Ordnance Survey more recently published the official dataset of the boundaries, which we could then import via our usual processes, though even this raised a small issue to be resolved. It turned out in the last data release OS had given the parts of two county council electoral divisions with detached parts (Lightwater, West End and Bisley and Thorpe St Andrew) different identifiers, which they had reverted in their new release, causing our import script to get a bit confused – resolved with a small manual script.

Displaying on TheyWorkForYou

In the period before the election, we knew people would be using our site as a postcode lookup, perhaps to look up their previous MP but perhaps also expecting something useful for the upcoming election, which we wanted to provide, and so we used Democracy Club’s API to show election candidates and link to their WhoCanIVoteFor and WhereDoIVote services. We also displayed your boundary changes using the new constituency data mentioned above.

TheyWorkForYou isn’t just the UK Parliament, though, it also covers the Scottish and Welsh Parliaments, and the Northern Ireland Assembly, so we also had to maintain the provision of that information to people – email alerts for those bodies continued throughout as usual, and the postcode lookup kept showing people their representatives in the devolved nations.

Once the election closed, we automatically updated our messaging, and the next day switched back to our normal behaviour of taking you directly to your MP page in England, and showing you your MP and other representatives elsewhere.

We had a fun issue where some people were getting their new MP, whereas some were getting the old MP – during the period of dissolution, when there are no MPs, we have a configuration flag to enable the site to know it should return the latest result even if it’s not current (you don’t want this all the time, when e.g. an MP has resigned or died), but once new data was being loaded in, one database query was returning results in a random order; fixed by adding some sorting by descending end date.

Election result data

At the last election in 2019, we took a live feed of election results from Democracy Club, who have collected all the candidate information for their Who Can I Vote For service – which all began as the result of a mySociety project back in 2010.

Democracy Club were performing the same service this time, and gratifyingly it was quite a small change to have our 2019 code work with any 2024 changes to the source information (incidentally, there aren’t a lot of narrative doctests in our codebase, but I quite like the one in use there!).

This script would do half the job, of taking in some source data (who has been elected, and including their TheyWorkForYou identifier if they already had one due to being a previous representative of some sort) and amending our source JSON data to add the newly elected representative.

The other half is loading that source data into the TheyWorkForYou database for display on the site. Our normal loading script works fine, but looks through all the source data to see if there have been any changes to take account of. For the election, we don’t need it to do all that, so I tweaked the script to only do the minimal necessary to load in newly created information.

These two scripts were then added to a cron on our server, running every few minutes through the night. I did stay up long enough to check that the first few worked okay, before leaving it to itself from then on. I also set it up to pipe its output to our Slack channel, so people could see it operating:

This also meant as the final few trickle through, it’s popping up reminding us it’s still doing its job:

All the results (bar the one we’re still waiting for) are now committed to the repository, joining all our other open data.

Support TheyWorkForYou and our work

TheyWorkForYou and WriteToThem are run by mySociety, a small UK charity. We’re a very efficient operation and do a lot with a small team; if we had bit more money, we could achieve a lot more.

We want to see a transparent, resilient democracy, with equal access to information, representation and voice for citizens. If you believe in this vision please donate today to enable greater transparency and accountability of the next government.

—

Image: Moritz Kindler

This a guest blog by Joe Mitchell from Democracy Club, a non-profit whose aim is to create the digital foundations to support everyone’s participation in democratic life.

The TL; DR

Democracy Club has produced an election information widget that you can add to any website. It’s free. It provides candidate and, where we have it, polling location information for any postcode.

Check it out:

The backstory

As you may be aware, the UK Parliamentary General Election will take place on 12 December.

You may be less aware that no public body takes responsibility for accurate, locally relevant digital information about elections.

Local governments publish election and candidate data as PDFs on their websites (or, in one notorious case, simply printed out and stuck up on the noticeboard outside the council offices).

Polling location information is printed on cards and sent, in theory, to every voter. It sometimes never arrives. And good luck if you live in a house of multiple occupancy, if you struggle to read the print, if you lose it or if you don’t have it with you when you need to refer to it.

Digital technology has massively improved access to information in many other areas of our lives, so a group of volunteer developers and digital types got together to try to apply the approach to elections in the UK.

That group formed Democracy Club, which is now several thousand volunteers and a small core team constantly working to bring together election, candidate and results data. We also work with local government to aggregate local polling location data and make it available online.

The candidate data we produce is published openly and is used by news media, campaign organisations and, ultimately, voters to learn more and participate in the campaigns. The polling location data we produce is available via an API.

Not everyone has the time to develop a stand-alone product with our data. So we produced a polling location finder widget, which has been popular among local newspapers and local councils. Today, we’ve introduced a widget which includes  candidates data too. Users pop in their postcode and away they go. They can click through to a candidate’s page on WhoCanIVoteFor.co.uk for more information.

The civic need for this information is clear. At the last general election, polling location data was accessed (via our website, The Electoral Commission’s website or via the widget) over 1.8m times at the last general election. Candidates information was accessed over 1m times, but this doesn’t count all the uses powered by a one-off CSV download.

The fact that this information — increasingly critical to our functioning as a democratic society — is managed and produced by a tiny non-profit is not a ringing endorsement of our democratic institutions’ fitness for the 21st century.

Democracy Club is working hard to convince public bodies to take on the basic open data elements of our work: when are elections happening, for which area, who are the candidates, what is their preferred contact method, what were the results, etc. And civic user needs go beyond elections — we can’t get to a world of user-friendly, accessible information about democratic processes until the raw data exists for local democracy too.

But for now, at least there’s a widget.

Let us know what you think! You’re welcome to hop into the Democracy Club Slack — or reach out via Twitter or email.

—

Image: Justgrimes (CC by-sa/2.0)

If you’re a user of mySociety’s Democracy-focused sites, such as TheyWorkforYou and WriteToThem, you may notice a few changes during the election period.

Generally speaking, the sites just work. Sure, there are a bunch of tasks we’re managing on a daily basis behind the scenes, but none of those need bother you, the user. To employ a tired old metaphor, the sites glide swanlike, while under the water there’s some busy paddling to ensure that the latest debates, votes and representatives’ contact details are all present and correct.

During an election, though, that paddling becomes a bit more visible, and some services may be interrupted.

WriteToThem

You want to contact your MP? Here’s the thing: officially, you don’t have one at the moment.

Parliament has dissolved. The representatives formerly known as MPs are no longer allowed to refer to themselves as such, and their parliamentary email addresses have been withdrawn.

So when you visit WriteToThem, you’ll see this message where we normally provide the link for writing to your MP:

Note that you can still use WriteToThem to contact all your other representatives, from local councillors to MSPs, Assembly members, MEPs, etc — provided that your issue is relevant to them (you’ll see a short list of the types of issue each representative deals with, on the site).

If you’ve got something to say about the current political situation or a matter that you’d like your MP to vote on, though, you’ll just have to wait. Even if your former MP is standing for re-election, they’re most likely dedicating a lot of their time to canvassing, and of course they won’t be taking any issues into the debating chamber just now because Parliament is not in session.

Where it becomes a little more tricky is if you have a constituency issue you want an MP to help with. Perhaps consider if it’s something your local councillor/s may be able to help with instead — it’s always worth asking them, anyway. If not, and if it’s an urgent matter, it may be worth calling your former MP’s office, as some (especially those standing for re-election) will still be running a bare bones service.

If your issue is not urgent, then wait until a couple of weeks after the election. In particular, if you find yourself with a brand new MP they’ll be finding their feet, setting up staff and office equipment, etc.

TheyWorkForYou

You’ll see the word ‘former’ used a lot, if you visit TheyWorkForYou over the next few weeks. For example, the homepage generally has a prominent link to direct you towards your own MP’s page. These days, it looks like this:

And if you do click through to any MP’s page, you’ll see that they now have this below their name:

On the page where we list all MPs, you’ll see this factually accurate message at the top:

If you want a list of who the MPs were, it’s still there, you just have to click the link.

And then there’s one more thing: of course, as there are no debates taking place in Parliament, we’re not sending out Westminster email alerts (you’ll still get those from Scottish Parliament and the London Assembly, though).

When will everything be back to normal?

Our friends at Democracy Club collate the election results as they come in, producing data that we can then import. Thanks to them we’re generally able to update TheyWorkForYou pretty much in real time. So, when you wake up in the morning you’ll hopefully be able to:

• Check who your MP is;
• If it’s someone new, sign up for alerts so you get an email when they speak.

For a little while, of course, new MPs will have very little content on their pages: you’ll see a message to say that data will start to appear once they’ve done a bit more.

WriteToThem takes a little longer to get back up to speed: that’s because we need to import all the MPs’ email addresses, and these can take a while to come through. If we’re using an official parliamentary email address, experience shows that they may not even be set up by Parliament for a short while.

So please be patient — as we mentioned earlier, it’s probably best to wait a couple of weeks before contacting your brand new MP in any case.

Meanwhile…

While mySociety sites are fully operational in the periods between elections, there are other organisations who swing into action and do their best work during this time.

So here are a few things you can do, thanks to those other orgs, while you wait for mySociety’s democracy services to return to normal.

• Visit WhoCanIVoteFor and WhereDoIVote from Democracy Club to discover who your local candidates for the General Election are, what they stand for, and where to find your nearest polling station.
• Upload scans of the political mailouts coming through your door to ElectionLeaflets, and help build a permanent archive of promises that elected representatives can be held to account for further down the line.
• Get the Who Targets Me extension on your browser to see clearly who is behind the political ads you’re being served on Facebook.

And finally: if you have questions about the whole electoral process, read the beginner’s guide to the UK General Elections we put together in 2017. While the names and dates have changed since then, the facts are still the same.

—

Image: Reproduced with the permission of Parliament

This week, we heard from a user whose MP’s agent had threatened to take him to court if he shared an image, showing TheyWorkForYou data, on social media. Here’s what we think of that.

Available to all

Prior to an election, you’ll see all sorts of messaging trying to turn you towards, or against, specific candidates — some from political parties, some from independent campaigning groups, and some just from individuals who feel strongly.

At mySociety, we’re non-partisan: we strive for neutrality in our websites and services, and they are available to everyone, no matter which part of the political spectrum you are on. We won’t tell you how to vote; we will, however, present the facts and give you the tools that allow you to make up your own mind.

When we looked into the image our user had wanted to share, we found that there are many similar images, generated from a single source, using TheyWorkForYou voting data to highlight the voting records of Conservative MPs in marginal seats. Here’s what they look like:

And for political balance, here’s an image with a similar intent, highlighting a Labour MP’s voting record (but not from this election, sorry: we have been unable to find anything more up to date, but feel free to send us any you’ve seen and we’ll add them to this post):

Share facts

We have no problem with our data being shared in this way, so long as the wording is unchanged, and the source is credited (as clearly it hasn’t been in our latter example). Adding the source benefits everyone, because while top-line statements like these are, of necessity, brief in a shareable image, they are backed up on TheyWorkForYou with links to the actual votes that substantiate them.

As we say, this data is available to anyone, and TheyWorkForYou covers every MP, so there’s no unfair political advantage being gained here. The votes are statements of fact; and indeed there may well be people looking at a list like that and finding that, actually, they quite agree with everything on the list, in which case the image would be having the opposite effect from that intended.

If you read our blog post from yesterday, you’ll know that we’ve recently introduced Facebook and Twitter share buttons to make it super-easy to share any MP’s votes. So, in short: share our stuff. That’s part of what it’s for.

Reliable data

And yet, the user we mentioned had been told by someone working on behalf of the MP’s campaign that he would be ‘taken to court’ if he shared such an image, as it was ‘based on unreliable data’.

All of our voting analyses are based on the official data put out by Parliament, and we do our utmost to ensure that they are fair: while much of TheyWorkForYou’s content is published out via automated processes, we recognise that voting data is too subtle and sensitive to manage in any way other than manually. That’s why all our voting information is painstakingly compiled by hand, in a process we’ve described previously in this post.

MPs do occasionally contact us to question the wording of certain voting topics, and we are always happy to explain how we arrived at them, and improve them if we agree that the votes have been misrepresented.

We would be quite happy to hear directly from the MP in question and to discuss any information which they perceive as inaccurate: we note that their voting page has been in place on TheyWorkForYou since August 2015 (it has been viewed by over 5,500 people, 67 of them from within the Houses of Parliament) and in that time we have not been contacted with any query.

—
Image: Al King (CC by/2.0)

Would you say you’re pretty clued up about the political system in the UK? If you’re reading this blog, it’s fair to assume that you know a bit more about politics than the average bear.

But that’s not true of everyone who uses our sites, and that’s why we’ve put out a Beginners’ Guide to the General Election.

It’s in three short parts — each takes only 5 minutes or so to read — and they cover the background and timetable of the Election; how to make an informed vote; and finally the logistics of actually casting your ballot.

There’s also a summary countdown so you can make sure you’ve done everything you need to.

Think this is a bit simplistic? Through the emails we receive via User Support, we have a really immediate insight into the common questions, worries and misconceptions around UK politics. Some of them are complex and esoteric. Others make us realise that if we haven’t set out the basics, we’re failing in our aim of making democracy easier for everyone to understand.

Equally, if we step away from the mySociety bubble and dip into social media, it’s easy to see the level of confusion around the pending General Election. Questions I’ve seen recently on my local community’s Facebook page include:

Do we vote twice, once for Corbyn/May and once for our local MP?

You don’t have to join a party to vote for it, do you?

Can I vote in my neighbouring constituency to help my chosen party win, if they’re a dead cert in my own constituency?

These were the real spur towards writing a simple explainer. If you come across friends, family or workmates who have similar queries, we hope you’ll find it useful to share our beginners’ guide.

—

Image: Jay Allen (Crown Copyright)

It’s official, there’s going to be a General Election in the UK on June 8th.

As you might suspect mySociety has lots of tools and services that you might find useful during the campaign whether you just want to find out the voting record of your current MP or if you’re planning on building a website or app to cover the campaign.

TheyWorkForYou

First things first: TheyWorkForYou.com already covers in lots of detail who your MPs are and how they voted. This should be your first port of call so that you can evaluate your incumbent MP, especially when you’re thinking about who to vote for next.

Over the next couple of weeks we are going to make some changes here and there to make relevant parts of the voting record more prominent, and more clearly explain how we calculate the voting records themselves.

If you’re planning on using the data we have in TheyWorkForYou you can access information on UK politicians, parliamentary debates, written answers, and written ministerial statement via our API at theyworkforyou.com/api

Tomorrow we’ll share a blog post explaining in a little more technical detail how to access the API and some advice on how to get the most out of the service.

MapIt

Building a service or website that covers all or part of the country and want an easy way to let your users identify which constituency they are in? Then MapIt is your friend.

It already powers most of our own services and is widely used by the likes of Government Digital Services and our friends at Democracy Club.

You can sign up for for free at mapit.mysociety.org and if you need more calls it’s easy to upgrade to a monthly plan – you can get 10,000 calls a month for free if you are a charity or working on an open project – if you think you are going to be busier than that (a) congrats and (b) drop us an email at mapit@mysociety.org

Helping Democracy Club

Speaking of Democracy Club we’re going to be wholeheartedly supporting their efforts to crowdsource a full set of candidate data in the run up to the election – they are gathering all of their ideas together in this Google Doc https://goo.gl/8WtZvc

We had planned to make some updates and amends to the YourNextRepresentative service that supports Democracy Club’s WhoCanIVotefor.co.uk site in the quiet period between major elections, ahem, but with the snap election called we’ll be doing what we can to make the site run faster and make whatever UI tweaks and fixes we can in the time available.

They will no doubt be looking for help in sourcing candidate data, so please do sign up to help and find out what you can do democracyclub.org.uk/blog/2017/04/18/its-ge2017

In summary and to make it easy you can find all of our relevant #GE2017 datasets and APIs here data.mysociety.org/datasets/?category=ge2017

It’s not too late to let your current MP know what you think on any subject of your choice via WriteToThem.com.

And finally, don’t forget to register to vote yourself at gov.uk/register-to-vote

Image courtesy of Maurice on Flickr.com

So, the results are in. Some of us have a brand new MP. Others will see the same familiar face returning to the benches of Westminster.

Either way, the important questions remain the same:

• What will your MP do in Parliament?
• Will they speak about the things that matter to you?
• How will they vote in your name?

The easy way to keep up

TheyWorkForYou.com makes it very easy to keep check: you can even sign up to receive an email whenever your MP speaks. These are in the form of a daily digest, and we only send them on days when your MP has actually contributed to a debate.

It’s the low-effort way to see exactly what your MP is getting up to, with no spin, just the facts. Click here for our easy sign-up.

All change

If you already receive alerts, but your prior MP has lost their seat, be sure to set up an alert for the new one now. We’ll be sending reminders to all current subscribers.

There’s no need to cancel the previous alert, however: if your old MP isn’t in Parliament, we simply won’t be sending any more emails about them.

—

Image: Parliament Acts by Jeroen van Luin (CC)

Spare a thought for us over the night of May 7th – for, when the nation wakes up to the General Election results, we’ll have been up all night updating TheyWorkForYou.

As you might imagine, elections are bitter-sweet times for us here at mySociety. On the one hand, swingometers, marginals and ballot boxes are about as exciting as life gets for a bunch of political geeks. On the other, we have only a short window of time in which to ensure our parliamentary websites reflect the new administration.

In previous years, this has meant manually updating an XML file and running an import script 650 times – slightly arduous, even for the most dedicated civic coder. This year, we’re taking advantage of the fact that YourNextMP exists and several of us will be staying up anyway to see the results, and hoping to do things a little differently.

As each result is announced (or potentially even earlier, if it’s clear that there’s only one possible winner), site administrators will be logged in to YourNextMP, where they’ll have access to a “this person won!” button. We’ll be on a rota throughout the night, sharing duties with the equally dedicated Democracy Club volunteer team.

When that button is clicked, YourNextMP will update, and TheyWorkForYou will notice and automatically update its underlying JSON data.

This is the data we match you with when you input your postcode on the homepage, meaning that TheyWorkForYou should be a great place to find out who your next MP is as soon as you wake up (assuming the results are in) on 8th May.

If you value this service, please consider donating to keep TheyWorkForYou going.

UPDATE: If you are interested in the technical aspects of the YourNextMP and TheyWorkForYou updates, you may like to read more about it in this thread.

A blank canvas

Note that new MPs will not have a great deal on their pages yet: TheyWorkForYou’s MP pages are built up of voting and debating activity, past positions and expenses, etc, and of course, totally new MPs will have none of that. But there’s one important feature that you should take advantage of on Friday—the ‘subscribe’ button.

Sign up, and we’ll send you an email every time your new MP speaks in Parliament, so you can keep track of exactly what he or she is saying in your name. If you were previously following an MP who has resigned or lost their seat, don’t forget to follow the new one! We’ll be sending out a message straight after the election to remind you.

WriteToThem

Another website which will require a lot of attention post-election is WriteToThem, which matches you with your local and national politicians so that you can contact them.

Unfortunately, WriteToThem takes a little longer to update, as we rely on data, including email addresses, from external sources. We’ll be updating as soon as we can. Meanwhile, if you have an urgent message for your MP or councillors, you may find that you can locate direct email addresses on the official Parliament and council websites.

—

Image: William Murphy (CC)

Have you ever typed a phrase like ‘what’s the time in New York’ or ‘what is 28 km in miles’ into Google? If you’ve done so in the last couple of years, you’ll have seen answers given on the results page itself, inside dedicated answer boxes.

For the next couple of weeks, Google users in Britain who search for key election data will see the answers presented in-page in this increasingly familiar way.

We’re delighted to share the news that these answers are being supplied as open standard data from Democracy Club’s brilliant, volunteer-powered YourNextMP project, as well as our own long running TheyWorkForYou. The aim is to make information easily accessible to anyone who seeks it: we are one of a few data sources to be supplying Google.

We’re particularly excited about this because YourNextMP is powered by PopIt, a Poplus Component which is used to create nice clean open standards Popolo data on politicians.

We think that adoption of open standards data by companies as big as Google points towards a promising world in which there is a lot more good quality, open standard data on political issues of all kinds.

Try it for yourself: search for uk election candidates or a question like who can I vote for?

How was this made possible?

We’re so delighted to see essential civic data being brought to the search engine’s vast numbers of users. It’s all down to the power of open standards and re-usable open source software.

YourNextMP provides feeds via an API, which are available for anyone—large organisations like Google, or individual people like you, perhaps—to use in their own projects.

YourNextMP’s use of the Popolo open standard for government means that the data is clean, machine-readable and easy to slot in anywhere—including Google.

Three cheers

We think a few shout-outs are due. This simple but far-reaching usage of YourNextMP data is only possible because of many good people bringing good things together:

• Democracy Club, and the huge amount of work that this volunteer-run organisation put in to gather and check candidate data
• The many volunteers who gave their time in adding and refining that data
• Popolo, driven by the work of James McKinney
• Poplus, because YourNextMP is based on the people-and-positions storing Component, PopIt
• Google.org, whose funding allowed the creation of Poplus in the first place.

What YourNextMP has achieved

Just as was hoped, YourNextMP data has been used to underpin a variety of projects by many individuals and organisations.

Google may be the latest and the biggest, but we’re no less pleased to see how it has made possible numerous tools to educate or inform the public before the election, as well as powering stories and infographics in several national newspapers. See our previous post for more details on this.

And there’s more. YourNextMP, because it’s built on Open Source code, won’t be going into hibernation until the next election in this country.

The UK voting may be over on May 8, but there are elections all over the world still to come. Our friends in Latin America will be taking the code and adapting it for use in Argentina whose election process starts in August. ¡Viva YourNextMP!

Back in December, we told you about a project to collect the details of every election candidate in the UK— YourNextMP.com.

YourNextMP isn’t a mySociety project. It falls under the wide umbrella of Democracy Club, a loose confederation of volunteers doing interesting digital things, with the overarching aim of helping people be more informed before the election. We have, however, been lending our technical skills.

That database now contains details of every candidate and we’re really glad to see that many projects have been built on the back of it, from national newspaper visualisations to voter advice applications to single-issue sites and more.

Back in December, YourNextMP was a tool for crowd-sourced data-gathering. As well as providing free, open source data via its API, it has now matured into a useful static site in its own right. In a neat virtuous circle, it not only shows you who your candidates are, but also displays feeds from many of the sites using its own data.

What does that mean? Go and input your postcode and you’ll find not only:

• A list showing every prospective parliamentary candidate standing in your constituency, and including links to their Twitter stream, Facebook page, homepage and Wikipedia entry, where possible —

but also:

• Pictures of leaflets which have been delivered to residents in your constituency — from ElectionLeaflets.org, another crowdsourced project which is creating an archive of leaflets from all over the country, to stand as a permanent record of promises made pre-election
• Details of where you can go and see your candidates speak  — from MeetYourNextMP.com, which crowdsources details of hustings in each area
• CVs from your local candidates  — from Democracy Club’s CVs project
• News stories which mention your constituency or candidates  — from electionmentions.com.

In many cases, these sites are just like YourNextMP: they’re relying on the time and energy of people like you, to add information. They’ve all made it as easy as possible though, so whether you fancy snapping an election leaflet on your phone and uploading it, or asking your candidates to provide a CV, it really does only take a couple of minutes.

You can also still continue to add more data (such as email addresses) to the candidates on YourNextMP, if you have time to contribute, and some basic Googling skills.