You may remember that in August this year, mySociety and Open Knowledge International launched a survey, looking for the sources of digital files that hold electoral boundaries… for every country in the world. Well, we are still looking!
There is a good reason for this hunt: the files are integral for people who want to make online tools to help citizens contact their local politicians, who need to be able to match users to the right representative. From mySociety’s site TheyWorkForYou to Surfers against Sewage’s Plastic Free Parliament campaign, to Call your Rep in the US, all these tools required boundary data before they could be built.
We know that finding this data openly licensed is still a real challenge for many countries, which is of course why we launched the survey. We encourage people to continue to submit links to the survey, and we would love if people experienced in electoral boundary data, could help by reviewing submissions: if you are able to offer a few hours of help, please email firstname.lastname@example.org
The EveryBoundary survey FAQs tell you everything you need to know about what to look for when boundary hunting. But we also wanted to share some top tips that we have learnt through our own experiences.
- Start the search by looking at authoritative sources first: electoral commissions, national mapping agencies, national statistics bodies, government data portals.
- Look for data formats (.shp, .geojson, kml etc), and not just a PDF.
- Ask around if you can’t find the data: if a map is published digitally, then the data behind it exists somewhere!
- Confuse administrative boundaries with electoral boundaries — they can be the same, but they often aren’t (even when they share a name).
- Assume boundaries stay the same — check for redistricting, and make sure your data is current.
If you get stuck
- Electoral boundaries are normally defined in legislation; sometimes this takes the form of lists of the administrative subdivisions which make up the electoral districts. If you can get the boundaries for the subdivisions you can build up the electoral districts with this information.
- Make FOI requests to get hold of the data.
- If needed, escalate the matter. We have heard of groups writing to their representatives, explaining the need for the data. And don’t forget: building tools that strengthen democracy is a worthwhile cause.
mySociety is asking people to share electoral boundary data as part of efforts to make information on every politician in the world freely available to all, and support the creation of a Democratic Commons. Electoral boundary files are an essential part of the data infrastructure of a Democratic Commons. A directory of electoral boundary sources is a potential benefit to many people and organisations — so let’s keep up the search!
Photo: Chase Clark
We, and Open Knowledge International, are looking for the digital files that hold electoral boundaries, for every country in the world — and you can help.
Yeah, we know — never let it be said we don’t know how to party.
But seriously, there’s a very good reason for this request. When people make online tools to help citizens contact their local politicians, they need to be able to match users to the right representatives.
So head on over to the Every Boundary survey and see how you can help — or read on for a bit more detail.
Data for tools that empower citizens
If you’ve used mySociety’s sites TheyWorkForYou — or any of the other parliamentary monitoring sites we’ve helped others to run around the world — you’ll have seen this matching in action. Electoral boundary data is also integral in campaigning and political accountability, from Surfers against Sewage’s ‘Plastic Free Parliament’ campaign, to Call your Rep in the US.
These sites all work on the precept that while people may not know the names of all their representatives at every level — well, do you? — people do tend to know their own postcode or equivalent. Since postcodes fall within boundaries, once both those pieces of information are known, it’s simple to present the user with their correct constituency or representative.
So the boundaries of electoral districts are an essential piece of the data needed for such online tools. As part of mySociety’s commitment to the Democratic Commons project, we’d like to be able to provide a single place where anyone planning to run a politician-contacting site can find these boundary files easily.
And here’s why we need you
Electoral boundaries are the lines that demarcate where constituencies begin and end. In the old days, they’d have been painstakingly plotted on a paper map, possibly accessible to the common citizen only by appointment.
These days, they tend to be available as digital files, available via the web. Big step forward, right?
But, as with every other type of political data, the story is not quite so simple.
There’s a great variety of organisations responsible for maintaining electoral boundary files across different countries, and as a result, there’s little standardisation in where and how they are published.
How you can help
We need the boundary files for 231 countries (or as we more accurately — but less intuitively — refer to them, ‘places’), and for each place we need the boundaries for constituencies at national, regional and city levels. So there’s plenty to collect.
As we so often realise when running this sort of project, it’s far easier for many people to find a few files each than it would be for our small team to try to track them all down. And that, of course, is where you come in.
Whether you’ve got knowledge of your own country’s boundary files and where to find them online, or you’re willing to spend a bit of time searching around, we’d be so grateful for your help.
Fortunately, there’s a tool we can use to help collect these files — and we didn’t even have to make it ourselves! The Open Data Survey, first created by Open Knowledge International to assess and display just how much governmental information around the world is freely available as open data, has gone on to aid many projects as they collect data for their own campaigns and research.
Now we’ve used this same tool to provide a place where you can let us know where to find that electoral boundary data we need.
Where to begin
Thanks for your help — it will go on to improve citizen empowerment and politician accountability throughout the world. And that is not something everyone can say they’ve done.
Image credit: Sam Poullain
mySociety was built on its Democracy practice, a pioneer in providing simple- to-use tools that demystify the democratic process, allow citizens to understand how decisions are being made on their behalf and ensure that their voices are heard by elected representatives.
We’ve been on a long journey, from the early days of FaxYourMP which eventually became WriteToThem, to our pivotal TheyWorkForYou service which has both stretched the ambitions of Parliament in the UK and led us to develop similar services in Kenya, South Africa and beyond.
Amidst all of this has been our ongoing push to better standardise and make accessible more Open Data on politicians around the world; initially through our Poplus and Pombola projects, but more recently – and with more success – through our EveryPolitician service which has blossomed into a remarkable dataset of almost 4 million datapoints on over 72,000 politicians in 233 countries and territories.
Despite these successes I don’t think we’ve yet sufficiently cracked the challenge at scale of enabling more organisations to monitor and report upon the work of more politicians in more countries. We need to do something about that.
One of the principles that has always underpinned mySociety is that we carry our work out in the open, freely available for others to use. But, as is common with many Open Source projects, we do most of the development work ourselves internally. While community contributions are very welcome, practicality has dictated that more often than not, these are more commonly directed to raising tickets rather than making changes to the actual code.
Unchecked, this situation could lead to us being too internally focused; on developing everything ourselves rather than recognising where we can achieve our objectives by supporting other projects.
Fortunately our collaboration with Wikidata, announced earlier this year, suggests what promises to be a clear way forward to scaling up the impact of our work: we recognised that EveryPolitician could only become sustainable at scale as part of a wider community effort if we want our data to be used more widely.
By contributing to what we’ll call the Democratic Commons — a concept of shared code, data and resources where anyone can contribute, and anyone can benefit — we can help build and strengthen core infrastructure, tools and data that allow other democracy organisations and campaigners to hold their own governments to account.
This was notably put into practice for the snap General Election in the UK in June, where rather than build something new ourselves we directly supported the work of Democracy Club in their efforts to source candidate data and ensured that our existing services like MapIt, TheyWorkForYou and WhatDoTheyKnow were easily accessible for other campaigning and democracy organisations to put into use.
More recently we’ve established a commercial partnership with Facebook to provide them with accurate and independent lists of candidates and elected representatives matched to their relevant Facebook profile pages for the UK, French and Kenyan elections.
There’s a wider benefit to this kind of commercial work, beyond its being a useful source of additional revenue for mySociety. More importantly, it will allow us to feed the data that we source back into the Democratic Commons. It can contribute to EveryPolitician and Wikidata, and even improve boundary data internationally through OpenStreetMap, which in turn powers our own Global MapIt service.
Why is this important now?
Well, it’s not just the rather obvious observation that working with other people is a good idea. The reality is that we need to face the fact that our Democratic practice is just not fully funded, and, as with WhatDoTheyKnow.com, at best we’ll need to consider how more of our services in the UK can be run and directly supported by volunteers and the wider community.
At worst it’s quite possible that we’ll be forced to close some of our popular UK services and restrict the further development of our democracy work internationally.
In April next year we come to the end of our six-year grant agreement with the Omidyar Network who have given us tremendous support over that time. This will leave a substantial hole in our core funding and it’s one reason why we’ve been so diligently focused on developing appropriate new commercial services like FixMyStreetPro and WhatDoTheyKnowPro.
Without sufficient unrestricted core funding — that is, funding which can be applied wherever in the organisation it is most needed — we need to rely much more on specific project funding wherever we can find it. In most cases, however, this project funding comes with its own set of tasks to deliver, and there’s a tendency to want new shiny things, rather than supporting the maintenance of our existing projects. This is especially true of our Democracy work which relies more heavily on grant funding than commercial alternatives.
Sensibly directing our own work more towards contributions to external projects is also a hedge, should we need to find new homes for our services or shutter them for the time being.
In the meantime we’ll be speaking to more funders who we hope might recognise the importance of supporting and building the essential infrastructure of the Democratic Commons, but in the event that isn’t forthcoming we’ll do what it takes to ensure our work to date continues to have some value and impact.
As we start to map out a path to a sustainable future for mySociety and its community, I’d appreciate all thoughts on where we go next with this — after all, we can’t do this without your help.
Last year, when we were helping to develop YourNextMP, the candidate-crowdsourcing platform for the General Election, we made what seemed like an obvious decision.
We decided to use PopIt as the site’s datastore — the place from which it could draw information about representatives: their names, positions, et cetera. We’d been developing PopIt as a solution for parliamentary monitoring sites, but we reckoned it would also be a good fit for YourNextMP.
That turned out to be the wrong choice.
YourNextMP was up and running in time for the election, but at the cost of many hours of intensive development as we tried to make PopIt do what was needed for the site.
Once you’ve got an established site in production, changing the database it uses isn’t something you do lightly. But on returning to the codebase to develop it for international reuse, we had to admit that, in the words of mySociety developer Mark Longair, PopIt was “actually causing more problems than it was solving”. It was time to unpick the code and take a different approach.
Mark explains just what it took to decide to change course in this way, over on his own blog.
The post contains quite a bit of technical detail, but it’s also an interesting read for anyone who’s interested in when, and why, it’s sometimes best to question the decisions you’ve made.
There are websites built on mySociety code in many countries across the world.
Whatever the site you’re planning, you’ll find it a lot easier with our support and development help.
Our quarterly call for applications closes on October 30, so make sure you have yours in soon. Want to know exactly what’s involved? Start here.
Just how quickly can we hit the 200 countries mark on EveryPolitician? That’s what we’ll be finding out this week, and one thing’s for sure — we’ll get a lot further with your help.
This week is GLOW, the Global Legislative Openness Week, and we’re marking it with a concerted drive for more data.
Tony, the project lead, has consistently added one new country every day since EveryPolitician launched four months ago, and now it’s time to put a rocket behind our efforts.
The site currently contains data for 134 countries. We’ll be going flat out to see how quickly we can reach 200, and as the excitement ramps up, we hope you will help spread the word and get involved, too. Tony will carry on working as hard as he can to fill in the gaps, but we need your help to get further, faster.
What is EveryPolitician?
How can I help?
- Help us find data for more countries! We don’t currently know where to find the politician data for many countries. Here’s a list of the ones we need and here’s a page about how to contribute. If you get stuck, give us a shout.
- Write a scraper If you have the know-how, you can help us enormously by helping scrape the data from the places we do know about. See this page for guidance on how to go about writing a scraper. You’ll find lots of examples here.
- You can also help by spreading the word – tell your friends, tweet, blog, get up on a platform and talk, and just generally share this post. Thank you!
Why do we need this data?
Politician data is readily available for most countries, but it comes in a massive variety of inconsistent formats. Most of those formats aren’t ‘machine readable’, that is to say, the data can’t readily be extracted and re-used by programmers, and pretty much every country differs on what information it provides about each politician.
That being the case, anyone who wants to build an online tool that deals with politicians from more than one country, or who would like their tool to be available to people in other places, or would like to adapt an existing tool to be used elsewhere, would first have to adapt their tool to cope with the data.
EveryPolitician saves them the trouble, and the structured format also means that the tools they build will be compatible with any other tools that use it.
What kind of tools?
EveryPolitician data will be useful for all kinds of projects.
It’ll be much easier to build a website that shows people how to contact a politician. Or one that holds a government to account and educates people about what politicians are doing. Or one that helps voters make choices by displaying facts about what their politicians believe.
It can go further than that, though — with these building blocks in place, developers can really use their imagination to put together all kinds of projects, many of which we haven’t even begun to imagine. And don’t forget that, if a tool has been built to use the standardised data, it’ll also be easy for others to redeploy elsewhere.
If you’d like to see a concrete way in which the data’s already being used, check out Gender Balance.
How can I keep up to date?
We’ll be putting out regular updates via Twitter as the number of countries covered increases — plus you can watch the map turn green on http://everypolitician.org/countries.html as we progress.
As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
If you need data on the people who make up your parliament, another country’s parliament, or indeed all parliaments, you may be in luck.
What’s more, it’s all provided as Open Data to anyone who would like to use it to power a civic tech project. We’re thinking parliamentary monitoring organisations, journalists, groups who run access-to-democracy sites like our own WriteToThem, and especially researchers who want to do analysis across multiple countries.
But isn’t that data already available?
Yes and no. There’s no doubt that you can find details of most parliaments online, either on official government websites, on Wikipedia, or on a variety of other places online.
But, as you might expect from data that’s coming from hundreds of different sources, it’s in a multitude of different formats. That makes it very hard to work with in any kind of consistent fashion.
Every Politician standardises all of its data into the Popolo standard and then provides it in two simple downloadable formats:
- csv, which contains basic data that’s easy to work with on spreadsheets
- JSON which contains richer data on each person, and is ideal for developers
This standardisation means that it should now be a lot easier to work on projects across multiple countries, or to compare one country’s data with another. It also means that data works well with other Poplus Components.
What can I do with it?
Need a specific example? Yesterday, we introduced Gender Balance, the game that gathers data about women in politics.
As you’ll know if you’ve already given it a try, Gender Balance works by displaying politicians that make up one of the world’s legislatures, one by one.
That data all comes from Every Politician, and it’s meant that the developers have been able to concentrate on making a smooth and functional interface, knowing that the data side of things has already been taken care of.
That’s just one way to use Every Politician data, though. If you’d like to use it in your own site or app, you can find out more here.
We still need more data
As you may have noticed, there are more than 100 parliaments in the world. In fact, despite having reached what feels like a fairly substantial milestone, we’re still barely half way to getting some data for every parliament.
So we could use your help in finding data for the parliaments we don’t yet cover, and historic information for the ones we do. Read more about how you can help out.
The Poplus community is spread all over the world—but that doesn’t stop us getting together whenever we can.
Poplus is a worldwide federation of people and organisations with an interest in civic tech. This Friday, we will be holding a virtual Show and Tell, hearing from two very different projects:
- Andrew Mandelbaum from SimSim in Morocco will be speaking about Nouabook. This is an application which enables anyone to contact their politicians in public, through Facebook. It uses the Poplus Component WriteIt.
- Matthew Landauer from OpenAustralia Foundation will be speaking about Cuttlefish, one of the latest pieces of software to be certified as a Poplus Component.
As well as hearing all about these projects, there’ll be a chance to catch up and have a chat about all things Poplus/civic tech. Everyone is welcome.
We’ll be using an online platform called QiqoChat to host this call: sign up here.
You can create a free account using Facebook/Google/LinkedIn/Meetup/Twitter or a regular email address. Instructions for connecting by phone or computer microphone are available when you sign in and click “Participate”.
This Friday, 12 June. Times are as follows:
• 5 AM – US Pacific
• 7 AM – US Eastern
• 8 AM – Chile/Argentina
• 12 PM – UK
• 7 PM – Taiwan and Malaysia
• 9 PM – Sydney
• 11 PM- New Zealand
As you’ll know if you’re a regular reader of this blog, YourNextMP crowd-sourced details of every candidate who stood in the UK general election.
But, just because our own election is over, doesn’t mean we’ll be letting YourNextMP gather dust. On the contrary—we want to see it being re-used wherever there are elections being held, and citizens needing information! We’re already seeing the first re-use case, and we’d love to see more.
Opening up data
YourNextMP’s main purpose was to provide a free, open database of candidates, so that anyone who wanted to could build their own tools on top of it, and it was very successful with that aim.
The traditional source of candidate data for such projects has been through expensive private providers, not least because the official candidate lists are published just a few days before the election.
Thanks to YourNextMP’s wonderful crowd-sourcing and triple-checking volunteers, we reckon that we had the most complete, most accurate data, the earliest. And it was free.
Directly informing over a million citizens
YourNextMP also came into its own as a direct source of information for the UK’s electorate. This hadn’t been the priority when the project was launched, but it was helped greatly by the fact that constituency and candidate pages ranked very highly in search engines from early on, so anyone searching for their local candidates found the site easily.
Once they did so, they found a list of everyone standing in their constituency, together with contact details, links to their online profiles such as web pages, social media and party websites, and feeds from spin-off projects (themselves built on YourNextMP data) such as electionleaflets.org and electionmentions.com.
YourNextMP had more than a million unique users. In the weeks just prior to the UK general election, it was attracting approximately 20,000 visitors per day, and on the day before the election, May 6th, there was suddenly a massive surge: that day the site was visited by nearly 160,000 people.
So, in a nutshell: YourNextMP has not only enabled a bunch of projects which helped people become more informed before our election—it also directly informed over a million citizens.
A reusable codebase
And, in the spirit of Poplus, the codebase is open for anyone to re-use in any country.
It’s already being pressed into use for the upcoming elections in Argentina, and we hope that developers in many other countries will use it to inform citizens, and inspire great web tools for the electorate, when their own elections come around.
If that’s something that interests you, please come and talk, ask questions and find out what’s involved, over on the Democracy Club mailing list.