Last year, when we were helping to develop YourNextMP, the candidate-crowdsourcing platform for the General Election, we made what seemed like an obvious decision.
We decided to use PopIt as the site’s datastore — the place from which it could draw information about representatives: their names, positions, et cetera. We’d been developing PopIt as a solution for parliamentary monitoring sites, but we reckoned it would also be a good fit for YourNextMP.
That turned out to be the wrong choice.
YourNextMP was up and running in time for the election, but at the cost of many hours of intensive development as we tried to make PopIt do what was needed for the site.
Once you’ve got an established site in production, changing the database it uses isn’t something you do lightly. But on returning to the codebase to develop it for international reuse, we had to admit that, in the words of mySociety developer Mark Longair, PopIt was “actually causing more problems than it was solving”. It was time to unpick the code and take a different approach.
Mark explains just what it took to decide to change course in this way, over on his own blog.
The post contains quite a bit of technical detail, but it’s also an interesting read for anyone who’s interested in when, and why, it’s sometimes best to question the decisions you’ve made.
There are websites built on mySociety code in many countries across the world.
Whatever the site you’re planning, you’ll find it a lot easier with our support and development help.
Our quarterly call for applications closes on October 30, so make sure you have yours in soon. Want to know exactly what’s involved? Start here.
Just how quickly can we hit the 200 countries mark on EveryPolitician? That’s what we’ll be finding out this week, and one thing’s for sure — we’ll get a lot further with your help.
This week is GLOW, the Global Legislative Openness Week, and we’re marking it with a concerted drive for more data.
Tony, the project lead, has consistently added one new country every day since EveryPolitician launched four months ago, and now it’s time to put a rocket behind our efforts.
The site currently contains data for 134 countries. We’ll be going flat out to see how quickly we can reach 200, and as the excitement ramps up, we hope you will help spread the word and get involved, too. Tony will carry on working as hard as he can to fill in the gaps, but we need your help to get further, faster.
What is EveryPolitician?
How can I help?
- Help us find data for more countries! We don’t currently know where to find the politician data for many countries. Here’s a list of the ones we need and here’s a page about how to contribute. If you get stuck, give us a shout.
- Write a scraper If you have the know-how, you can help us enormously by helping scrape the data from the places we do know about. See this page for guidance on how to go about writing a scraper. You’ll find lots of examples here.
- You can also help by spreading the word – tell your friends, tweet, blog, get up on a platform and talk, and just generally share this post. Thank you!
Why do we need this data?
Politician data is readily available for most countries, but it comes in a massive variety of inconsistent formats. Most of those formats aren’t ‘machine readable’, that is to say, the data can’t readily be extracted and re-used by programmers, and pretty much every country differs on what information it provides about each politician.
That being the case, anyone who wants to build an online tool that deals with politicians from more than one country, or who would like their tool to be available to people in other places, or would like to adapt an existing tool to be used elsewhere, would first have to adapt their tool to cope with the data.
EveryPolitician saves them the trouble, and the structured format also means that the tools they build will be compatible with any other tools that use it.
What kind of tools?
EveryPolitician data will be useful for all kinds of projects.
It’ll be much easier to build a website that shows people how to contact a politician. Or one that holds a government to account and educates people about what politicians are doing. Or one that helps voters make choices by displaying facts about what their politicians believe.
It can go further than that, though — with these building blocks in place, developers can really use their imagination to put together all kinds of projects, many of which we haven’t even begun to imagine. And don’t forget that, if a tool has been built to use the standardised data, it’ll also be easy for others to redeploy elsewhere.
If you’d like to see a concrete way in which the data’s already being used, check out Gender Balance.
How can I keep up to date?
We’ll be putting out regular updates via Twitter as the number of countries covered increases — plus you can watch the map turn green on http://everypolitician.org/countries.html as we progress.
As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
If you need data on the people who make up your parliament, another country’s parliament, or indeed all parliaments, you may be in luck.
What’s more, it’s all provided as Open Data to anyone who would like to use it to power a civic tech project. We’re thinking parliamentary monitoring organisations, journalists, groups who run access-to-democracy sites like our own WriteToThem, and especially researchers who want to do analysis across multiple countries.
But isn’t that data already available?
Yes and no. There’s no doubt that you can find details of most parliaments online, either on official government websites, on Wikipedia, or on a variety of other places online.
But, as you might expect from data that’s coming from hundreds of different sources, it’s in a multitude of different formats. That makes it very hard to work with in any kind of consistent fashion.
Every Politician standardises all of its data into the Popolo standard and then provides it in two simple downloadable formats:
- csv, which contains basic data that’s easy to work with on spreadsheets
- JSON which contains richer data on each person, and is ideal for developers
This standardisation means that it should now be a lot easier to work on projects across multiple countries, or to compare one country’s data with another. It also means that data works well with other Poplus Components.
What can I do with it?
Need a specific example? Yesterday, we introduced Gender Balance, the game that gathers data about women in politics.
As you’ll know if you’ve already given it a try, Gender Balance works by displaying politicians that make up one of the world’s legislatures, one by one.
That data all comes from Every Politician, and it’s meant that the developers have been able to concentrate on making a smooth and functional interface, knowing that the data side of things has already been taken care of.
That’s just one way to use Every Politician data, though. If you’d like to use it in your own site or app, you can find out more here.
We still need more data
As you may have noticed, there are more than 100 parliaments in the world. In fact, despite having reached what feels like a fairly substantial milestone, we’re still barely half way to getting some data for every parliament.
So we could use your help in finding data for the parliaments we don’t yet cover, and historic information for the ones we do. Read more about how you can help out.
The Poplus community is spread all over the world—but that doesn’t stop us getting together whenever we can.
Poplus is a worldwide federation of people and organisations with an interest in civic tech. This Friday, we will be holding a virtual Show and Tell, hearing from two very different projects:
- Andrew Mandelbaum from SimSim in Morocco will be speaking about Nouabook. This is an application which enables anyone to contact their politicians in public, through Facebook. It uses the Poplus Component WriteIt.
- Matthew Landauer from OpenAustralia Foundation will be speaking about Cuttlefish, one of the latest pieces of software to be certified as a Poplus Component.
As well as hearing all about these projects, there’ll be a chance to catch up and have a chat about all things Poplus/civic tech. Everyone is welcome.
We’ll be using an online platform called QiqoChat to host this call: sign up here.
You can create a free account using Facebook/Google/LinkedIn/Meetup/Twitter or a regular email address. Instructions for connecting by phone or computer microphone are available when you sign in and click “Participate”.
This Friday, 12 June. Times are as follows:
• 5 AM – US Pacific
• 7 AM – US Eastern
• 8 AM – Chile/Argentina
• 12 PM – UK
• 7 PM – Taiwan and Malaysia
• 9 PM – Sydney
• 11 PM- New Zealand
As you’ll know if you’re a regular reader of this blog, YourNextMP crowd-sourced details of every candidate who stood in the UK general election.
But, just because our own election is over, doesn’t mean we’ll be letting YourNextMP gather dust. On the contrary—we want to see it being re-used wherever there are elections being held, and citizens needing information! We’re already seeing the first re-use case, and we’d love to see more.
Opening up data
YourNextMP’s main purpose was to provide a free, open database of candidates, so that anyone who wanted to could build their own tools on top of it, and it was very successful with that aim.
The traditional source of candidate data for such projects has been through expensive private providers, not least because the official candidate lists are published just a few days before the election.
Thanks to YourNextMP’s wonderful crowd-sourcing and triple-checking volunteers, we reckon that we had the most complete, most accurate data, the earliest. And it was free.
Directly informing over a million citizens
YourNextMP also came into its own as a direct source of information for the UK’s electorate. This hadn’t been the priority when the project was launched, but it was helped greatly by the fact that constituency and candidate pages ranked very highly in search engines from early on, so anyone searching for their local candidates found the site easily.
Once they did so, they found a list of everyone standing in their constituency, together with contact details, links to their online profiles such as web pages, social media and party websites, and feeds from spin-off projects (themselves built on YourNextMP data) such as electionleaflets.org and electionmentions.com.
YourNextMP had more than a million unique users. In the weeks just prior to the UK general election, it was attracting approximately 20,000 visitors per day, and on the day before the election, May 6th, there was suddenly a massive surge: that day the site was visited by nearly 160,000 people.
So, in a nutshell: YourNextMP has not only enabled a bunch of projects which helped people become more informed before our election—it also directly informed over a million citizens.
A reusable codebase
And, in the spirit of Poplus, the codebase is open for anyone to re-use in any country.
It’s already being pressed into use for the upcoming elections in Argentina, and we hope that developers in many other countries will use it to inform citizens, and inspire great web tools for the electorate, when their own elections come around.
If that’s something that interests you, please come and talk, ask questions and find out what’s involved, over on the Democracy Club mailing list.
The Knight Foundation’s News Challenge offers funding to innovative projects. We wonder whether they’ve ever had a bid whose collaborators span six different countries before.
You can read more about the plans on the bid page—and please click the little pink heart to give us ‘applause’!
In short, we want to build on the success that the YourNextMP crowdsourcing platform has had here in the UK.
Right now, YourNextMP offers open data on every candidate for the UK general election. That data is being used by major media companies and internet giants, and underlies several innovative online tools. On top of that, it’s getting thousands of visits every day from people who simply want more information about who’s standing in their area.
With some modification, other countries could use the same tech in advance of their own elections, giving their citizens the same opportunities to become more informed about those standing, and to develop still more useful online tools.
This is a ‘Yay for Poplus’ moment
Because Poplus is an international federation of organisations with similar needs, we can come together to forge plans that will benefit all of us, and then work together to make them a reality.
Our plans wouldn’t just benefit those six countries, either. Like every bit of Poplus tech, it’d be available as open source software for anyone to use, anywhere in the world. And that’s what Poplus is all about: maximum impact from every bit of code.
Join the Poplus mailing list to find out more about Poplus activities
Give some applause to our Knight Foundation News Challenge bid
The open federation for sharing civic tech, Poplus, is inviting YOU to get involved.
If you’re not sure what Poplus is, or how it works, there’s a great opportunity coming up to learn more.
On Wednesday February 18th there will be a Poplus live online Hangout which everyone is welcome to watch via Google Hangouts-on-Air. The specific aim of this Hangout is to invite new people and explain just what Poplus is. There will also be the opportunity to ask questions.
Steven Clift, Poplus’ Engagement Lead, writes:
Poplus is a global federation for the next generation of civic tech. We share knowledge and technology to help our organisations help citizens.
We do this through collaborative civic coding. We share reusable open source civic technology components. We leverage open government data or make our own across many countries.
This online video-based event specially invites those new to Poplus. Everyone is welcome.
The presentation includes Tom Steinberg with mySociety and a few project participants from around the world on the Hangout-on-Air “stage.” Questions will be taken via comments on the Hangout event page. The video stream is one-way, commenting two-way.
How to get involved
Share this event by forwarding this email or by sharing this hangout here.
Join our friendly online group with 350+ members from over 60 nations and representing 200+ organisations. Read the awesome participant introductions to see the amazing talent coming together. Add your introduction. Be part of what’s next with civic tech sharing.
Check out the Poplus components – test them – explore the code on GitHub – deploy them – enhance them – and learn about turning your open source code into a new Poplus Component for global sharing and impact.
- Follow the blog – Lightly follow this initiative and stay tuned for future events with occasional email updates.For questions about Poplus, post to our friendly online group or contact our engagement team.
Much of mySociety’s work is only possible thanks to generous funding from a number of philanthropic foundations.
Today, we are delighted to announce that we have been awarded a major strategic investment from Omidyar Network totalling up to $3.6m over three years.
This is the third time we’ve been supported by Omidyar Network, and this represents the biggest investment we’ve ever had. Alongside organisations like the Open Society Foundation, Google.org and the Indigo Trust, Omidyar has been central in our transformation from a tiny UK-focused non-profit, to a global social enterprise of nearly 30 staff.
Being supported by Omidyar Network means more than just vital financial support. It means access to their amazing networks of other investees, and advice and guidance from a range of sources. And, also crucial for an organisation that seeks technical excellence, it means the stamp of support from an organisation that ultimately traces its DNA back to the giant internet successes that are eBay and Paypal.
What is the money for?
mySociety’s main ambition, over the next three years, is to help a couple of dozen other organisations, spread around the world, to grow popular citizen empowerment tools that are big enough to really matter to the citizens of a wide range of countries. This means building and growing tools that help people to check up on politicians, demand information and answers, or report and track problems, in hugely varying contexts.
In addition to this, we will continue to maintain and grow the network of users of our technology and support the growing Poplus federation.
It’s a tough goal, and one that will require even more from the organisations we partner with, than from our own colleagues. But the very fact that we can even try to help groups at this scale, is because Omidyar Network enables us to imagine it.