mySociety was built on its Democracy practice, a pioneer in providing simple- to-use tools that demystify the democratic process, allow citizens to understand how decisions are being made on their behalf and ensure that their voices are heard by elected representatives.
We’ve been on a long journey, from the early days of FaxYourMP which eventually became WriteToThem, to our pivotal TheyWorkForYou service which has both stretched the ambitions of Parliament in the UK and led us to develop similar services in Kenya, South Africa and beyond.
Amidst all of this has been our ongoing push to better standardise and make accessible more Open Data on politicians around the world; initially through our Poplus and Pombola projects, but more recently – and with more success – through our EveryPolitician service which has blossomed into a remarkable dataset of almost 4 million datapoints on over 72,000 politicians in 233 countries and territories.
Despite these successes I don’t think we’ve yet sufficiently cracked the challenge at scale of enabling more organisations to monitor and report upon the work of more politicians in more countries. We need to do something about that.
One of the principles that has always underpinned mySociety is that we carry our work out in the open, freely available for others to use. But, as is common with many Open Source projects, we do most of the development work ourselves internally. While community contributions are very welcome, practicality has dictated that more often than not, these are more commonly directed to raising tickets rather than making changes to the actual code.
Unchecked, this situation could lead to us being too internally focused; on developing everything ourselves rather than recognising where we can achieve our objectives by supporting other projects.
Fortunately our collaboration with Wikidata, announced earlier this year, suggests what promises to be a clear way forward to scaling up the impact of our work: we recognised that EveryPolitician could only become sustainable at scale as part of a wider community effort if we want our data to be used more widely.
By contributing to what we’ll call the Democratic Commons — a concept of shared code, data and resources where anyone can contribute, and anyone can benefit — we can help build and strengthen core infrastructure, tools and data that allow other democracy organisations and campaigners to hold their own governments to account.
This was notably put into practice for the snap General Election in the UK in June, where rather than build something new ourselves we directly supported the work of Democracy Club in their efforts to source candidate data and ensured that our existing services like MapIt, TheyWorkForYou and WhatDoTheyKnow were easily accessible for other campaigning and democracy organisations to put into use.
More recently we’ve established a commercial partnership with Facebook to provide them with accurate and independent lists of candidates and elected representatives matched to their relevant Facebook profile pages for the UK, French and Kenyan elections.
There’s a wider benefit to this kind of commercial work, beyond its being a useful source of additional revenue for mySociety. More importantly, it will allow us to feed the data that we source back into the Democratic Commons. It can contribute to EveryPolitician and Wikidata, and even improve boundary data internationally through OpenStreetMap, which in turn powers our own Global MapIt service.
Why is this important now?
Well, it’s not just the rather obvious observation that working with other people is a good idea. The reality is that we need to face the fact that our Democratic practice is just not fully funded, and, as with WhatDoTheyKnow.com, at best we’ll need to consider how more of our services in the UK can be run and directly supported by volunteers and the wider community.
At worst it’s quite possible that we’ll be forced to close some of our popular UK services and restrict the further development of our democracy work internationally.
In April next year we come to the end of our six-year grant agreement with the Omidyar Network who have given us tremendous support over that time. This will leave a substantial hole in our core funding and it’s one reason why we’ve been so diligently focused on developing appropriate new commercial services like FixMyStreetPro and WhatDoTheyKnowPro.
Without sufficient unrestricted core funding — that is, funding which can be applied wherever in the organisation it is most needed — we need to rely much more on specific project funding wherever we can find it. In most cases, however, this project funding comes with its own set of tasks to deliver, and there’s a tendency to want new shiny things, rather than supporting the maintenance of our existing projects. This is especially true of our Democracy work which relies more heavily on grant funding than commercial alternatives.
Sensibly directing our own work more towards contributions to external projects is also a hedge, should we need to find new homes for our services or shutter them for the time being.
In the meantime we’ll be speaking to more funders who we hope might recognise the importance of supporting and building the essential infrastructure of the Democratic Commons, but in the event that isn’t forthcoming we’ll do what it takes to ensure our work to date continues to have some value and impact.
As we start to map out a path to a sustainable future for mySociety and its community, I’d appreciate all thoughts on where we go next with this — after all, we can’t do this without your help.
Last year, when we were helping to develop YourNextMP, the candidate-crowdsourcing platform for the General Election, we made what seemed like an obvious decision.
We decided to use PopIt as the site’s datastore — the place from which it could draw information about representatives: their names, positions, et cetera. We’d been developing PopIt as a solution for parliamentary monitoring sites, but we reckoned it would also be a good fit for YourNextMP.
That turned out to be the wrong choice.
YourNextMP was up and running in time for the election, but at the cost of many hours of intensive development as we tried to make PopIt do what was needed for the site.
Once you’ve got an established site in production, changing the database it uses isn’t something you do lightly. But on returning to the codebase to develop it for international reuse, we had to admit that, in the words of mySociety developer Mark Longair, PopIt was “actually causing more problems than it was solving”. It was time to unpick the code and take a different approach.
Mark explains just what it took to decide to change course in this way, over on his own blog.
The post contains quite a bit of technical detail, but it’s also an interesting read for anyone who’s interested in when, and why, it’s sometimes best to question the decisions you’ve made.
There are websites built on mySociety code in many countries across the world.
Whatever the site you’re planning, you’ll find it a lot easier with our support and development help.
Our quarterly call for applications closes on October 30, so make sure you have yours in soon. Want to know exactly what’s involved? Start here.
Just how quickly can we hit the 200 countries mark on EveryPolitician? That’s what we’ll be finding out this week, and one thing’s for sure — we’ll get a lot further with your help.
This week is GLOW, the Global Legislative Openness Week, and we’re marking it with a concerted drive for more data.
Tony, the project lead, has consistently added one new country every day since EveryPolitician launched four months ago, and now it’s time to put a rocket behind our efforts.
The site currently contains data for 134 countries. We’ll be going flat out to see how quickly we can reach 200, and as the excitement ramps up, we hope you will help spread the word and get involved, too. Tony will carry on working as hard as he can to fill in the gaps, but we need your help to get further, faster.
What is EveryPolitician?
How can I help?
- Help us find data for more countries! We don’t currently know where to find the politician data for many countries. Here’s a list of the ones we need and here’s a page about how to contribute. If you get stuck, give us a shout.
- Write a scraper If you have the know-how, you can help us enormously by helping scrape the data from the places we do know about. See this page for guidance on how to go about writing a scraper. You’ll find lots of examples here.
- You can also help by spreading the word – tell your friends, tweet, blog, get up on a platform and talk, and just generally share this post. Thank you!
Why do we need this data?
Politician data is readily available for most countries, but it comes in a massive variety of inconsistent formats. Most of those formats aren’t ‘machine readable’, that is to say, the data can’t readily be extracted and re-used by programmers, and pretty much every country differs on what information it provides about each politician.
That being the case, anyone who wants to build an online tool that deals with politicians from more than one country, or who would like their tool to be available to people in other places, or would like to adapt an existing tool to be used elsewhere, would first have to adapt their tool to cope with the data.
EveryPolitician saves them the trouble, and the structured format also means that the tools they build will be compatible with any other tools that use it.
What kind of tools?
EveryPolitician data will be useful for all kinds of projects.
It’ll be much easier to build a website that shows people how to contact a politician. Or one that holds a government to account and educates people about what politicians are doing. Or one that helps voters make choices by displaying facts about what their politicians believe.
It can go further than that, though — with these building blocks in place, developers can really use their imagination to put together all kinds of projects, many of which we haven’t even begun to imagine. And don’t forget that, if a tool has been built to use the standardised data, it’ll also be easy for others to redeploy elsewhere.
If you’d like to see a concrete way in which the data’s already being used, check out Gender Balance.
How can I keep up to date?
We’ll be putting out regular updates via Twitter as the number of countries covered increases — plus you can watch the map turn green on http://everypolitician.org/countries.html as we progress.
As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
If you need data on the people who make up your parliament, another country’s parliament, or indeed all parliaments, you may be in luck.
What’s more, it’s all provided as Open Data to anyone who would like to use it to power a civic tech project. We’re thinking parliamentary monitoring organisations, journalists, groups who run access-to-democracy sites like our own WriteToThem, and especially researchers who want to do analysis across multiple countries.
But isn’t that data already available?
Yes and no. There’s no doubt that you can find details of most parliaments online, either on official government websites, on Wikipedia, or on a variety of other places online.
But, as you might expect from data that’s coming from hundreds of different sources, it’s in a multitude of different formats. That makes it very hard to work with in any kind of consistent fashion.
Every Politician standardises all of its data into the Popolo standard and then provides it in two simple downloadable formats:
- csv, which contains basic data that’s easy to work with on spreadsheets
- JSON which contains richer data on each person, and is ideal for developers
This standardisation means that it should now be a lot easier to work on projects across multiple countries, or to compare one country’s data with another. It also means that data works well with other Poplus Components.
What can I do with it?
Need a specific example? Yesterday, we introduced Gender Balance, the game that gathers data about women in politics.
As you’ll know if you’ve already given it a try, Gender Balance works by displaying politicians that make up one of the world’s legislatures, one by one.
That data all comes from Every Politician, and it’s meant that the developers have been able to concentrate on making a smooth and functional interface, knowing that the data side of things has already been taken care of.
That’s just one way to use Every Politician data, though. If you’d like to use it in your own site or app, you can find out more here.
We still need more data
As you may have noticed, there are more than 100 parliaments in the world. In fact, despite having reached what feels like a fairly substantial milestone, we’re still barely half way to getting some data for every parliament.
So we could use your help in finding data for the parliaments we don’t yet cover, and historic information for the ones we do. Read more about how you can help out.
The Poplus community is spread all over the world—but that doesn’t stop us getting together whenever we can.
Poplus is a worldwide federation of people and organisations with an interest in civic tech. This Friday, we will be holding a virtual Show and Tell, hearing from two very different projects:
- Andrew Mandelbaum from SimSim in Morocco will be speaking about Nouabook. This is an application which enables anyone to contact their politicians in public, through Facebook. It uses the Poplus Component WriteIt.
- Matthew Landauer from OpenAustralia Foundation will be speaking about Cuttlefish, one of the latest pieces of software to be certified as a Poplus Component.
As well as hearing all about these projects, there’ll be a chance to catch up and have a chat about all things Poplus/civic tech. Everyone is welcome.
We’ll be using an online platform called QiqoChat to host this call: sign up here.
You can create a free account using Facebook/Google/LinkedIn/Meetup/Twitter or a regular email address. Instructions for connecting by phone or computer microphone are available when you sign in and click “Participate”.
This Friday, 12 June. Times are as follows:
• 5 AM – US Pacific
• 7 AM – US Eastern
• 8 AM – Chile/Argentina
• 12 PM – UK
• 7 PM – Taiwan and Malaysia
• 9 PM – Sydney
• 11 PM- New Zealand
As you’ll know if you’re a regular reader of this blog, YourNextMP crowd-sourced details of every candidate who stood in the UK general election.
But, just because our own election is over, doesn’t mean we’ll be letting YourNextMP gather dust. On the contrary—we want to see it being re-used wherever there are elections being held, and citizens needing information! We’re already seeing the first re-use case, and we’d love to see more.
Opening up data
YourNextMP’s main purpose was to provide a free, open database of candidates, so that anyone who wanted to could build their own tools on top of it, and it was very successful with that aim.
The traditional source of candidate data for such projects has been through expensive private providers, not least because the official candidate lists are published just a few days before the election.
Thanks to YourNextMP’s wonderful crowd-sourcing and triple-checking volunteers, we reckon that we had the most complete, most accurate data, the earliest. And it was free.
Directly informing over a million citizens
YourNextMP also came into its own as a direct source of information for the UK’s electorate. This hadn’t been the priority when the project was launched, but it was helped greatly by the fact that constituency and candidate pages ranked very highly in search engines from early on, so anyone searching for their local candidates found the site easily.
Once they did so, they found a list of everyone standing in their constituency, together with contact details, links to their online profiles such as web pages, social media and party websites, and feeds from spin-off projects (themselves built on YourNextMP data) such as electionleaflets.org and electionmentions.com.
YourNextMP had more than a million unique users. In the weeks just prior to the UK general election, it was attracting approximately 20,000 visitors per day, and on the day before the election, May 6th, there was suddenly a massive surge: that day the site was visited by nearly 160,000 people.
So, in a nutshell: YourNextMP has not only enabled a bunch of projects which helped people become more informed before our election—it also directly informed over a million citizens.
A reusable codebase
And, in the spirit of Poplus, the codebase is open for anyone to re-use in any country.
It’s already being pressed into use for the upcoming elections in Argentina, and we hope that developers in many other countries will use it to inform citizens, and inspire great web tools for the electorate, when their own elections come around.
If that’s something that interests you, please come and talk, ask questions and find out what’s involved, over on the Democracy Club mailing list.
The Knight Foundation’s News Challenge offers funding to innovative projects. We wonder whether they’ve ever had a bid whose collaborators span six different countries before.
You can read more about the plans on the bid page—and please click the little pink heart to give us ‘applause’!
In short, we want to build on the success that the YourNextMP crowdsourcing platform has had here in the UK.
Right now, YourNextMP offers open data on every candidate for the UK general election. That data is being used by major media companies and internet giants, and underlies several innovative online tools. On top of that, it’s getting thousands of visits every day from people who simply want more information about who’s standing in their area.
With some modification, other countries could use the same tech in advance of their own elections, giving their citizens the same opportunities to become more informed about those standing, and to develop still more useful online tools.
This is a ‘Yay for Poplus’ moment
Because Poplus is an international federation of organisations with similar needs, we can come together to forge plans that will benefit all of us, and then work together to make them a reality.
Our plans wouldn’t just benefit those six countries, either. Like every bit of Poplus tech, it’d be available as open source software for anyone to use, anywhere in the world. And that’s what Poplus is all about: maximum impact from every bit of code.
Join the Poplus mailing list to find out more about Poplus activities
Give some applause to our Knight Foundation News Challenge bid
The open federation for sharing civic tech, Poplus, is inviting YOU to get involved.
If you’re not sure what Poplus is, or how it works, there’s a great opportunity coming up to learn more.
On Wednesday February 18th there will be a Poplus live online Hangout which everyone is welcome to watch via Google Hangouts-on-Air. The specific aim of this Hangout is to invite new people and explain just what Poplus is. There will also be the opportunity to ask questions.
Steven Clift, Poplus’ Engagement Lead, writes:
Poplus is a global federation for the next generation of civic tech. We share knowledge and technology to help our organisations help citizens.
We do this through collaborative civic coding. We share reusable open source civic technology components. We leverage open government data or make our own across many countries.
This online video-based event specially invites those new to Poplus. Everyone is welcome.
The presentation includes Tom Steinberg with mySociety and a few project participants from around the world on the Hangout-on-Air “stage.” Questions will be taken via comments on the Hangout event page. The video stream is one-way, commenting two-way.
How to get involved
Share this event by forwarding this email or by sharing this hangout here.
Join our friendly online group with 350+ members from over 60 nations and representing 200+ organisations. Read the awesome participant introductions to see the amazing talent coming together. Add your introduction. Be part of what’s next with civic tech sharing.
Check out the Poplus components – test them – explore the code on GitHub – deploy them – enhance them – and learn about turning your open source code into a new Poplus Component for global sharing and impact.
- Follow the blog – Lightly follow this initiative and stay tuned for future events with occasional email updates.For questions about Poplus, post to our friendly online group or contact our engagement team.