If you need data on the people who make up your parliament, another country’s parliament, or indeed all parliaments, you may be in luck.
What’s more, it’s all provided as Open Data to anyone who would like to use it to power a civic tech project. We’re thinking parliamentary monitoring organisations, journalists, groups who run access-to-democracy sites like our own WriteToThem, and especially researchers who want to do analysis across multiple countries.
But isn’t that data already available?
Yes and no. There’s no doubt that you can find details of most parliaments online, either on official government websites, on Wikipedia, or on a variety of other places online.
But, as you might expect from data that’s coming from hundreds of different sources, it’s in a multitude of different formats. That makes it very hard to work with in any kind of consistent fashion.
Every Politician standardises all of its data into the Popolo standard and then provides it in two simple downloadable formats:
- csv, which contains basic data that’s easy to work with on spreadsheets
- JSON which contains richer data on each person, and is ideal for developers
This standardisation means that it should now be a lot easier to work on projects across multiple countries, or to compare one country’s data with another. It also means that data works well with other Poplus Components.
What can I do with it?
Need a specific example? Yesterday, we introduced Gender Balance, the game that gathers data about women in politics.
As you’ll know if you’ve already given it a try, Gender Balance works by displaying politicians that make up one of the world’s legislatures, one by one.
That data all comes from Every Politician, and it’s meant that the developers have been able to concentrate on making a smooth and functional interface, knowing that the data side of things has already been taken care of.
That’s just one way to use Every Politician data, though. If you’d like to use it in your own site or app, you can find out more here.
We still need more data
As you may have noticed, there are more than 100 parliaments in the world. In fact, despite having reached what feels like a fairly substantial milestone, we’re still barely half way to getting some data for every parliament.
So we could use your help in finding data for the parliaments we don’t yet cover, and historic information for the ones we do. Read more about how you can help out.
Has he or she reflected your interests? One key way of checking that is to look at their voting record.
We’d like everyone to know exactly how their MP voted over the last parliament, so we’ve made some changes to TheyWorkForYou that make votes easier to understand.
See an example here, or read on to find out how to check your own MP’s voting record.
A complex matter
TheyWorkForYou publishes activity from Parliament each day.
This content includes parliamentary votes, along with the debates that they are part of. But it’s not always obvious to the lay reader exactly what’s being voted on.
Take a look at this debate, for example, on exemptions for smoke-free premises. By the time you’ve waded through the first clause,
“The appropriate national authority may make regulations providing for specified descriptions of premises, or specified areas within specified descriptions of premises, not to be smoke-free despite section 2”
– you may well be lost. And who would blame you?
Making it nice and simple
We don’t think you should have to be an expert to check your own MP’s voting record, and our new pages for each voting stance are here to help.
For some time now we’ve given you summaries of how your MP voted on certain topics, with a link to the votes that helped us understand each MP’s position on that stance.
Now we’ve created a page for each stance, and worded it in plain English so that anyone can understand exactly what it means.
See for yourself
Here’s how to see how your own MP voted (or we should say ‘previous MP’, since until the General Election, no MPs are now in office):
Go to TheyWorkForYou.com and input your postcode on the homepage.
You’ll be taken to the page of your (former) MP. Click on the ‘voting record’ tab.
Choose a topic you’re interested in, and click the ‘Details’ link on the far right.
You’ll see a plain English description of the stance, followed by descriptions of all the votes that were considered to contribute to it.
Want to see the context? Click on ‘show full debate’ and you’ll be taken to the full record of that vote.
Let us know what you think
These pages are still a work in progress, so we’ve included a feedback box at the top of each voting stance page. Do be sure to let us know if there’s anything else you’d like to see on them.
If you have feedback about how your MP has voted, mind you, that’s another matter… one you might want to reflect at the ballot box.
There was some excitement here at mySociety this week, as the People’s Assembly website launched in South Africa. It’s the result of a year’s partnership with PMG and a good test of some of our newest collaborative software.
The site contains a vast amount of information, all available in the same place for the first time, and offering a simple way for South African citizens to keep an eye on what their representatives are doing. There are pages for each representative, Hansard and parliamentary Questions and Answers, records of members’ interests, and more.
Locating, processing and displaying this data was quite a challenge: it has been taken from a wide range of sources, and came in an even greater range of formats, including PDF documents, Word documents, Excel files, CSV files and sometimes just e-mailed lists of information.
But perhaps most significant is the site’s Representative Locator function. For the first time, South African citizens can now find out, with ease, who represents them – not as simple as it might seem at first.
The Proportional Representative system means that members of the National Assembly and National Council of Provinces are not directly elected from constituencies. Political parties are, however, funded to run constituency offices and to allocate representatives to those offices. We believe that this is the first time this data has been consolidated and presented as a simple search tool.
The software that runs the site
As you’ll know if you read our recent blog post about SayIt, our recent focus has been reaching out to provide software for civic or democratic-focused websites anywhere in the world.
The idea is that such groups no longer need worry about writing code from scratch, since we’ve already done it – and their energies can be better expended on gathering data or adjusting the software to work within the local governmental systems.
People’s Assembly is a great example of this. It utilises two underpinning pieces of technology:
Firstly, the Pombola platform, our software for running parliamentary monitoring websites.
If you’re reading this in the UK, you may be familiar with our own parliamentary monitoring site, TheyWorkForYou. Pombola provides several tools that make it easy to do much of what TheyWorkForYou does: it provides a structured database of the names and positions of those in power; it allows people to look up their elected representatives by inputting their location, and to isolate and see what a specific MP has contributed to discussions in Parliament’s committees and plenaries; albeit, in the case of Hansard, after a six-month delay necessitated by South Africa’s own protocols.
We first developed Pombola for Kenya’s Mzalendo.com, and it’s been re-used for ShineYourEye.org in Nigeria and Odekro.org in Ghana. It’s superb to see this re-use, as it’s exactly what we set out to acheive.
Secondly, People’s Assembly is the very first site to use SayIt, which is embedded as a Django app to power the Hansard, Questions and Committees content. SayIt is one of our Components, built under the Poplus project, and we’re truly delighted to see it in place, proving its worth and being used as we first envisaged.
Thanks are due
The main work on the People’s Assembly has been funded by the Indigo Trust, and the SayIt component work was funded by Google.org as part of the Poplus Project. We also wish to thank Geoff Kilpin, who helped greatly with the scrapers and templating.
Transparency, accountability and open government are huge themes for African citizens as the number of internet and mobile phone users jump up across the continent. People are connecting and realising that the internet provides them with a quick and easy way to engage with politics, be that via social media or citizen engagement websites.
One group have just launched a parliamentary monitoring platform for Zimbabwe using our Pombola platform. We helped them with the original set up, some small technical issues and some general platform advice, but KuvakaZim has only gotten to launch due to the huge dedication and work of its founders, Regina and Peter.
“The KuvakaZim project was born from a general concern regarding the accountability and activities of Zimbabwean Members of parliament and their duties in regard of their representative role,” Regina Dumba, lead volunteer of the project, tells the world in her press release.
“Many articles, books and studies have explored the issue of good governance in African countries and how it relates to transparency, accountability, and Government performance. Knowing the causes and effects of these plights, we believe it is now time for action in Africa and in Zimbabwe. Until we start putting words into action, only then can we rebuild our country.” She continues on the KuvakaZim blog.
Creating the site.
Regina, Cleopatra and Peter, who has been volunteering technical skills for the project, contacted mySociety in July after being inspired by Kenya’s Mzalendo. Since then they have been working tirelessly to gather MP data and information on constitutional rights, how democracy works in Zimbabwe, electoral law and political parties. The site now allows Zimbabweans to learn more about how their government works, as well as the duties of their MP and whether they are carrying these out. This has been especially timely because of the recent elections on July 31, 2013.
That’s not to say that the site has got to this stage without any hitches however..
It’s been difficult to find official boundary data for Zimbabwe, which means we haven’t yet managed to load an MP look up onto the site. The hope is that this will come in the future, along with other features such as Hansard and the potential to write to your politician.
Despite this the team have managed to gain some on the ground volunteer support and launch the site this week. If you want to learn more about KuvakaZim the check out their blog and their twitter stream. We’ll be following their progress too!
Image credits: Patola Connection by Whologwhy | Hands up by Pim Geerts | Bend in the Road by Andrew Ashton | All Creative Commons licensed photographs. Thank you for making your content creative commons.
mySociety is an organisation with many priorities, and they often compete for attention. Right now, we have some time and budget to lavish on TheyWorkForYou. We need your input to help us understand what development we should prioritise for the site.
Note: if you don’t know much about TheyWorkForYou, your opinion still counts! See the foot of this blog post for an overview of the site and its aims.
Some suggested improvements
Below is a list of improvements that other users have suggested, or that we think are desirable. Which improvements would you most like to see – from this list, or based on your own needs?
- Easier sharing via social media If you see a debate you want to share with your Twitter or Facebook buddies, all you’d have to do is click a button. More details
- When your MP voted Letting you know, via email alerts, when your MP has taken part in a vote. More details
- Option to search just headings At the moment, search covers all content of debates, including everything anyone said. This option would allow you to only search headings, meaning that you could be sure the results were entirely focused on your topic. More details
- Tweeting debate headings or future business A Twitter account which would tweet, and link to, every debate in Parliament, or upcoming events. More details
- Signposting of big ‘events’ such as the Budget These are not always easy to find if you don’t know your way around, so we’d make sure the big events were always trumpeted on the site. More details
Great ideas, or utter bunkum? Let us know. You can give us feedback via any of the following methods:
1. Leave a comment under this blog post;
3. If you’d like to see the whole list of suggestions and issues, you can do so on our development list at Github (and the ‘more details’ links in the list above go to the issues on there). Note that anyone is welcome to add comments to these issues, or even to create your own (please search first to make sure you’re not duplicating an existing issue). Github may look complex, but it’s easy enough to use – you just need to set up a user account here.
We’re keen to understand whether we’re serving all kinds of users, so it’d also be helpful if you could tell us whether you consider yourself to be someone who knows a bit about Parliament (through work, interest, or experience) or a novice user.
Note – you can see what we’re currently working on here. Some changes were obvious – for example, we’re improving MPs’ individual pages.
What is TheyWorkForYou for?
TheyWorkForYou has been running since 2004. We know why we launched it, though the way you use it may be totally different – and if so, we want to hear about that. Its aim is to give a window into Parliament, for everyone, but including people who may never have previously thought that parliamentary proceedings had anything to do with them.
TheyWorkForYou does a lot of things. It lets you find out who your MP is – if you don’t know – and then it tells you all about them.
It publishes the written record of debates in Parliament, and lets you search it, and link to it easily.
It allows you to set up alerts, so you get an email every your chosen words or phrases are mentioned in Parliament – or every time a particular person speaks.
It publishes future business (there are alerts for that, too), written answers, Public Bill Committees, and more.
So, it does a lot – but we know that it still doesn’t do everything our users request, and it doesn’t neccessarily do everything in the way that they want, either. Some changes are obvious, and we’re working on them – right now, for example, we are improving individual MPs’ pages. But we want your thoughts too.
Photo by Lindsay Bremner (CC)
We’ve got some exciting news at mySociety.
As you know we’ve been helping with the Kenyan Mzalendo website for a while now. And, we’ve been lucky enough to gather interest in Mzalendo‘s codebase from a number of other countries. These range from Ghana to South Africa, and even as far afield as Paraguay. It’s amazing and humbling for us, but we’ve recently realised one thing: Mzalendo has a wealth of history in Kenya, and an amazingly complex political association. It’s also the name of the website there. So we needed a new name that would allow the code to be changed without the change being associated with the original Kenyan Mzalendo.
Enter Pombola. This is the new name for the codebase which powers TheyWorkForYou, Mzalendo, Odekro, ShineYourEye, work-in-progress Kuvakazim and other parliamentary monitoring websites across Africa and the world.
You may ask, “Why Pombola? What does it mean?”
Well, it is a pretty easy word to remember. And no one else is using it (possibly because we created it!). The word is a mix of the initials PMO (Parliamentary Monitoring Organisation) and Tombola.
“A Tombola?” I hear you cry in surprise. Well, in a Tombola, you are making a choice with no information at all – just selecting a ticket and hoping that you get lucky. A Pombola site aims to be the opposite – you’ll get as much information as possible about your elected representatives, so that when you make a choice in future, you’ll have all the facts.
If you’re interested in using the code you’ll find the repository here on github, along with some documentation.
If you’re not technical but still want to use this then please contact me and we can discuss what we can do!
And remember, this may monitor parliaments now, but you could use it for anything (*)
(*) Disclaimer: please only use for good. Girl with Balloons from Courtney Air map from OpenFlights.org
These ‘websites in a box’ are a key part of our strategy to help people develop more successful civic and democratic websites around the world, but they are only the first half of our plan. Today I wanted to talk about the other half.
There are some use-cases for software in which most people are entirely happy to take some software off the shelf, press ‘Go’, and start using it. WordPress is a good example, and so is Microsoft Office.
However, there are some kinds of social issues that vary so much between different countries and regions that we believe one-size-fits-all tools for attacking them are impracticable.
This problem is particularly acute in the arena of sites and apps that allow people to track the activities of politicians. In this area there are several dozen different sites globally, almost all of which are powered by software that was written bespoke for that particular usage.
What drives this pattern of people re-writing every site from scratch is that people in different places care about different aspects of politics. In some countries what really counts is how politicians vote, in others the crux is campaign finance contributions, in others it is information on who has criminal records, and in others still it is whether public money has been vanishing suspiciously.
To build an off-the-shelf software platform that could handle all this data equally well in every country would be an immense coding task. And more important than that, we believe that it would create a codebase so huge and complex that most potential reusers would run away screaming. Or at least ignore it and start from scratch.
In short – we don’t believe there can be a WordPress for sites that monitor politicians, nor for a variety of other purposes that relate to good governance and stronger democracies.
We believe that the wrong answer to this challenge is to just say “Well then, everyone should build their own sites from scratch.” Over the years we at mySociety have been witness to the truly sad sight of people and organisations around the world wearing themselves out and blowing their budgets just trying to get the first version of a transparency website out the door. All too often they fail to create popular, long lasting sites because the birthing process is just so exhausting and resource-consuming that there’s nothing left to drive the sites to success. Often they don’t even get to launch.
A painful aspect of this problem is that the people who work on such sites are genuine altruists who are trying to solve serious problems in their part of the world; too much of their passion and energy is used up on building tools, when there’s still so much work beyond that that’s needed to make such sites successful. However, as we pointed out above, giving them a complete package on a plate isn’t an option. So what can we do?
Our Proposed Answer – The Components
We start from the following observation: coders and non-coders like simple, minimal, attractive tools that help them achieve bigger goals. Simple tools don’t make anyone run away screaming – they encourage exploration and deliver little sparkles of satisfaction almost immediately. But simple tools have to be highly interoperable and reliable to form the foundation of complex systems.
Our plan is to collaborate with international friends to build a series of components that deliver quite narrow little pieces of the functionality that make up bigger websites. These include:
There will be more, possibly many more. Our goal is to radically collapse the time it takes to build new civic and democratic (and possibly governmental) websites and apps, without putting constraints on creativity.
- PopIt – A Component to store and share the names of politicians, and the jobs they have.
- MapIt – A Component to store and share information on the locations of administrative boundaries, like counties, regions or cities.
- SayIt – A Component to store and share information on the words that public figures say or put out in writing.
Characteristics of each Component
There are some crucial architecture decisions that have been baked into the Components, to truly make them ‘small pieces loosely joined’.
- Each Component is fundamentally a tool for storing and sharing one or two kinds of common data – they’re intentionally minimalist.
- As a developer, you just use the Components that make sense for your goals – you simply don’t have to look at or learn about the Components that contain functionality that doesn’t matter to you.
- You don’t have to install anything to get started – you can always begin by playing with a hosted Component.
- We won’t impose our taste in programming languages on you. You can code your website in whatever language you want. The Components are not ‘modules’ – they don’t plug into some overarching framework like Drupal or WordPress. They are stand-alone tools which just present you data over REST APIs, and which you can write data into using REST APIs.
- Each Component’s data structures will offer as much flexibility as makes sense given the goal of keeping each Component really good at one or two tasks. We’ll listen to feedback carefully to get this right.
- Each Component has a clean, simple web front end so you can explore the data held in a store without having to write lots of SQL queries. Often you will be able to edit the data this way, too.
- Get started in seconds – each Component offers at least some functionality which is available inside a minute after getting involved.
- Non coders are welcome – we are building the Components so that non-coders can start gathering, editing and sharing data straight away, possibly long before they are in a position to launch a ‘real site’.
- Data can be added to the Components both through write APIs and through manual editing interfaces, suitable for non-coders.
- Learn from our mistakes – it is really easy to get the wrong data structure for civic, democratic or governmental data. Good practice data structures are baked into the Components, to save you pain later.
- Use our hosted versions, or install open source code locally. It will normally be quicker to get started in using the Components in a hosted environment, but if you want to run them locally, you’re entirely welcome. The code will be open source, and we’ll work hard to make sure it’s attractive and easy to install.
- The Components will talk to each other, and to the rest of the web using simple open schemas which will evolve as they are built. Where possible we’ll pick up popular data standards and re-use those, rather than building anything ourselves.
What the Components Aren’t
Sometimes in life it can be easier to describe things by what they aren’t:
- The Components are definitively not modules in a framework or platform. Each one is totally independent, and they will frequently be written in different languages – partly to force us to ensure that the APIs are truly excellent.
- The Components aren’t either Hosted or Local, they’re both. We’ll always offer a hosted version and a downloadable version, and you’ll always be able to move any data you have stored on the hosted versions down to your local copies.
- The Components aren’t all about mySociety. We’re planning to build the first ones in conjunction with some friends, and we’ll be announcing more about this soon. We want the family of Components to be jointly owned by a group of loving parents.
When can I see some of the Components in Action?
We’ll be blogging more about that tomorrow…
Footnote – To see the provenance of the extremely useful ‘small pieces loosely joined’ concept, see this.
DIY mySociety is all about making our code – and our experience – available to people who want to build similar websites in their own countries. We thought it would be helpful to list some examples of sites already using mySociety code, so you can see the variety of different possible outcomes.
It might seem like a simple task, but identifying sites in this way isn’t as straightforward as you might think – we don’t always know when people pick up our open source code! If we’ve missed any, please do comment below and we’ll add them.
There are also many sites around the world which were directly, or indirectly, “inspired by” ours. In these cases, the site’s owners have written their own code from scratch. That’s a subject – and a list – for another post. For now, here are all the international sites using mySociety’s code that we know about.
Alaveteli: our Right-to-Know Platform
WhatDoTheyKnow.com – our original Freedom of Information site
FYI.org.nz – New Zealand Freedom of Information site
Pravodaznam – Bosnia and Herzegovina Freedom of Information site
Queremossaber.br – Brazil Freedom of Information site
Informatazyrtare.org – Albania Freedom of Information site
Tuderechoasaber.es – Spain Freedom of Information site
AskTheEU – Europe Freedom of Information site
FixMyStreet: our fault-reporting Platform
FixMyStreet.com – our original fault-reporting site
Fiksgatami – Norway FixMyStreet
FixOurCity – Chennai FixMyStreet
FixMyStreet.br – Brazil FixMyStreet, based on both our code and FixMyStreet.ca from Canada
Parliamentary monitoring and access to elected representatives
TheyWorkForYou – our original parliamentary monitoring site
WriteToThem – our original ‘contact your representative’ site
Mzalendo – Kenya parliamentary monitoring site
Open Australia – Australia parliamentary monitoring site
Kildare Street – Ireland parliamentary monitoring site
Parlamany – Egypt parliamentary monitoring site
Mejlis – Tunisia parliamentary monitoring site
A community of people, waiting to help
Inspired by the examples above? If you’re thinking of going ahead and building your own site, we’re here to support you with our easy-to-understand guidebooks and our friendly mailing lists. In our online communities you’ll find many of the people who built the sites listed here. There’s no-one better to ask questions, because they’ve been through the process themselves, from early conception right up to completion.
If you are one of those people who has been through the whole process of building, launching and running a site like these (with or without our codebase), and lived to tell the tale, please shout in the comments below. And especially if you’re open to people approaching with questions. Perhaps add a note to say where you prefer to have those conversations – whether that’s via your favourite mailing lists, Twitter, email or simply in the comments to this post.
One last thought – it’s interesting to see that our code can be used for areas as small as a single city (FixMyStreet Chennai) or as large as a confederation of states (AskTheEU.org). In short, it’s scalable! How will you use it?
Image by Windell Oskay, used with thanks under the Creative Commons licence.
When TheyWorkForYou was built by a group of volunteer activists, many years ago, it was a first-of-a-kind website. It was novel because it imported large amounts of parliamentary data into a database-driven website, and presented it clearly and simply, and didn’t supply newspaper-style partisan editorial.
Mzalendo (which means ‘Patriot’ in Swahili) has been around for a few years too, as a blog and MP data website founded by volunteer activists Conrad and Ory. However, over the last few months mySociety’s team members Paul, Jessica and Edmund, plus the team at Supercool Design have been helping the original volunteers to rebuild the site from the ground up. We think that what’s launched today can stake a claim to being a true ‘second generation’ parliamentary monitoring site, for a few reasons:
- It is entirely responsively designed, so that it works on the simplest of mobile web browsers from day one.
- All the lessons we learned from storing political data wrongly have been baked into this site (i.e we can easily cope with people changing names, parties and jobs)
- Every organisation, position and place in the system is now a proper object in the database. So if you want to see all the politicians who went to Nairobi University, you can.
- There is lots of clear information on how parliament functions, what MPs and committees do, and so on.
- It synthesizes some very complex National Taxpayer’s Association data on missing or wasted money into a really clear ‘scorecard‘, turning large sums of money into numbers of teachers.
The codebase that Mzalendo is based on is free and open source, as always. It is a complete re-write, in a different language and framework from TheyWorkForYou, and we think it represents a great starting point for other projects. Over the next year we will be talking to people interested in using the code to run such sites in their own country. If this sounds like something of interest to you, get in touch.
Meanwhile, we wish Ory and Conrad the best of luck as the site grows, and we look forward to seeing what the first users demand.
If you haven’t got a penny,
A ha’penny will do,
If you haven’t got a ha’penny,
Then God bless you.
We wish you all a merry and prosperous Christmas – and for those of you who are already feeling quite prosperous enough, may we point you in the direction of our charitable donations page?
mySociety’s work is made possible by donations of all sizes and from all sorts of people. Those donations help fund all the online projects we create; projects that give easy access to your civic and democratic rights. If that’s important to you, show your appreciation, and we promise we’ll make the best use of every penny.
Thank you for sticking with us through this month-long post. We hope you’ve found it interesting and we wish you the very merriest of Christmases.
What’s behind the door? A letter to Santa.
If you can fit them down the chimney, here’s what we’re dreaming of:
More publicly available data Of course, we were delighted to hear in Mr Osborne’s autumn statement that all sorts of previously-inaccessible data will be opened up.
We’re wondering whether this new era will also answer any of our FixMyStreet geodata wishes. Santa, if you could allocate an elf to this one, we’d be ever so pleased.
Globalisation …in the nicest possible way, of course. This year has seen us work in places previously untouched by the hand of mySociety, including Kenya and the Philippines. And we continue to give help to those who wish to replicate our projects in their own countries, from FixMyStreet in Norway to WhatDoTheyKnow in Germany.
Santa, please could you fix it for us to continue working with dedicated and motivated people all around the world?
A mySociety Masters degree We’re lucky enough to have a team of talented and knowledgeable developers, and we hope we will be recruiting more in the coming year. It’s not always an easy task to find the kind of people we need – after all, mySociety is not your average workplace – so we’ve come to the conclusion that it’s probably easiest to make our own.
Back in February, Tom started thinking about a Masters in Public Technology. It’s still something we’re very much hoping for. Santa, is it true you have friends in academic circles?
FixMyTransport buy-in – from everyone! Regular users of FixMyTransport will have noticed that there are different kinds of response from the transport operators: lovely, fulsome, helpful ones, and formulaic ones. Or, worse still, complete refusal to engage.
Santa, if you get the chance, please could you tell the operators a little secret? Just tell them what those savvier ones already know – that FixMyTransport represents a chance to show off some fantastic customer service. And with 25,000 visitors to the site every week, that message is soon spread far and wide.