We’ve got some exciting news at mySociety.
As you know we’ve been helping with the Kenyan Mzalendo website for a while now. And, we’ve been lucky enough to gather interest in Mzalendo‘s codebase from a number of other countries. These range from Ghana to South Africa, and even as far afield as Paraguay. It’s amazing and humbling for us, but we’ve recently realised one thing: Mzalendo has a wealth of history in Kenya, and an amazingly complex political association. It’s also the name of the website there. So we needed a new name that would allow the code to be changed without the change being associated with the original Kenyan Mzalendo.
Enter Pombola. This is the new name for the codebase which powers TheyWorkForYou, Mzalendo, Odekro, ShineYourEye, work-in-progress Kuvakazim and other parliamentary monitoring websites across Africa and the world.
You may ask, “Why Pombola? What does it mean?”
Well, it is a pretty easy word to remember. And no one else is using it (possibly because we created it!). The word is a mix of the initials PMO (Parliamentary Monitoring Organisation) and Tombola.
“A Tombola?” I hear you cry in surprise. Well, in a Tombola, you are making a choice with no information at all – just selecting a ticket and hoping that you get lucky. A Pombola site aims to be the opposite – you’ll get as much information as possible about your elected representatives, so that when you make a choice in future, you’ll have all the facts.
If you’re interested in using the code you’ll find the repository here on github, along with some documentation.
If you’re not technical but still want to use this then please contact me and we can discuss what we can do!
And remember, this may monitor parliaments now, but you could use it for anything (*)
(*) Disclaimer: please only use for good. Girl with Balloons from Courtney Air map from OpenFlights.org
These ‘websites in a box’ are a key part of our strategy to help people develop more successful civic and democratic websites around the world, but they are only the first half of our plan. Today I wanted to talk about the other half.
There are some use-cases for software in which most people are entirely happy to take some software off the shelf, press ‘Go’, and start using it. WordPress is a good example, and so is Microsoft Office.
However, there are some kinds of social issues that vary so much between different countries and regions that we believe one-size-fits-all tools for attacking them are impracticable.
This problem is particularly acute in the arena of sites and apps that allow people to track the activities of politicians. In this area there are several dozen different sites globally, almost all of which are powered by software that was written bespoke for that particular usage.
What drives this pattern of people re-writing every site from scratch is that people in different places care about different aspects of politics. In some countries what really counts is how politicians vote, in others the crux is campaign finance contributions, in others it is information on who has criminal records, and in others still it is whether public money has been vanishing suspiciously.
To build an off-the-shelf software platform that could handle all this data equally well in every country would be an immense coding task. And more important than that, we believe that it would create a codebase so huge and complex that most potential reusers would run away screaming. Or at least ignore it and start from scratch.
In short – we don’t believe there can be a WordPress for sites that monitor politicians, nor for a variety of other purposes that relate to good governance and stronger democracies.
We believe that the wrong answer to this challenge is to just say “Well then, everyone should build their own sites from scratch.” Over the years we at mySociety have been witness to the truly sad sight of people and organisations around the world wearing themselves out and blowing their budgets just trying to get the first version of a transparency website out the door. All too often they fail to create popular, long lasting sites because the birthing process is just so exhausting and resource-consuming that there’s nothing left to drive the sites to success. Often they don’t even get to launch.
A painful aspect of this problem is that the people who work on such sites are genuine altruists who are trying to solve serious problems in their part of the world; too much of their passion and energy is used up on building tools, when there’s still so much work beyond that that’s needed to make such sites successful. However, as we pointed out above, giving them a complete package on a plate isn’t an option. So what can we do?
Our Proposed Answer – The Components
We start from the following observation: coders and non-coders like simple, minimal, attractive tools that help them achieve bigger goals. Simple tools don’t make anyone run away screaming – they encourage exploration and deliver little sparkles of satisfaction almost immediately. But simple tools have to be highly interoperable and reliable to form the foundation of complex systems.
Our plan is to collaborate with international friends to build a series of components that deliver quite narrow little pieces of the functionality that make up bigger websites. These include:
There will be more, possibly many more. Our goal is to radically collapse the time it takes to build new civic and democratic (and possibly governmental) websites and apps, without putting constraints on creativity.
- PopIt – A Component to store and share the names of politicians, and the jobs they have.
- MapIt – A Component to store and share information on the locations of administrative boundaries, like counties, regions or cities.
- SayIt – A Component to store and share information on the words that public figures say or put out in writing.
Characteristics of each Component
There are some crucial architecture decisions that have been baked into the Components, to truly make them ‘small pieces loosely joined’.
- Each Component is fundamentally a tool for storing and sharing one or two kinds of common data – they’re intentionally minimalist.
- As a developer, you just use the Components that make sense for your goals – you simply don’t have to look at or learn about the Components that contain functionality that doesn’t matter to you.
- You don’t have to install anything to get started – you can always begin by playing with a hosted Component.
- We won’t impose our taste in programming languages on you. You can code your website in whatever language you want. The Components are not ‘modules’ – they don’t plug into some overarching framework like Drupal or WordPress. They are stand-alone tools which just present you data over REST APIs, and which you can write data into using REST APIs.
- Each Component’s data structures will offer as much flexibility as makes sense given the goal of keeping each Component really good at one or two tasks. We’ll listen to feedback carefully to get this right.
- Each Component has a clean, simple web front end so you can explore the data held in a store without having to write lots of SQL queries. Often you will be able to edit the data this way, too.
- Get started in seconds – each Component offers at least some functionality which is available inside a minute after getting involved.
- Non coders are welcome – we are building the Components so that non-coders can start gathering, editing and sharing data straight away, possibly long before they are in a position to launch a ‘real site’.
- Data can be added to the Components both through write APIs and through manual editing interfaces, suitable for non-coders.
- Learn from our mistakes – it is really easy to get the wrong data structure for civic, democratic or governmental data. Good practice data structures are baked into the Components, to save you pain later.
- Use our hosted versions, or install open source code locally. It will normally be quicker to get started in using the Components in a hosted environment, but if you want to run them locally, you’re entirely welcome. The code will be open source, and we’ll work hard to make sure it’s attractive and easy to install.
- The Components will talk to each other, and to the rest of the web using simple open schemas which will evolve as they are built. Where possible we’ll pick up popular data standards and re-use those, rather than building anything ourselves.
What the Components Aren’t
Sometimes in life it can be easier to describe things by what they aren’t:
- The Components are definitively not modules in a framework or platform. Each one is totally independent, and they will frequently be written in different languages – partly to force us to ensure that the APIs are truly excellent.
- The Components aren’t either Hosted or Local, they’re both. We’ll always offer a hosted version and a downloadable version, and you’ll always be able to move any data you have stored on the hosted versions down to your local copies.
- The Components aren’t all about mySociety. We’re planning to build the first ones in conjunction with some friends, and we’ll be announcing more about this soon. We want the family of Components to be jointly owned by a group of loving parents.
When can I see some of the Components in Action?
We’ll be blogging more about that tomorrow…
Footnote – To see the provenance of the extremely useful ‘small pieces loosely joined’ concept, see this.
DIY mySociety is all about making our code – and our experience – available to people who want to build similar websites in their own countries. We thought it would be helpful to list some examples of sites already using mySociety code, so you can see the variety of different possible outcomes.
It might seem like a simple task, but identifying sites in this way isn’t as straightforward as you might think – we don’t always know when people pick up our open source code! If we’ve missed any, please do comment below and we’ll add them.
There are also many sites around the world which were directly, or indirectly, “inspired by” ours. In these cases, the site’s owners have written their own code from scratch. That’s a subject – and a list – for another post. For now, here are all the international sites using mySociety’s code that we know about.
Alaveteli: our Right-to-Know Platform
WhatDoTheyKnow.com – our original Freedom of Information site
FYI.org.nz – New Zealand Freedom of Information site
Pravodaznam – Bosnia and Herzegovina Freedom of Information site
Queremossaber.br – Brazil Freedom of Information site
Informatazyrtare.org – Albania Freedom of Information site
Tuderechoasaber.es – Spain Freedom of Information site
AskTheEU – Europe Freedom of Information site
FixMyStreet: our fault-reporting Platform
FixMyStreet.com – our original fault-reporting site
Fiksgatami – Norway FixMyStreet
FixOurCity – Chennai FixMyStreet
FixMyStreet.br – Brazil FixMyStreet, based on both our code and FixMyStreet.ca from Canada
Parliamentary monitoring and access to elected representatives
TheyWorkForYou – our original parliamentary monitoring site
WriteToThem – our original ‘contact your representative’ site
Mzalendo – Kenya parliamentary monitoring site
Open Australia – Australia parliamentary monitoring site
Kildare Street – Ireland parliamentary monitoring site
Parlamany – Egypt parliamentary monitoring site
Mejlis – Tunisia parliamentary monitoring site
A community of people, waiting to help
Inspired by the examples above? If you’re thinking of going ahead and building your own site, we’re here to support you with our easy-to-understand guidebooks and our friendly mailing lists. In our online communities you’ll find many of the people who built the sites listed here. There’s no-one better to ask questions, because they’ve been through the process themselves, from early conception right up to completion.
If you are one of those people who has been through the whole process of building, launching and running a site like these (with or without our codebase), and lived to tell the tale, please shout in the comments below. And especially if you’re open to people approaching with questions. Perhaps add a note to say where you prefer to have those conversations – whether that’s via your favourite mailing lists, Twitter, email or simply in the comments to this post.
One last thought – it’s interesting to see that our code can be used for areas as small as a single city (FixMyStreet Chennai) or as large as a confederation of states (AskTheEU.org). In short, it’s scalable! How will you use it?
Image by Windell Oskay, used with thanks under the Creative Commons licence.
When TheyWorkForYou was built by a group of volunteer activists, many years ago, it was a first-of-a-kind website. It was novel because it imported large amounts of parliamentary data into a database-driven website, and presented it clearly and simply, and didn’t supply newspaper-style partisan editorial.
Mzalendo (which means ‘Patriot’ in Swahili) has been around for a few years too, as a blog and MP data website founded by volunteer activists Conrad and Ory. However, over the last few months mySociety’s team members Paul, Jessica and Edmund, plus the team at Supercool Design have been helping the original volunteers to rebuild the site from the ground up. We think that what’s launched today can stake a claim to being a true ‘second generation’ parliamentary monitoring site, for a few reasons:
- It is entirely responsively designed, so that it works on the simplest of mobile web browsers from day one.
- All the lessons we learned from storing political data wrongly have been baked into this site (i.e we can easily cope with people changing names, parties and jobs)
- Every organisation, position and place in the system is now a proper object in the database. So if you want to see all the politicians who went to Nairobi University, you can.
- There is lots of clear information on how parliament functions, what MPs and committees do, and so on.
- It synthesizes some very complex National Taxpayer’s Association data on missing or wasted money into a really clear ‘scorecard‘, turning large sums of money into numbers of teachers.
The codebase that Mzalendo is based on is free and open source, as always. It is a complete re-write, in a different language and framework from TheyWorkForYou, and we think it represents a great starting point for other projects. Over the next year we will be talking to people interested in using the code to run such sites in their own country. If this sounds like something of interest to you, get in touch.
Meanwhile, we wish Ory and Conrad the best of luck as the site grows, and we look forward to seeing what the first users demand.
If you haven’t got a penny,
A ha’penny will do,
If you haven’t got a ha’penny,
Then God bless you.
We wish you all a merry and prosperous Christmas – and for those of you who are already feeling quite prosperous enough, may we point you in the direction of our charitable donations page?
mySociety’s work is made possible by donations of all sizes and from all sorts of people. Those donations help fund all the online projects we create; projects that give easy access to your civic and democratic rights. If that’s important to you, show your appreciation, and we promise we’ll make the best use of every penny.
Thank you for sticking with us through this month-long post. We hope you’ve found it interesting and we wish you the very merriest of Christmases.
What’s behind the door? A letter to Santa.
If you can fit them down the chimney, here’s what we’re dreaming of:
More publicly available data Of course, we were delighted to hear in Mr Osborne’s autumn statement that all sorts of previously-inaccessible data will be opened up.
We’re wondering whether this new era will also answer any of our FixMyStreet geodata wishes. Santa, if you could allocate an elf to this one, we’d be ever so pleased.
Globalisation …in the nicest possible way, of course. This year has seen us work in places previously untouched by the hand of mySociety, including Kenya and the Philippines. And we continue to give help to those who wish to replicate our projects in their own countries, from FixMyStreet in Norway to WhatDoTheyKnow in Germany.
Santa, please could you fix it for us to continue working with dedicated and motivated people all around the world?
A mySociety Masters degree We’re lucky enough to have a team of talented and knowledgeable developers, and we hope we will be recruiting more in the coming year. It’s not always an easy task to find the kind of people we need – after all, mySociety is not your average workplace – so we’ve come to the conclusion that it’s probably easiest to make our own.
Back in February, Tom started thinking about a Masters in Public Technology. It’s still something we’re very much hoping for. Santa, is it true you have friends in academic circles?
FixMyTransport buy-in – from everyone! Regular users of FixMyTransport will have noticed that there are different kinds of response from the transport operators: lovely, fulsome, helpful ones, and formulaic ones. Or, worse still, complete refusal to engage.
Santa, if you get the chance, please could you tell the operators a little secret? Just tell them what those savvier ones already know – that FixMyTransport represents a chance to show off some fantastic customer service. And with 25,000 visitors to the site every week, that message is soon spread far and wide.
As a new edition has just been released, and I’ve had to tweak the parser to cope with the new highlighting, it’s a good time to write a brief article on TheyWorkForYou’s handling of the House of Commons Register of Members’ Financial Interests (Register of Members’ Interests as was before the current edition). Way back in the day, a scraper/parser was written (by either Julian or Francis) that monitors the Register pages on www.parliament.uk for new editions, and downloads and broadly parses the HTML into machine-readable data. The XML produced can be found at http://ukparse.kforge.net/parldata/scrapedxml/regmem/ – TheyWorkForYou then pulls in this XML into its database, and makes the latest data available on every MP’s page.
However, as it’s been scraping/parsing the Register since 2000, we can do more than that. Each MP’s page contains a link to a page giving the history of their entry in the Register – when things were added, removed, or changed. You can also view the differences between one edition of the Register and the next, or view a particular edition in a prettier form than the official site. There’s a central page containing everything Register-related at http://www.theyworkforyou.com/regmem/
UPDATE: If you approve of what we did this week, and what to help make sure we can still do it in the future, please pledge to support us: http://www.pledgebank.com/supportmysociety
The vote on concealing MPs’ expenses has been cancelled by the government!
In other words – we won!
This is a huge victory not just for transparency, it’s a bellwether for a change in the way politics works. There’s no such thing as a good day to bury bad news any more, the Internet has seen to that.
Over 7000 people joined a Facebook group, they sent thousands of emails to over 90% of all MPs. Hundreds of thousands of people found out about the story by visiting TheyWorkForYou to find something they wanted to know, reading an email alert, or simply discovered what was going on whilst checking their Facebook or Twitter pages. Almost all of this happened, from nowhere, within 48 hours, putting enough pressure on Parliament to force change.
Make no mistake. This is new, and it reflects the fact that the Internet generation expects information to be made available, and they expect to be able to make up their own minds, not be spoon fed the views of others. This campaign was always about more than receipts, it was about changing the direction of travel, away from secrecy and towards openness.
Today we stopped moving in the wrong direction. Tomorrow we start moving the right way. Sign up to our news mailing list (box on the right) to get updates on what mySociety gets up to.
Update: WE WON! [the following is now for historical interest]
Uh oh. Ministers are about to conceal MPs’ expenses, even though the public has just paid £1m to get them all ready for publication, and even though the tax man expects citizens to do what MPs don’t have to. They buried the news on the day of the Heathrow runway announcement. This is heading in the diametric wrong direction from government openness.
You can help in the following three ways:
1. Please write to your MP about this www.WriteToThem.com – ask them to lobby against this concealment, and tell them that TheyWorkForYou will be permanently and prominently noting those MPs who took the opportunity to fight against this regressive move. The millions of constituents who will check this site before the next election will doutbtless be interested.
2. Join this facebook group and invite all your least political friends (plus your most political too). Send them personal mails, phone or text them. Encourage them to write to their politicians too.
3. Write to your local paper to tell them you’re angry, and ask them to ask their readers to do the above. mySociety’s never-finished site http://news.mysociety.org might be able to help you here.
NB. mySociety is strictly non-partisan, by mission and by ethics. However, when it looks like Parliament is about to take a huge step in the wrong direction on transparency, we’ve no problem at all with stepping up when changes happen that threaten both the public interest and the ongoing value of sites like TheyWorkForYou and WhatDoTheyKnow.
Update: Every page on TheyWorkForYou, our biggest site, is now strongly encouraging people to join the protest.
Update: We’ve sailed past 1000 members to our Facebook group. Onward and upward!
Update: And now past 3000 members! Also, some MPs are claiming that they need to vote for this Order to protect their addresses, even though they already changed to law to do this. Doh!
Update: Now we’re past 6500, and our supporters have mailed their constituency MPs in over 90% of the constituencies in the UK. And rather helpfully, President Obama has just given us a concise explanation for MPs why this is a much bigger issue than some bits of paper and some minor embarrassment:
“And those of us who manage the public’s dollars will be held to account – to spend wisely, reform bad habits, and do our business in the light of day – because only then can we restore the vital trust between a people and their government.”
TheyWorkForYou doesn’t currently include EDMs for a couple of reasons:
1. They’re clearly less important to what Parliament does than Bills and committees, and are way down the priorities list as a consequence. We’ve nearly finished the process of adding bill committees, but we need the help of everyone who can lend a hand to overcome the opposition of senior unelected officials to get Bills published in a way we and others can use. If you haven’t already, please join the Free Our Bills campaign.
2. We are very tightly constrained by the amount of money we have, and we barely have enough to keep the site running, let alone add every new feature requested.
3. We’re a bit wary about EDMs in general because they’re not tied to any actual power. But that’s a reason to warn and educate readers, not exclude them from the account of Parliament.
So, if you really want EDMs on TheyWorkForYou, then, there are two things you can do. The first is donate to our parent charity, UKCOD. The second is to push in every way you can to get Bills published in the way we’ve asked for. That means writing to your MP, or if you are an MP, asking the modernisation committee and the Leader’s office what they’re doing to push past internal opposition to get Bills done right.
NB. Interestingly, the internal opposition is so ill-informed that we have it in writing they they think that TheyWorkForYou already includes EDMs – doh!
My house mate just said that his friend, who is at sixth form college, just had a PSHE (personal, social and health education) lesson in which they studied the website TheyWorkForYou.com.
Apparently it is good and I should go to it.