Yesterday we told you how the data on EveryPolitician had expanded wildly in the last week. One side effect is that there are 64 new countries to play on Gender Balance.
Our gender classification game (read more about it here) runs on politician data from EveryPolitician, so by adding a whole bunch of countries, we also expanded Gender Balance’s range.
It also means that, as those countries get played, we’ll be gathering even more informative and useful data about the proportions of women to men in the world’s legislatures.
That’s all we have to say, except, 3,2,1… get playing!
Amazing—we did it!
When we decided to mark Global Legislative Openness Week with a drive to get the data for 200 countries up on EveryPolitician, in all honesty, we weren’t entirely sure it could be done.
And without the help of many people we wouldn’t have got there. But last night, we put live the data for North Korea and Sweden, making us one country over the target.
The result? There is now consistently-structured, reusable data representing the politicians in 201 countries, ready for anyone to pick up and work with. We hope you will.
That’s not to say that our job is over… far from it! There’s still plenty more to be done, as we’ll explain below.
Here’s how it happened
Getting the data for each country was a multi-step process, aided by many people. First, a suitable online source had to be located. Then, a scraper would be written: a piece of code that could visit that source and pull out the information we needed—names, districts, political parties, dates of office, etc—and put it all in the right format.
Because each country’s data had its own idiosyncrasies and formatting, we needed a different scraper for every country.
Once written, we added each scraper to EveryPolitician’s list. Crucially, scrapers aren’t just a one-off deal: ideally they’ll continue to work over time as legislatures and politicians change.
The map above shows our progress during GLOW week, from 134 countries, where we began, up to today’s count of 201.
mySociety’s Tony, Lead on the EveryPolitician project, worked non-stop this week to get as many countries as possible online. But this week we’ve seen EveryPolitician reach some kind of momentum, as it takes off as a community project. It’s an ambitious idea, and it can only succeed with the help of this kind of community effort. Thanks to everyone who helped, including (in no particular order):
Duncan Walker for writing the scraper for Uganda; Joshua Tauberer for helping with the USA data; Struan Donald for handling Ecuador, Japan, Hong Kong, Serbia and the Netherlands; Dave Whiteland, with ThaiNetizen helpfully finding the data source for Thailand; Team Popong for South Korean data; Jenna Howe for her work on El Salvador; Rubeena Mahato, Chris Maddock, Kätlin Traks, François Briatte, @confirmordeny, and @foimonkey for lots of help on finding data; Henare Degan and OpenAustralia who made the scraper for Ukraine; Matthew Somerville for covering the Falkland islands and Sweden; Liz Conlan for lots of help with Peru and American Samoa; Jaroslav Semančík who provided data for, and assistance with, Slovakia; Mathias Huter who supplied current data for Austria while Steven Hirschorn wrote a scraper for the historic data; Andy Lulham who wrote a scraper for Gibraltar; Abigail Rumsey who wrote a scraper for Sri Lanka; everyone who tweeted encouragement or retweeted our requests for help.
But there’s more
There are still 40 or so countries for which we have no data at all: you can see them here. This week has provided an enormous boost to our data, but the site’s real target is, just like the name says, to cover every politician in the world.
And once we’ve done that, there’s still the matter of both historic data, and more in-depth data for the politicians we do have. Thus far, we mostly have only the lower houses for most countries which have two — and for many countries we only have the current politicians. Going into the future we need to include much richer data on all politicians, including voting records, et cetera.
Meanwhile, our first target, to have a list of the current members of every national legislature in the world, is starting to look like it’s not so very far away. If you’d like to help us reach it, here’s how you still can.
As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
How many people visit mySociety’s websites?
That’s a question we don’t ask ourselves as much as many other organisations. Much of our current funding is dependent on transactions (that is, the number of people using the site to complete an action such as making an FOI request, writing to a politician, or signing up to receive emails when their MP speaks), and rightly so, since that is a better measure of the sites’ actual effectiveness.
All the same, visitor numbers* do tell us about things like how much public awareness there is of what we do, and which of our sites is more visible than the others, so it’s good to take a proper look now and again.
Which of our UK sites is most visited?
By far our most popular site in terms of visitor numbers is our Freedom of Information site WhatDoTheyKnow. With over 4.5 million visitors 2014-15, it’s had three times more users than its closest competitor, TheyWorkForYou.
As well as allowing users to submit FOI requests, WhatDoTheyKnow also puts the responses into the public domain, so that the information becomes openly available. Every request receives, on average, twenty readers, meaning that transactions do not show the whole picture for this site.
WhatDoTheyKnow’s user numbers are also rising steadily. It’s up 8% on last year, and March 2015 was its highest month for unique users since its launch in 2008, at 470,509.
Which is least visited?
This dubious honour goes to WriteToThem, which nonetheless welcomed 457,209 visitors during the year, either helping them to write to their representatives, or simply showing them who those representatives were.
This was still a decent 11% rise on the previous year, despite a real rollercoaster where some months dipped substantially from the previous year.
Which made the most gains in the last year?
FixMyStreet saw the biggest percentage change, with a 21% rise in visitor numbers compared to the previous year; we talked a bit more about that in a recent blog post. WhatDoTheyKnow had the highest rise in actual visitor numbers: over 360,000 up on 2013-14.
Which fell by the most in the last year?
TheyWorkForYou saw a 12% drop in visitor numbers year on year (and also the biggest drop in real terms)—disappointing, but something we hope to rectify with the new voting pages, an ongoing process of rolling redesign, and some grassroots outreach.
How much effect do external events have on visitor numbers?
We already know that, as you’d expect, when Parliament is on holiday, MPs, debates and legislation aren’t in the news, and TheyWorkForYou visitor numbers fall. There’s also a weekly pattern for all our sites, where far fewer people use them at the weekends, presumably indicating that lots of our users access them from work.
It’s too early to say exactly what effect the election has had on our sites: as I write, people are eagerly checking out the voting records of newly-appointed cabinet ministers on TheyWorkForYou.
One thing we know for sure is that fewer people will have been using WriteToThem, because there have been no MPs to write to for the last few weeks. We’ve removed the “write to your MP” links from TheyWorkForYou, which always drove a good deal of WriteToThem’s traffic.
FixMyStreet enjoyed a boost back in June, when it was featured on the Channel 4 programme ‘The Complainers’—and the nice thing is, user numbers never receded back to their previous levels after the programme was over. Maybe people just need to use FixMyStreet to see how useful it is.
How many people visit mySociety’s UK websites in total?
This is a difficult figure for us to produce with accuracy, because we don’t trace whether you’re the same person visiting a number of our different sites.
However, the aggregate total of visitors to all our UK sites (WriteToThem, TheyWorkForYou, FixMyStreet and WhatDoTheyKnow) for 2014-15 is 6,983,028. Thanks very much if you were one of them
How can I help?
Glad you asked! If you find mySociety sites useful, you can help us spread the word by telling friends, sharing the URLs with any groups you are a member of, posting on Facebook or Twitter, or writing to your local paper.
We have a number of materials for FixMyStreet which can be found here; we hope to create similar materials for our other sites too, and we’ll make sure we announce it on here when we do.
* Note: all references to ‘users’ refer to unique users within the period discussed. So, users in a year means individual people who may have visited any number of times over that year, but are only counted once; same with monthly users.
Back in December, we told you about a project to collect the details of every election candidate in the UK— YourNextMP.com.
YourNextMP isn’t a mySociety project. It falls under the wide umbrella of Democracy Club, a loose confederation of volunteers doing interesting digital things, with the overarching aim of helping people be more informed before the election. We have, however, been lending our technical skills.
That database now contains details of every candidate and we’re really glad to see that many projects have been built on the back of it, from national newspaper visualisations to voter advice applications to single-issue sites and more.
Back in December, YourNextMP was a tool for crowd-sourced data-gathering. As well as providing free, open source data via its API, it has now matured into a useful static site in its own right. In a neat virtuous circle, it not only shows you who your candidates are, but also displays feeds from many of the sites using its own data.
What does that mean? Go and input your postcode and you’ll find not only:
- A list showing every prospective parliamentary candidate standing in your constituency, and including links to their Twitter stream, Facebook page, homepage and Wikipedia entry, where possible —
- Pictures of leaflets which have been delivered to residents in your constituency — from ElectionLeaflets.org, another crowdsourced project which is creating an archive of leaflets from all over the country, to stand as a permanent record of promises made pre-election
- Details of where you can go and see your candidates speak — from MeetYourNextMP.com, which crowdsources details of hustings in each area
- CVs from your local candidates — from Democracy Club’s CVs project
- News stories which mention your constituency or candidates — from electionmentions.com.
In many cases, these sites are just like YourNextMP: they’re relying on the time and energy of people like you, to add information. They’ve all made it as easy as possible though, so whether you fancy snapping an election leaflet on your phone and uploading it, or asking your candidates to provide a CV, it really does only take a couple of minutes.
You can also still continue to add more data (such as email addresses) to the candidates on YourNextMP, if you have time to contribute, and some basic Googling skills.
That’s not really the mySociety way, though.
All the same, we wanted to share some facts and figures about everything we got up to last year. It’s in the nature of our work that people tend to know about one part of it—say, our international work, or the sites we run here in the UK—but nothing else.
Well, to give you a more rounded picture, here is the mySociety annual report, featuring, among other things, the pop group One Direction, some vikings, and the TV presenter Phillip Schofield.
Welcome to mySociety in 2014… and if you enjoy it, please do share it around!
mySociety volunteers help us in all kinds of ways, and not just with coding stuff. This time we need skills and experience that only teachers can bring.
Here’s the thing: we’ve often heard from teachers of subjects like Politics, Citizenship and Social Studies that they’d love to integrate TheyWorkForYou, WriteToThem – and maybe even FixMyStreet – into their classroom activities.
We’d love it too. Our remit is to make democratic processes more accessible to all parts of society, and if this means that a whole new generation see contacting your politician as a perfectly normal and easy thing to do, well, that’d be a big win.
We want to provide downloadable lesson plans and resources – but we are not experts and we want to make sure that we get this right. Obviously, materials need to fit in with the present curricula, and be genuinely viable for classroom use.
There’s another possibility here, too – some of our software could be used in the classroom for students interested in coding and creating a new wave of online democracy projects themselves.
So: if you’re a teacher with a particular interest in democracy or digital technology, and you’d be willing to have a quick chat and then prepare some materials that we could provide for schools all across the UK to download, well – we’d love to hear from you. Or if that sounds like too much commitment, but you just have some ideas, let us know. Please mail us on email@example.com.
Every day, thousands of planning applications are submitted to local councils around the country by people applying to demolish a garage, erect a fence or convert a loft. More often than not these applications disappear into proprietary systems that, despite being publicly available, make it hard for members of the public to find out what’s going on in their area.
Last week, we kicked off the first sprint of an exciting new piece of work with the Hampshire Hub Partnership to build a prototype, open source web application to help members of the public find out more about planning applications in their area.
We jumped at the chance to work on this for a number of reasons.
Serving the needs of the public
Firstly, it has the needs of the general public as its focus. The planning process can be baffling if you’re new to it and this tool aims to help make it easier to understand. We’ll be helping people answer some of the most common questions they have about planning applications: What applications are happening near me? What decisions have been made in the past on applications like mine? How likely is it that my application will be dealt with on time?
A wireframe illustrating the potential functionality of the search results page
The site will help people browse planning application data by location — whether a postcode or a street address — and by type — whether it’s an extension, a loft conversion, or a major development like a retail park or commercial warehouse.
Built on Open Data
Secondly, it’s being made possible by the release of open data from local councils, once Ordnance Survey has granted the necessary exemption for locations derived from their data. Many of our projects rely on organisations publishing open data, so it’s great to have the chance to help demonstrate the value of releasing this kind of data openly.
The Hampshire Hub team has already spent a lot of time working with the LGA, DCLG and LeGSB to define a schema for how planning application data should be published. They’ve collaborated with local authorities, in particular Rushmoor Borough Council, to gather planning application data. And they’ve worked with Swirrl to set up an open data platform to collect all of this together, publish it openly and give us and others access to it.
Reuse, don’t rebuild
And finally, rather than build something from scratch, we’ll be using the fabulous PlanningAlerts.org.au open source codebase as a starting point. Planning Alerts is a piece of software built in Ruby on Rails by our friends down under at Open Australia. It gives us a lot of the functionality that we need for free. We plan in time to repay them for their kindness by submitting the features we develop back into their codebase (if they want them, of course).
We’ll also be using a customised version of our administrative boundaries service http://mapit.mysociety.org to store and query the geographical boundaries of different planning authorities in Hampshire (including National Park boundaries from Natural England as well as local council boundaries.)
We’ve just started our second sprint of work atop the Open Australia codebase, building the search functionality we need to help people find applications by location and category. We’re looking forward to seeing the tool grow, get into the hands of users and fill up with data.
About 6 weeks ago we arrived back from Monrovia, having just undertaken our first design exercise out there. Paul wrote about our experience in this blog as a broad overview. After further long distance design calls we wanted to delve a little more deeply into the process we’re following and what we’re learning about it.
To begin with, I should mention is that this is the first project where mySociety International will be leading on the implementation of a project using Design Thinking (the South African was a trial using a cut-down version of the approach and furthermore the implementation is being carried out solely by a local team).
Another important point for us is that the Design Thinking approach encompasses far more than just thinking about software development. The aim of the process is to develop an understanding what is required to ensure that the users’ need is addressed. Some of the solution might be technical, but much is likely to be about the processes and people that are required to ensure that needs are met.
For example, in the case of the Liberian FOI project where the internet penetration is low and the day to day obstacles people need to overcome are significantly more difficult than in the global north, a large proportion of the project time and resources will be dedicated to delivering offline services.
These provision of these services will tend to take a shape that fits into citizens’ current experiences. An example might be setting up an SMS short-code that allows people to contact a support team to call them back, in order to draft an FOI request on their behalf. They will then physically deliver hard copies of those FOI requests to the relevant ministries in Monrovia. This type of solution could be particularly beneficial for people who live outside the capital and do not have the time or resources to travel there to submit requests directly themselves.
There are two critical differences between the Design Thinking approach and other projects we have run with groups in the past. The first is that, with non-design centred partnerships, most groups start the process with a firm sense of the “type” of thing that they want from the outset – for example an instance of our Pombola platform that is used to power Mzalendo.com.
This is totally understandable, and in many cases what the funders of these projects are looking for, but it doesn’t necessarily mean that the users will get the most impactful solution.
Where we have sufficient funding to undertake this process available we intend for all of our new international projects to be starting with no expectations about the ultimate product – the outcome might even be that we don’t end up producing any software at all, because the best solution might consist of a desk, a phone and some radio advertising.
The second difference is that we have usually relied upon the local implementing partner to provide the insight and define the specification.
For our Design Thinking based projects we’ll have a closer relationship with the local partner and together we’ll identify potential end user groups we can talk to about their needs.
After doing a first round of in-depth interviews, the team then synthesise the information – essentially sharing what we’ve learnt with the rest of the group to pick out the most important points. The next stage is empathy mapping, where we figure out what people have said, thought, felt and done. This is a key stage in helping to identify the needs of the users.
It might seem simpler to ask them what they need, and often we did say something like “What would make this process easier for you?”. Yet actually analysing what they’re saying about the process and at what points they seemed frustrated or blocked – that tells us a lot more about points where we could change and hopefully improve the process than a straight up “What do you need?” question.
This is the stage we’re in at the moment with the Liberian project, though we have done some brief forays into Ideation – coming up with ideas for how to address the needs, and we are now starting to thinking about prototyping these ideas.
Of course, this method doesn’t mean we’ll completely stop using software solutions, or looking at A/B testing and Analytics as measures of the success of website. However we will also be looking at other measures of success or failure based on the product we’re building and the change we’re trying to achieve.
In the case of the Liberia FOI project, many of the users are likely to have no direct contact with the software themselves so we’ll need to design a monitoring system that measures the effect the changes have on their experience of making FOI requests.
One thing we’ve learnt is that a Design Thinking approach doesn’t only affect the first iteration of a solution. This may seem obvious, but from our brief work with this process we’ve seen that uses/users can be hard to predict at the outset – though in the case of the latter group we worked hard to spread the net widely in order to find potential users in Liberia.
So we’re interested to see, when we get to that point, what the prototype testing brings back and what new changes, improvements and tweaks need to be made.
More about our experiences with this process will be shared the lifetime of the project, as we learn, change and iterate ourselves.
The right conference, held at the right time and attended by people with common problems, can sometimes give birth to whole new organisations. I was at OpenTech when the Open Rights Group was born, and on a grander scale the Red Cross and the UN both featured conferences at catalytic moments in their early history.
Last week in Santiago, Chile, a conference took place that felt like exactly such a moment – PoplusCon. People from 27 countries spent two days talking about their shared goals and desires, and from it the skeleton of a new federation – the Poplus federation – started to take shape.
Not everyone at the conference worked on identical projects, or had identical skills. Some people were specialists in tracking suspicious relationships (‘This guy’s brother-in-law gets all the contracts’), others were big into training journalists how to use FOI, others specialised in making important datasets more accessible to members of the public, others still were journalists, skilled at constructing stories. But one theme emerged pretty quickly – people wanted better, easier, more reliable ways of sharing knowledge and sharing technology, so that they could all save time, effort and money.
What could a new federation do for you?
And so that is how the conversation turned to the idea of founding a new federation – an organisation that could serve the needs of many different groups without being run or owned by any one of them. In a brainstorm session about what people wanted from a new federation, the following ideas were raised:
- Running events to facilitate more sharing of ideas and tech
- Publishing stories about successful and unsuccessful projects, especially where those stories need to cross language barriers to spread
- Vetting and endorsing data standards
- Access to a community of peers (for sharing experience, encouragement, tips and tricks etc)
- Resources for projects that are running short
- Help and advice on making projects sustainable
- Certification of what counts as a Poplus Component
- Where groups face common challenges, perhaps coordinate advocacy
- Organisation of mentorship, exchanges and placements
This wish list is clearly far more than a nascent organisation could arrange in the near future, but there was some informal voting and the top priorities fairly quickly emerged. People really wanted access to their peers, and to the stories that they tell. And there was a strong wish to see Poplus Components become more official, and better explained.
Getting Real – Getting Involved
But a list is just a list without people willing to make it real. And so without doubt the most awesome thing that took place at PoplusCon was that eight people immediately volunteered to form a committee that would bring Poplus into being, representing half a dozen countries in different parts of the world.
This committee, which is completely open for anyone to join, will be meeting a couple of times in the next few weeks to agree on a plan for the first 12 months of the Poplus federation. It will work out how the new-born federation should govern itself, and what the first things that this entirely volunteer-run group should be doing. It’s an exciting, fragile moment and I’ve not seen anything like it in my ten-odd years working in this field. There’s no boss, no leader, just some people trying to build something of shared value.
Right now there are no rules, no barriers to entry, no bureaucracy. In fact there’s nothing but some hope, enthusiasm and some shared dreams of a stronger community of individuals and organisations.
I hope that if you read this and think that Poplus sounds cool, that you’ll consider joining the committee too. All you have to do is join the mailing list and ask where and when to show up. If you come to online committee meetings a couple of times, you’re de facto one of the people who runs Poplus. What happens next is – quite literally – down to you.