— Ian Chard (@Flupsybunny) September 30, 2015
Sadly, we’ll be saying farewell to our current SysAdmin Ian shortly. We’d like our websites to keep running after his departure, ideally, so we’re recruiting for a replacement.
A replacement who can keep our servers secure, maintain our back-ups infrastructure, and resolve performance bottlenecks, among other daily challenges.
If that sounds like you, or someone you know, you can find the full details here.
What’s it like working for mySociety, you ask? Take a look at this page.
Scottish Parliamentary proceedings are now back on TheyWorkForYou.
Back in August 2014, the Scottish Parliament changed the way it published the Official Report of its debates.
TheyWorkForYou works by fetching data from various parliamentary sources—and in this case, unfortunately, the change at the Scottish Parliament end meant that our code no longer worked. We replaced our ‘debates’ section with an apologetic note.
Well, thanks to the Scottish Parliament kindly republishing the data in almost the format we used to use, we’ve managed to make some small tweaks and restore that content—including debates from the previously missing period. If you’re subscribed to alerts, you should have received an email digest with links to the backdated content (always supposing there was any that matched your chosen keywords).
And if you’re not subscribed to alerts? Now is a great time to rectify that. We’ll send you an email every time your chosen word or phrase is mentioned in Parliament, or every time your chosen representative speaks.
While we were doing this work, we also modified TheyWorkForYou so that it now pulls in ministerial data from the Scottish Parliament API. This is a welcome time-saver for us: previously we were creating a list manually from the official PDFs, while we can now automatically fetch it and reformat it into Popolo JSON, meaning it’s consistent with all our other data.
Thanks for your patience; we know that many people were awaiting this repair, and for longer than we would have liked. Enjoy!
Ever feel sorry for the less popular kids at school?
Excellent, then you’re just the sort of person we need: you may empathise with some of the countries on Gender Balance that aren’t getting quite as much attention as the rest.
Thanks to our recent data drive, Gender Balance now contains many more countries, all waiting for you to play.
But we’ve noticed that some countries aren’t getting quite as much attention as others. Gender Balance’s ultimate aim is to provide data for researchers, and we’d hate to feel that we had patchier data for those studying the less popular places.
So, to encourage take-up, we’ve now added a ‘featured country’ spot. Accept the invitation to play the highlighted place, and you’ll receive double points, propelling you all the faster towards a coveted place on the Gender Balance leaderboard. Time to get playing!
You’ve probably seen the recent news story about the guy who bought, then vastly increased the price of a vital drug: it’s been widely shared on social media in the last couple of days.
Not long ago, we blogged about the work we’d done on the Patent Oppositions Database.
You may have found the concepts involved somewhat abstract. If so, now’s a great time to go and read it again, with this news story in mind. It’s an excellent example of what that project is hoping to prevent.
Image: David Goehring (CC)
Our International team get many enquiries from people and organisations who want to re-use our code, all around the world, and would like a little help doing so. As, sadly, there is limited time in the day, we find that we can’t donate our resources to everyone who asks.
Up until now, we’ve had a fairly ad-hoc approach. Typically, someone makes contact, we send emails back and forth to find out more about their proposed project, and then we make a decision about whether we can offer some developer time and help.
But that’s not really fair: it means that, if we accept one project and then the next week another approach comes in from a project that is just as suitable, we could have committed all our developer time and resource to the first group.
So, we plan to put a new system in place. Here’s the deal:
- Those who would like our help will be asked to fill in an application form with all the details that we’d normally be extracting during those back-and-forth emails
- These applications will be assessed on a quarterly basis
- We’ll let applicants know whether they have been successful within seven days of the closing date
- Not everyone who applies will be successful, but they’ll have another three months in which to reapply with additional information, should they wish
We think that this system is fairer for everyone, and we hope you agree.
If you’ve recently approached us to enquire about getting our help, please bear with us while we transition to this new system: we’ll be in touch soon.
If you’re a group or an individual that might be interested in our help, you can start your application here.
According to a postscript on that story, TfL have since commented:
This map was produced for engineering works planning and wasn’t designed for customer use, however we are happy to make any maps available which help our customers to travel in London. This map will therefore be added to our website.
Great result. We hope that thanks to Buzzfeed’s viral spread, from today, plenty more people understand the potential of FOI to change things for the benefit of many.
Last month we wrote about a Freedom of Information request, submitted through WhatDoTheyKnow, on West Ham’s lease of the former Olympic Stadium. The resulting information was interesting enough that it became the subject of a BBC TV programme.
The request, submitted by WhatDoTheyKnow user Richard Hunt on behalf of a group of Football Supporters’ Trusts, received some but not all of the information required. The rest was held back on grounds of ‘commercial confidentiality’, a decision which Richard asked the ICO to review.
On 3rd September the ICO found in his favour: the stadium owners, London Legacy Development Corporation, must now provide the full contract, including details of the rent being paid. Read more in this BBC story, which states:
The commissioner said neither West Ham nor LLDC had been able to show how revealing the details of the tenancy agreement would place them at a commercial disadvantage or how this information could be exploited by a competitor.
This is an important point, Richard says:
The commercial confidentiality excuse is a huge problem. I think that if a private company is involved, and the requester doesn’t have a business background, they just assume that it must be reasonable. So they don’t challenge it.
Fortunately we had plenty of business brains, and in particular an understanding of the football business. We challenged the excuse in detail… The overall lesson is, if you get this excuse, try to gain knowledge of the business in question, and ask yourself how exactly a competitor could take advantage of the company if the information was disclosed, and whether the damage outweighs the public interest.
Richard says he is happy to share their submissions to the ICO with anyone facing similar difficulties in obtaining information. You can contact him via WhatDoTheyKnow.
The London Legacy Development Corporation now has until 8th October to disclose the full details of the tenancy agreement.
Yesterday we told you how the data on EveryPolitician had expanded wildly in the last week. One side effect is that there are 64 new countries to play on Gender Balance.
Our gender classification game (read more about it here) runs on politician data from EveryPolitician, so by adding a whole bunch of countries, we also expanded Gender Balance’s range.
It also means that, as those countries get played, we’ll be gathering even more informative and useful data about the proportions of women to men in the world’s legislatures.
That’s all we have to say, except, 3,2,1… get playing!
Amazing—we did it!
When we decided to mark Global Legislative Openness Week with a drive to get the data for 200 countries up on EveryPolitician, in all honesty, we weren’t entirely sure it could be done.
And without the help of many people we wouldn’t have got there. But last night, we put live the data for North Korea and Sweden, making us one country over the target.
The result? There is now consistently-structured, reusable data representing the politicians in 201 countries, ready for anyone to pick up and work with. We hope you will.
That’s not to say that our job is over… far from it! There’s still plenty more to be done, as we’ll explain below.
Here’s how it happened
Getting the data for each country was a multi-step process, aided by many people. First, a suitable online source had to be located. Then, a scraper would be written: a piece of code that could visit that source and pull out the information we needed—names, districts, political parties, dates of office, etc—and put it all in the right format.
Because each country’s data had its own idiosyncrasies and formatting, we needed a different scraper for every country.
Once written, we added each scraper to EveryPolitician’s list. Crucially, scrapers aren’t just a one-off deal: ideally they’ll continue to work over time as legislatures and politicians change.
The map above shows our progress during GLOW week, from 134 countries, where we began, up to today’s count of 201.
mySociety’s Tony, Lead on the EveryPolitician project, worked non-stop this week to get as many countries as possible online. But this week we’ve seen EveryPolitician reach some kind of momentum, as it takes off as a community project. It’s an ambitious idea, and it can only succeed with the help of this kind of community effort. Thanks to everyone who helped, including (in no particular order):
Duncan Walker for writing the scraper for Uganda; Joshua Tauberer for helping with the USA data; Struan Donald for handling Ecuador, Japan, Hong Kong, Serbia and the Netherlands; Dave Whiteland, with ThaiNetizen helpfully finding the data source for Thailand; Team Popong for South Korean data; Jenna Howe for her work on El Salvador; Rubeena Mahato, Chris Maddock, Kätlin Traks, François Briatte, @confirmordeny, and @foimonkey for lots of help on finding data; Henare Degan and OpenAustralia who made the scraper for Ukraine; Matthew Somerville for covering the Falkland islands and Sweden; Liz Conlan for lots of help with Peru and American Samoa; Jaroslav Semančík who provided data for, and assistance with, Slovakia; Mathias Huter who supplied current data for Austria while Steven Hirschorn wrote a scraper for the historic data; Andy Lulham who wrote a scraper for Gibraltar; Abigail Rumsey who wrote a scraper for Sri Lanka; everyone who tweeted encouragement or retweeted our requests for help.
But there’s more
There are still 40 or so countries for which we have no data at all: you can see them here. This week has provided an enormous boost to our data, but the site’s real target is, just like the name says, to cover every politician in the world.
And once we’ve done that, there’s still the matter of both historic data, and more in-depth data for the politicians we do have. Thus far, we mostly have only the lower houses for most countries which have two — and for many countries we only have the current politicians. Going into the future we need to include much richer data on all politicians, including voting records, et cetera.
Meanwhile, our first target, to have a list of the current members of every national legislature in the world, is starting to look like it’s not so very far away. If you’d like to help us reach it, here’s how you still can.
International emergency aid charity Médecins Sans Frontiers are one of the biggest purchasers of medicine worldwide, and naturally it’s important that the drugs they buy are cost-effective. Where possible, they choose generics—white label medicines that contain the same ingredients even if they don’t carry the well-known brand names: think ‘ibuprofen’ or ‘aspirin’ rather than ‘Nurofen’ or ‘Anadin’.
But when a specific medicine is only available as a patented product from a big drugs company and with an equally big price tag attached, MSF, like everyone else, has little choice but to pay.
Curiously, this turned out to be a problem that can be solved, in part, through good web design. Here’s the story.
Obviously, drugs companies have an interest in keeping their medicines under patent. As MSF explained, patents, and in particular the practice of ‘evergreening’ them (extending their life indefinitely by making slight modifications to the medicine’s make-up), give pharmaceutical companies a monopoly on pricing, and can impede access to patients who would benefit from them.
MSF’s online project, the Patent Oppositions Database (PODB) is a resource for helping people challenge medicine patents. PODB helps groups around the world to find each other and work on cases together, and to share previous examples of art and arguments used in lawsuits which may help others in future oppositions.
The site was already up, running and functional, and the concept was sound. But it wasn’t attracting much take-up. On analysis, it became clear that this was because there was no focused experience on the site, encouraging users towards the core interactions which would power the whole concept of collaborating and sharing knowledge.
Where design came in
MSF asked us to suggest improvements that would enable groups to communicate about specific cases, and to improve the sense of community. Our solutions will add intuitive user paths that lead people to existing opposition cases and the information they need, then encourage them to join in by placing discussions and information about contributors on the page.
It’s crucial for MSF that the project reaches its full potential, and with the in-depth design changes we’ve suggested, and have now been asked to implement, we know it will.
You can read more about how we approached this project in our latest case study, over at the mySociety Services website.