Jenna Corderoy, Alaveteli Professional Advocate, brings us an update on the project.
Since our last blog post on Alaveteli Professional — our Freedom of Information toolkit for journalists, campaigners and activists — there have been a few exciting developments.
The batch request feature is coming along nicely: this will allow users of the service to send one Freedom of Information request to multiple authorities and help them to easily manage large volumes of responses.
We’re going to be working with a small group of our beta testers to develop this feature and make sure we release it in a useful and responsible form (click here to apply as a beta tester and get a year’s free access to WhatDoTheyKnowPro, the UK version of the service).
We’ve been pleased to see the first news story to emerge as the result of a request made through WhatDoTheyKnowPro: a response to the Foreign and Commonwealth Office showed which are the countries where UK holidaymakers are most likely to get arrested. The full list was covered in the Birmingham Mail.
But we also have plans for this Freedom of Information toolkit to go international: Alaveteli Professional will be a bolt-on option for anyone already running an FOI site on our software platform Alaveteli.
In April, mySociety team members traveled to roll out the first such project, with Info Pro Všechny, the Czech Republic’s Alaveteli site.
We were able to introduce beta users to the features we’ve been developing, such as the ability to keep requests private until the story has been published.
While in the Czech Republic, we held a roundtable discussion with journalists and campaigners, swapping Freedom of Information battle stories and sharing tips and tricks for getting the best results when submitting requests for information, as well as experiences of filing requests to European Union institutions.
mySociety was also invited to give a talk to student journalists based in Olomouc about Info Pro Všechny and Alaveteli Professional in general, discussing success stories generated from our Freedom of Information sites from around the world.
We’re currently working on subscription options, which will allow us to officially launch WhatDoTheyKnowPro as a paid-for service in the UK, and later in the year, we plan to introduce the Pro toolkit to the Belgian Alaveteli site Transparencia.be, which has been making a splash in Belgian politics.
If you’re in a wheelchair, it can be tricky enough getting around. So it’s particularly disappointing to learn that some taxi firms charge wheelchair passengers extra, and that some drivers refuse to take passengers in wheelchairs at all.
If you’re thinking ‘surely that’s illegal’ — well, it is. Only from quite recently, though: it was last April that a law came in which imposed a £1,000 fine for drivers who refused or charged extra for those in wheelchairs.
But there’s a complication. This fine can only be imposed by councils who keep a designated list of all wheelchair-accessible public hire vehicles: no list, no fines.
Does it matter? Well, that depends on how many councils are intending to compile the list. And as WhatDoTheyKnow volunteer Doug Paulley knows very well, there’s one good way to find out information from every local authority: via a Freedom of Information request.
Doug used WhatDoTheyKnow to submit FoI requests to all 366 taxi licensing councils, and Transport for London, who administer taxi licensing on behalf of all the London boroughs. The results of his research can be seen in full here, or you can quickly check your own local council on this map.
As indicated, if your council is one of the 59% who, by not keeping a list, are unable to implement the anti-discrimination law, you might like to contact your councillors to let them know how you feel about that.
You may know Dr Ben Goldacre from his ‘Bad Science’ and ‘Bad Pharma’ campaigns, which fight misinformation around medicine. Ben has just launched his latest project, the AllTrials Transparency Index — and mySociety helped with the website side of things.
The AllTrials campaign focuses around the fact that a shockingly large proportion of clinical trials do not have their results publicly published.
Not only does this devalue the time, goodwill and even potential risk put in by participants, but there are also issues around bias. Those trials published tend to be the ones which show positive results: if that doesn’t sound like such a terrible situation to you, try playing this game from the Economist magazine, which graphically depicts the problems with skewed coverage. At worst, such selective publication can be dangerous, or lead to poor choices from bodies making medical purchasing decisions.
Transparency and data visualisation are two areas where mySociety has a long history, and so it came to be that we fashioned the deceptively simple AllTrials Transparency Index site, on which anyone can browse the transparency index of the world’s major drug companies, and dive in deeper to the data to see how it was compiled. The source data is free for others to download too, so anyone can integrate it into other projects.
AllTrials are also tracking whether companies register new trials:
That’s the part that anyone can understand — and now, notes for the more technically-inclined who may be wondering how we took the data on each drugs company and presented it in a way that can be quickly and easily taken in.
This is an ongoing campaign with a commitment to future audits, so we wanted to make it easy for the AllTrials team to update the site and republish the source data each time they do.
It’s a static Jekyll site. We wrote a custom plug-in to parse a CSV and produce a page for each company within that CSV, as well as creating some summary data that feeds into the graphs on the front page.
This data is then pulled from the CSV, and D3 is employed to build the graphs and insert them into the generated pages.
The end result is a site that looks good and which can automatically update whenever the underlying data changes. We hope we’ll have played a small part in helping to ensure that it does — and for the better.
What would Eddie Grundy do if he came across a pothole? And how would Linda Snell deal with flytipping on the site of the Ambridge village fete?
Fortunately, these fictional characters now enjoy the same access to FixMyStreet as the rest of us, thanks to the new demo site we’ve built.
The thinking behind it is not, of course, to gather reports from an entirely fictional world. We’re not that mad. Rather, we needed a sandbox interface where we could show councils exactly how FixMyStreet works, and allow them to play about with both the customer end and the admin side, all without causing any major repercussions to the running of the standard site. Enter FixMyStreet Borsetshire.
Prospective buyers of the system from local councils can experience the various levels of administration that the back-end allows. Just log in with the credentials seen on this page and see exactly how reports can be shortlisted, actioned, or moderated.
So, we’re expecting reports of pigs on the loose, flooded culverts and perhaps even a flying flapjack. But if you’re hoping to find out the precise location of Ambridge, unfortunately you’ll be disappointed: the map is actually centred around Chipping Sodbury, far from the village’s supposed Midlands locale.
For the last few years mySociety’s research output has been living in its own little area of the main website. At the start this was fine, but as we’ve produced more research (which is good!) the website was not good at making clear what we had previously released and why you should read it (which is bad!).
To fix that we’ve brought all our research reports, papers and blog post together in one place. We also wanted to take the opportunity to make our research easier to access. For all our research going back to 2015, we now have a nice, mobile-responsive, easy-to-read version, as well as a text and a kindle .mobi file to go along with that. In several cases papers that had been published externally were released by the publisher under a Creative Commons licence – meaning these could be converted to the new format.
And don’t forget that you can sign-up for our research newsletter for exciting research updates!
Image: Nico Kaiser
Access to information is a particularly powerful tool in countries like Colombia, where corruption is high and vital peace treaties are underway.
To make accessing information easier for citizens and public authorities alike, a group of journalists in Bogotá including DataSketch, have recently set up the Freedom of Information request platform QueremosDatos (the name of which translates as “We want data/information”).
The platform uses our Alaveteli software, and we thoroughly enjoyed working with the Colombian team to set the site up with them.
We asked María Isabel Magaña, who is coordinating the QueremosDatos project, about the site and its impacts so far:
Why did you decide to set up QueremosDatos?
I first learned about Alaveteli in Spain while I was doing my Masters in Investigative Journalism. There I was introduced to the platform TuDerechoASaber.es and to the power that FOIA and transparency had. I just knew Colombia needed something like that, especially since the Congress had just approved the first law regarding this matter.
What made you choose to use Alaveteli software for your platform?
What I love about Alaveteli is how easy it is to use for both users and admins. Designing the platform and making it useful for any type of person was the most attractive feature Alaveteli had. But also, because of the people behind it. Gemma, Gareth, and so many more people were ready to help me achieve this goal despite the different time zones and how much time it took to get it up and running.
What impact do you hope the site has?
It has been almost six months since we launched the site. The impact has been great! We have helped people make 274 requests to more than 6,000 authorities. The Government has been interested in the project and has helped us get in touch with different authorities to help them learn about FOIA and the Colombian law and how to work with people through the platform. Users love it, especially journalists.
Which responses on the site have you been most excited about seeing?
My favourite response so far has been one regarding victims of the Colombian conflict. It was very exciting to get the information because of what it meant for the person who was requesting it, and because of the historical context my country is going through. I also enjoyed seeing the transformation the police had when giving their answers: at first they always sent a response asking the user to call them. After a few explanations, they’re now sending complete answers to the requests via the site.
Do you know of examples where information obtained through the site has been used?
Yes! Journalists have used it mostly in ongoing investigations regarding medicines, drug trafficking, and abortion. Students have used it for journalism classes and homework too.
What are your future plans for QueremosDatos?
We are confirming an alliance with the government to promote the site in public offices and to teach public servants about what the Right to Know is, and their responsibilities with it. This pedagogy will be replicated in universities to teach different users about their power to request information.
Many thanks to María for answering our questions. It’s been great to see the impact the site has already had on authorities and citizens alike, especially the change in behaviour by certain public authorities.
We’re really looking forward to following the project’s continuing work, and wish the team the utmost success in their quest to make Colombia a much more transparent society!
Artificial intelligence and machine learning seem to be everywhere at the moment – every day there’s a new story about the latest smart assistant, self-driving car or the impending take over of the world by robots. With FixMyStreet having recently reached one million reports, I started wondering what kind of fun things could be done with that dataset.
Inspired by a recent post that generated UK place names using a neural network, I thought I’d dip my toes in the deep learning sea and apply the same technique to FixMyStreet reports. Predictably enough the results are a bit weird.
I took the titles from all the public reports on fixmystreet.com as the training data, and left the training process to run overnight. The number crunching was pretty slow and the calculations had barely reached 5% in the morning. I suspect the training set was a bit too large, at over 1M entries, but end result still gives enough to work with.
The training process produces checkpoints along the way, which you can use to see how the learning is progressing. After 1000 iterations the model was starting to be aware that it should use words, but didn’t really know how to spell them:
Mertricolbes Ice does thrown campryings Sunky riking proper, badger verwappefing cars off uping is! Finst Knmp Lyghimes Jn fence Moadle bridge is one descemjop
After 15000 iterations it’s starting to get the hang of real words, though still struggling to form coherent sentences.
Untaxed cacistance. Broken Surface in ARRUIGARDUR. Widdy movering Cracked already nail some house height avenue. Light not worky I large pot hole Dumped shood road nod at street. Grim Dog man Ongorently obstructing sofas. This birgs. Serious Dirches
After 68000 iterations there seems to be enough confusion in the training data that things start to go south again with the default parameters:
Urgely councille at jnc swept arobley men. They whention to public bend to street? For traffic light not working
Tweaking the ‘temperature’ of the sampling process produces increasingly sensible results:
Large crumbling on pavement Potholes all overgrown for deep pothole Very van causing the road Very deep potholes on pavement Weeds on the pavement Several potholes in the road Rubbish Dumped on the road markings Potholes on three away surface blocking my peride garden of the pavement Potholes and rubbish bags on pavement Poor road sign damaged Poor street lights not working Dog mess in can on road bollard on pavement A large potholes and street light post in middle of road
As well as plenty of variations on the most popular titles:
Pot hole Pot hole on pavement Pot holes and pavement around Pot holes needings to path Pothole Pothole dark Pothole in road Pothole/Damaged to to weeks Potholes Potholes all overgrown for deep pothole Potholes in Cavation Close Potholes in lamp post Out Potholes in right stop lines sign Potholes on Knothendabout Street Light Street Lighting Street light Street light fence the entranch to Parver close Street light not working Street light not working develter Street light out opposite 82/00 Tood Street lights Street lights not working in manham wall post Street lights on path Street lights out
It also seems to do quite well at making up road names that don’t exist in any of the original reports (or in reality):
Street Light Out - 605 Ridington Road Signs left on qualing Road, Leave SE2234 4 Phiphest Park Road Hasnyleys Rd Apton flytipping on Willour Lane The road U6!
Here are a few of my favourites for their sheer absurdity:
Huge pothole signs Lack of rubbish Wheelie car Keep Potholes Mattress left on cars Ant flat in the middle of road Flytipping goon! Pothole on the trees Abandoned rubbish in lane approaching badger toward Way ockgatton trees Overgrown bush Is broken - life of the road. Poo car Road missing Missing dog fouling - under traffic lights
Aside from perhaps generating realistic-looking reports for demo/development sites I don’t know if this has any practical application for FixMyStreet, but it was fun to see what kind of thing is possible with not much work.
Help us innovate in Civic TechnologyDonate now
This month, FixMyStreet.com sent one more report off to a council. There was nothing to distinguish it from all the other reports of fly-tipping, potholes and graffiti… except that it was the one millionth to be sent since the site began.
Back in 2007, when mySociety first launched FixMyStreet, we had a feeling it’d be useful — but we couldn’t have foreseen the take-up it’s had not only here in the UK, but across the world and in many forms. One million seems like a real milestone, so in celebration, here’s a whistle-stop tour of FixMyStreet’s life so far.
First through the doors
The first report ever sent to a council through FixMyStreet was this one, concerning a broken streetlight.
It was created by a mySociety staff member during beta testing of the site, and sent off to Oxford City Council — who fixed the streetlight. Proof of concept, and we were off.
Once it was clear that everything was working smoothly, FixMyStreet had its official launch that March.
Those who know and love FixMyStreet may be surprised to hear that in this first incarnation, it was given the slightly less snappy title of Neighbourhood Fix-It.
Just a week after launch, users had already filed over 1,000 reports — a sign that there really was a need for this site.
The reasons for its popularity? After all, all councils these days provide a fault-reporting system themselves, so why the enthusiastic take-up of a site that duplicates this functionality? We think the reasons are twofold:
- You don’t have to worry about which council is responsible for an issue: FixMyStreet just automatically sends it off to the right one. There are lots of reasons why you may not know where to send a street report, not least the UK’s two-tiered system of local authorities.
- We make the reporting process as simple as possible. It’s that whole ‘swans looking graceful but paddling like crazy under the waterline’ thing: we put in an awful amount of work to make sure that you don’t even notice the issues FixMyStreet has to deal with to make the user experience super-smooth. Back in 2012 we blogged about some of the thinking behind the site; for example here’s why FixMyStreet begins by asking just one simple question.
By June we’d realised that Neighbourhood Fix-It wasn’t the snappiest of names, and thus was born FixMyStreet as we know and love it.
In June 2008, Apple launched their app store.
Our developers saw the future, it seems: by December that year, we’d launched a FixMyStreet app (NB, the links in that 2008 post don’t work any more: if you’d like current versions of the app, you’ll find them here for Apple and here for Android).
The FixMyStreet apps have been downloaded more than 40,000 times, and we’re seeing a real growth in those who use it to make their reports: in the last year it accounted for 27% of reports. This reflects a general increase in the use of mobile (you can also use your mobile’s browser to access www.fixmystreet.com) — 55% of our visitors came via a phone or tablet in the last year.
Open for re-use
Like most mySociety software, the code that FixMyStreet runs on is Open Source: that means that anyone can pick it up for free, and run their own site on it.
In March 2011, a group of coders in Norway were the very first to do this, with their version FiksGataMi (it means FixMyStreet in Norwegian. They could have gone for Nabolaget Fikser Det, which means Neighbourhood Fix-It, but, well, you know…).
Since then, we’ve made real efforts to make the code easier for others to deploy, and ensured that the improvements we add to our own FixMyStreet are also available for all the others: just recently we rolled out version 2.1 of the codebase.
Taking a peek to see what’s being reported around the world is one of our favourite, if non-standard, means of armchair travelling.
A Norwegian puddle-prone footbridge gets in the way of christenings, confirmations and school meetings; meanwhile in Spanish city Alcalá de Henares, a resident complains about the smell created by rubbish lorries while allowing us a splendid view across the rooftops; and in Malaysia, a pack of stray dogs is causing problems for one reporter.
We’d wanted to provide a reporting system that bettered those offered by local councils: in June 2012 that goal was seemingly affirmed when some councils purchased the system to place on their own websites.
We officially launched FixMyStreet for Councils, with Bromley and Barnet being the very first local authorities to implement it. Since then, we’ve been in a continual process of improvement, driven by input and collaboration with many councils around the country. Several more have become clients, too. We’ll have more news on the latest developments soon (and meanwhile, if you are from a council, you can learn more here).
One of the nicest things about a codebase like FixMyStreet is that it can be deployed in many — sometimes surprising — ways. If you’ve followed our blog over the years, you’ll have seen the Channel 4 collaboration Empty Homes Spotter; the bicycle incident-reporting platform Collideoscope; and a project fighting corruption in Malaysia.
Bringing out the poetry in potholes
There’s something about FixMyStreet that inspires some users to exercise their powers of descriptive prose: we celebrated many of them in this 2014 post.
Then there are the reports which attract comments from other users. Lots of them, year in, year out. This one about seagulls in Brighton, for example, has become a one-stop forum for people all around the country to come together in their mutual despair of and/or love for our coastal avian friends.
Ever more reports
You can track the progress as we head towards the next million reports on our new stats page; where you might also be interested to see which councils are currently responding to issues most quickly, and what categories of problem are most-reported at any given time.
As you can see, at the moment the site is handling around 4,000 reports a week: but you can expect that to rise when the weather gets colder — we always get a lot more pothole reports in the winter.
And, are you wondering just what that millionth report was about? Nothing is ever simple: because some reports are made and then subsequently deleted at the user’s request, or because they contravene FixMyStreet’s house rules, we can’t just identify report number 1000000 as the millionth. Those deleted reports retain their original numbers, even though they’re not live.
But doing a quick bit of calculation, we suspect that the rightful millionth report might be this utterly unremarkable one in Knowsley. Long live the unsensational reports that simply get things fixed.
Help us run our citizen-empowering sitesDonate now
Image: Alison Benbow (CC by/2.0)
As we shared back in April, this September we’ll be hosting an extra edition of our TICTeC research conference, in Taipei.
TICTeC, or The Impacts of Civic Technology Conference, is the Civic Tech sector’s only conference that’s dedicated to promoting and sharing research into the impacts of online technologies and digital democracy around the world, to share what works and (crucially) what doesn’t.
TICTeC@Taipei will be the headline event at the Open Culture Foundation’s Civic Tech Fest, a week-long festival featuring a series of conferences, workshops and hackathons related to open data and open government. And Civic Tech Fest itself is an official parallel event of the World Congress on Information Technology (WCIT), one of the world’s largest gatherings of the IT industry.
It’s a really unique opportunity to bring together researchers and practitioners from across the globe in Asia’s Civic Tech hub, and to showcase our sector’s initiatives to the wider IT industry.
We’re delighted to announce that the TICTeC@Taipei agenda is now online. The conference will feature speakers from the Omidyar Network, UNICEF, the Web Foundation, government ministries of France and Taiwan, leading universities, and many more.
Early bird tickets are still available until 21st July and registration includes entry to all Civic Tech Fest and WCIT events. We’ll even be having the TICTeC after party in the Taipei 101 building (the tallest building in the photo above), at Google’s Taipei offices. Pretty cool huh?
Don’t miss out on this unique opportunity — book now!
Image: sama093 (CC BY-NC-ND 2.0)
This post is by Tony Bowden and Lucy Chambers from the EveryPolitician team. Today we officially launch our collaboration with Wikidata – here’s what to expect…
The story so far
You might have been following the progress: since 2015, through our project EveryPolitician, we’ve been gathering data for every national legislature in the world, from thousands of sources, and sharing it.
Now, two years on, we’ve started to see some great results. For example:
- It’s much easier to build simple Parliament Tracking sites (as mySociety partner organisations have in Zimbabwe and Nigeria). Those running the sites can work on providing information and context to hold politicians accountable — and don’t have to worry about wrangling data and software.
- Tools that allow citizens to write to their representatives (like Majlis Nameh in Iran, based on the WriteInPublic software, or Oxfam’s UK and Australia Campaigning Tool) can now be deployed in days rather than weeks, allowing groups to focus on local customisation.
- Vote-tracking sites like TheyVoteForYou in Australia can be more easily adapted to other countries — such as Ukraine, again without worrying about the major task of sourcing the politician data.
- Projects that highlight politicians’ activity can be augmented to show extra information against those politicians — so for example, Politwoops can show party affiliation next to politicians’ tweets.
- Investigative journalists can cross-match our lists of politicians to other sources (e.g. investigating shell companies for a recent Private Eye article or with the Panama Papers).
- mySociety’s collaboration with Facebook makes it easy for people to connect with their newly elected representatives.
All of this is possible because of two main tenets of EveryPolitician: having gathered the data, we structure it consistently, and we share it freely.
Not just current data
We’ve also discovered that there is a huge value for everyone in retaining historic information, too:
- Old data tends to disappear from many official sites, for example, when a new government comes into power and / or when parliaments decide to remodel their websites.
- Sometimes even the official sources no longer exist: for example, in Burkina Faso the Parliament building was burned down by protesters in 2014 and many crucial documents were lost.
- Research projects and sites run by civil society organisations sometimes run out of funding and have to shut down — meaning that the data people were relying on can vanish overnight.
The future: going one step further
So that’s all great, but we think EveryPolitician can do still more to help the worldwide community of Civic Tech coders and activists.
In particular, we want to go beyond data showing who the politicians are, and also provide information on what they do. That’s because we can see real value in the ability to answer questions like:
- When we look at politicians who vote on issues such as gay marriage, smoking bans, tax on sugary drinks, etc, anywhere in the world — are there any broad correlations like age or gender?
- As countries elect more women MPs, do those women also gain equivalent representation in committees? Does this affect attendance and participation rates?
- Are there standard political career paths that can be observed anywhere in the world? So for example, do certain Cabinet positions limit future progression; and if so, would it be feasible to spot politicians who are on the way up, or on their way out?
- Do politicians who move from the lower to upper house act, and vote, differently?
- Do politicians change their voting activity after certain types of intervention, for example after receiving funding from oil companies?
… and there are undoubtedly millions more questions, each one just as interesting and with answers that could enrich our understanding of the world. We are aiming for a future in which each should be answerable within minutes, rather than form the basis of a multi-year post-graduate research project.
How we plan to get there
You may remember our recent proposal to integrate more deeply with Wikidata. We’re delighted to say that our proposal was accepted, and that makes EveryPolitician’s path very clear.
Wikipedia is fast becoming one of the best sources of political information in many countries: it’s often updated more quickly than major news outlets or official parliamentary sites are.
Which is great, but there’s an issue when it comes to using that information for projects like the ones we’ve mentioned above: Wikipedia contains largely unstructured information (that is to say, information that comes in a wide variety of different formats) — as you’d expect from any project with multiple contributors and, often, a free-text input.
There are also a lot of differences between Wikipedias in different countries. Some countries’ data (particularly countries where the Wikipedia community is larger) gets updated very quickly. Smaller language Wikipedias can’t rely on such a large pool of editors, and it tends to be longer before they are updated.
Additionally, and as you might expect, Wikipedia content will appear fastest in the countries most directly affected by the change being documented, so for example when there are elections in Estonia, the Estonian Wikipedia may show the results almost immediately, but it can take a while for those changes to trickle into all languages.
But of course, nothing’s ever quite that simple. While Wikipedia has a wealth of unstructured political information, on Wikidata there’s still an awful lot of data missing. You may recall that we recently ran a drive to ensure that every country at least had its head of government entered, but that’s just the beginning: in order to answer the kinds of questions we mention above, we really need to ensure there’s consistently-structured information for all legislators, all elections… and much more.
What you’ll see in the coming months
While we plan for EveryPolitician to retain its own identity and keep its own front end, we’re excited to say that over the next few months, we’ll be teaming up with Wikidata communities across the world.
Our first objective is see if we can bring Wikidata up to the same level as EveryPolitician for as many countries as possible.
And once we hit that target, we plan to go much further. The beauty of Wikidata is that you can add pretty much any information you want to add to politicians (indeed, to anything!), so local communities can decide for themselves which information is most pertinent and make sure that it’s included.
To help us make all of this happen, we’re expanding. If you’ve read this far and you’re still finding the project interesting, the chances are you’d be a great addition to the team! We’re looking for a new community co-ordinator, working from anywhere in the world (compatible timezones allowing): you can see all the details here.
How you can get involved
There are plenty of ways that you can help with this drive to improve the structured information on Wikidata. Here are the obvious ones:
- There are active missions always ongoing on the Wikiproject Heads of State and Government. There’s one there now – check them out!
- Would you be interested in organising a push to Improve Wikidata for your country? Get in touch on email@example.com and we’ll do what we can to help.
- If this all sounds interesting, but you’re not sure where to start, or you’re unfamiliar with Wikidata – drop us an email and tell us what you’re interested in. We’re more than happy to help you get started.