Are you investigating, researching or gathering large quantities of data through Freedom of Information requests? Perhaps you’re a journalist, academic or NGO. We’re looking people based in the UK who’d like to try out our new ‘Projects’ feature for WhatDoTheyKnow Pro.
Projects allows you to crowdsource the extraction of data from multiple (or batch) FOI requests made to multiple authorities. You can set up a project with a brief description of what it is and what you are hoping to achieve, and some tasks that volunteers can complete to help you with this aim (like categorising responses, or answering questions about the data released).
Once that’s done, you can set it up to invite volunteers, who can help you to extract all the information you need from the released responses.
You’ll be able to download your volunteers’ input as a spreadsheet, meaning analysis of the data is much quicker and easier — so you can get on with the task of forming conclusions and writing up your findings.
What we’ll need from you
Projects is still in its nascent stage, so we need feedback from our testers. This will help us improve the service and tailor it to users’ needs, based on real life use cases.
Right now, we handle the setup and importing of the requests you want to work on manually (that is, our developers have to do it) — but we’re working on improving this aspect, and your feedback will be crucial in shaping the direction our development takes. We’re also looking for general comments, once you’ve used the service, on what’s useful and what’s missing; what you tried to do but couldn’t, and what made things easier for you.
If this sounds interesting, please get in touch at email@example.com. We look forward to hearing from you!
Image: Jessica Lee
At LocalGovCamp, our designer Martin ran an interactive exercise that took attendees through a ‘consequence scanning’ exercise, as a way to predict and mitigate all the outcomes, both positive and negative, of a proposed piece of development.
In this case, the service under discussion was a fictional parking violation reporting app.
Let’s just repeat that, in case of any angry reactions: fictional!
So, what could possibly go wrong with a piece of tech designed to encourage residents to grass on fellow citizens for their poor parking? You can see how it played out in this video:
Now you’ve seen a consequence scanning exercise in action. If you’d like to understand more about the process, read on: this is how Martin explained the whole idea to us here at mySociety, with more detail on the underlying principles:
We’ve been working on a few sensitive projects recently – specifically our work expanding FixMyStreet Pro to cover issues of a more social nature, like noise reporting, antisocial behaviour, that sort of thing.
As experienced as we are with the ‘make a report by sticking a pin in a map’ style of interaction design, we recognise the need for extra care when applying this to issues that are about people, rather than things. There’s an increased risk of building a tool that results in unintended negative consequences; especially where the service concerns an area already prone to controversy.
mySociety Board member Jonathan Flowers put us in touch with Connected Places Catapult, who had been using ‘Consequence Scanning’ for this very thing, and we realised it was just what we needed.
It’s a structured system for drawing out the consequences of a new idea, and giving people a say in what actions are used to mitigate or address them. It originated from the Doteveryone thinktank, and CPC have taken it forward and customised it for their needs.
In Consequence Scanning, consequences are classified as either intended or unintended, with the important distinction that intended consequences aren’t always positive, and unintended consequences aren’t always negative.
The process is delivered in a workshop format and works best with a good mixture of participants with diverse views and backgrounds, directly involved in the service on both sides. This means ideally both service users and service officers should take part and be prepared to be honest about consequences. For this reason it’s important to create a safe space where information can be shared honestly and openly.
The process is split into three parts:
Part one: What are the consequences?
Part two: What are the positive consequences we want to focus on?
Part three: What are the unintended consequences we should mitigate?
Part one: What are the consequences?
- What are the intended consequences for:
- Organisation – How might this affect our organisation?
- Users – How might this affect the users of this service?
- Community – What are the consequences that could affect the wider community?
- What are the unintended consequences? For the kind of work we do, unintended consequences tend to emerge in these areas:
- Lack of digital understanding:
- What can happen in a situation where there is a lack of digital skills or access to technology?
- Unintended uses and users
- What could be the unintended uses of this service?
- What could be the unintended users of this service? Eg private companies using public services for profit
- Weak security/reliability/poor support/monitoring
- What could happen in situations of technical failure, poorly equipped staff, or lack of budget etc?
- Changes in norms and behaviours
- How could this cause changes in societal norms and behaviours?
- Displacement (what will people do this instead of… )
- If people use this service instead of others what could result?
- Impact on environment
- How might this service result in consequences for the planet or local environment?
Part two: What are the positive consequences we want to focus on?
- Sort the list of intended consequences into groups by affinity (affinity sorting)
- Add further details or related information
Part three: What are the unintended consequences we want to mitigate?
- Sort the list of intended consequences into groups by affinity (affinity sorting)
- Use causal mapping to work out the relationships between the consequences and help determine where mitigations could have the greatest impact: eg, solve A before B, solve D and prevent E,F,G
- Use grouping and categorisation of consequences to show relationships
This system works best on a new, but defined idea. If it’s done too early in the design process, the consequences end up being very general, or people bring their own assumptions and often focus on the wrong things. It’s best to bring it in once scope has been defined.
The primary function is to identify the consequences and not to “solutionise” the mitigations, but the group should be free to discuss possible mitigations where they feel it’s important.
We’ve been using Consequence Scanning in our work on noise reporting and antisocial behaviour, and it’s also proving useful for our internal anti-racism action group, where we want to understand the potential unintended results of any future development in terms of who our services reach, and who they exclude.
Image: Drew Graham
This work will support users in taking the next steps, if appropriate, when their requests for information are denied.
A bit of background
In the last few years, there has been a significant and sustained decline in FOI requests being granted by the UK government.
According to the Institute for Government, the proportion of refused FOI requests reached a record level in the third quarter of 2019, with departments refusing to comply in full with more than half of all FOI requests that they received. This compares to around 40% in 2010 and around 30% in 2005.
And yet, our research found that, when challenged, a large proportion of refusals were overturned, suggesting that the fault did not lie with the type of request being made. 22% of internal reviews resulted in the full or partial release of information, and a further 22% of appeals to the ICO led to all or some of the information being released.
For local authorities, up to half of internal reviews – and just over half of all ICO appeals – led to the release of all or some of the information requested. In Scotland, with its own FOI regime, 64% of appeals to the Information Commissioner resulted in the full or partial release of information.
And so, while acknowledging that some refusals are certainly legitimate, there is a clear case for challenging such responses. But to do so is daunting, especially for novice requesters who can understandably be discouraged by an official response citing exemptions in legalese.
This new funding will allow us to approach the issue from four different, but interlinked directions, each intended to inform and support users in challenging government refusals of FOI requests.
- When a WhatDoTheyKnow user confirms that they’ve received a refusal, we’ll be integrating context-sensitive advice. This will inform the user of their right to appeal, give clear guidance on how to assess whether the authority has complied with the law, and also advise on other channels, beside FOI, by which information may be obtained.
- We’ll automatically identify which exemption has been cited in the refusal, giving us the ability to help users better understand why their request has been turned down.
- Based on this finding, we’ll offer context-specific advice for the exemption identified. For example, if the request has been turned down because of cost, we’ll show how to reframe it to fall below the ‘appropriate limit’.
- Finally, once the user has been fully informed, we’ll offer the support they need to escalate the request to an appeal.
Ultimately we hope that this work will help reset the balance on the public’s right to access information, better enabling citizens, journalists and civil society to effectively scrutinise and hold authorities to account.
As always, we’ll also be thinking hard about how to make all of this apply more universally, across the various legislatures that apply in jurisdictions where people are running sites on the Alaveteli platform.
If this interests you, watch this space. We’ll be sure to update when we’ve made some progress on the project.
Image: Tim Mossholder
We’ve added a new functionality to the Alaveteli Pro codebase, allowing you to download a zip file containing all correspondence and attachments from a batch, and a spreadsheet (csv) to show the progress status of every request.
Alaveteli Pro is our tool for professional users of Freedom of Information. If you’re UK-based, you’re probably most familiar with our local iteration WhatDoTheyKnow Pro — but don’t worry: when we talk about improvements to Alaveteli Pro, you can be sure they’re also part of the WhatDoTheyKnow toolkit.
How to export
You’ll find these tools at the foot of the batch container in the requests list.
Why data exports?
Of course, we like to think Alaveteli Pro is a useful tool in its own right: there’s a lot you can do within the Pro interface, and it was built specifically to help you keep track of all your FOI activity in one place.
But sometimes users want to use external tools – either because they’re just more familiar with them, or because they want to do something beyond the functionality we offer.
Now there’s a simple way to get data out of Alaveteli, allowing you to analyse it with the tools of your choice, or perhaps send a progress report to a supervisor or editor.
It’s part of a programme of work to support cross border journalism between European organisations, supported by Adessium Foundation, allowing us to refine and improve the codebase for the benefit of all Pro users.
The technical bit
Those with a bit of coding knowledge may be interested to hear how we approached the zip download functionality. mySociety developer Graeme explains:
“With batch requests potentially going to as many as 500 different authorities, each request can receive several responses and attachments in return.
“All these emails and files mean that compiling the zip for download could be a lengthy job and would normally cause the request to time out. So for this new feature we’re utilising file streaming to send chunks of the zip as they become available.
“This means that the zip starts downloading immediately and you don’t have to sit watching and wondering whether anything is happening – you can see more and more data being transmitted.”
We hope you find this new feature useful. Please do let us know how you’re using it and any feedback you may have.
Image: Startup Stock Photos
This brings some substantial improvements to the code. The update is available to anyone running a site on the FixMyStreet platform, which includes our own fixmystreet.com; the installations we provide for councils and authorities; and the FixMyStreet instances run by others, in places from Australia to Uruguay.
If you run a site on the FixMyStreet platform yourself, or are just interested in the technical details, you can read the release notes here.
Meanwhile, here’s a rundown of the new front-end features you might notice if you’re a user of FixMyStreet.
Run the site as an app
FixMyStreet can now be added to phones (and desktops for that matter) as a ‘progressive app’. Here’s what to look for when you visit fixmystreet.com:
On Chrome for Android:
Access from the bar at the bottom of the screen.
Click the share icon at the foot of the screen.
Then select ‘add to home screen’.
On Firefox for Android:
Look for the pop up notification or tap the home icon with a plus sign in it in the URL bar.
Any of these methods will install a version of FixMyStreet that will behave like an app, placing an icon on your desktop, browser start page or home screen.
This way there is no need to download or update from the app store, and changes to the main website (which are invariably released sooner than on the app) will be immediately available to you.
Cobrands (for example the councils that use FixMyStreet as part of their own websites, and people running FixMyStreet in their own countries) can provide their own logo and colourscheme as well.
Mobile browser improvements
Whether you install the progressive web app or just visit fixmystreet.com on your mobile browser, you may notice some nice new features.
- If you use the geolocation function (‘use my location’), your position will be displayed on the map:
- When viewing an area, you can access the filters to narrow the reports displayed down by their status (fixed/open etc) and category:
- If you’re about to report something that looks like a duplicate, you’ll not only be shown the report/s that have already been made, but you’ll also see a small inline map without having to scroll back to the main map to check where they are.
- The site recognises that when you’re on a mobile, the message about uploading a photo shouldn’t invite you to ‘drag and drop’, but rather to either take a new one or select a photo from your phone.
- If you’ve placed the pin incorrectly, the ‘try again’ process is clearer.
If a picture paints a thousand words, then your Twitter character count just went stratospheric. Now, when you share a report on places like Twitter or Facebook, if there’s a photo included in the report, that will also be pulled through.
Previously, the ‘open graph image’ that was shown by default was the same for every report — which could get a bit boring in aggregate, and certainly missed some of the impact that people might want to share when they’re posting about their own, or others’ reports.
Social media isn’t the only place that FixMyStreet reports can be piped to, though — the site also has several RSS capabilities that have been baked in since its early days.
For those not totally up to speed with RSS and what it can do, we’re now no longer displaying them as raw XML but as a nice simple web page that explains its purpose.
To see this in action, click ‘Local Alerts’ in the top menu of any page. Here’s a before and after:
What benefits one, benefits all
Much of this work is thanks to NDI, the National Democratic Institute.
NDI offer the FixMyStreet codebase as one of their DemTools, installing it in countries around the world as an innovation which empowers citizens to keep their neighbourhoods clean and safe.
Thanks to this partnership, NDI funded the addition of new features which they had identified as desirable — and which, thanks to the open codebase, will benefit users of every FixMyStreet site worldwide.
There are some other significant additions in this release, including integration, back end and security improvements, all of which will be of most interest to developers and site admins — so if you’d like to see them, head over to the full write up on the FixMyStreet platform blog.
Image: Max Fuchs
It’s obviously good citizen behaviour to report something that needs fixing to your council, whether it’s a pothole that could cause an accident, or a broken streetlight that has plunged the area into darkness.
But there’s one type of report that isn’t very useful to councils, and in fact brings unnecessary costs and inconvenience: when you tell the council about an issue that’s already been flagged up by someone else.
FixMyStreet has always been helpful in this regard. It was groundbreaking in displaying all reports in public, unlike most council systems when we were first developing it. A user who goes to make a report can see right away if there’s already a pin in that spot, and check whether the existing issue is the same one they were going to add.
Now we’ve taken that concept a step further in some work which we’re trialling on Bath & NE Somerset’s implementation of FixMyStreet Pro.
When a user starts to make a report, the system checks to see if there are any other open reports in the same category within a small radius. If it finds any, you’ll see a prompt, like this:
All similar reports will appear here. If you think one might be identical, but aren’t sure, you can click ‘read more’ to see the full text along with any photos attached to the report:
And if you recognise it as the issue you were about to report, you click the green button and will be given the option to subscribe to it, so you know when it’s being seen to, effectively being kept just as up to date as you would be if you’d made the original report:
If it’s not the same issue, no worries: just click ‘report a new problem’ and you can do just that:
Bath & NE Somerset will run this feature as a trial over the next month; then once they’ve got feedback from their users, we’ll hopefully offer it to every other council on the Avenue tier of FixMyStreet Pro.
If you come across this feature while making a report in Bath or environs, do let us know how it works for you.
Image: Kevin Grieve
If you’re reporting an issue on Buckinghamshire Council’s FixMyStreet installation, you might have seen yellow dots appearing on the map. These represent items such as streetlights, bins or drains, and we blogged about it when we first added the feature.
When it comes to assets like streetlights, it can save the council considerable time and effort if your report tells them precisely which light needs fixing: it’s far quicker to find an identified light than it is to follow well-meaning but perhaps vague descriptions like ‘opposite the school’!
But even when the assets are marked on a map, it’s not always easy for a user to identify exactly which one they want to report, especially if they’ve gone home to make the report and they’re no longer standing right in front of it.
After the system had been in place for a few weeks, the team at Buckinghamshire told us that users often weren’t pinpointing quite the right streetlight. So we thought a bit more about what could be done to encourage more accurate reports.
As you might have noticed, streetlights are usually branded with an ID number, like this:Buckinghamshire, as you’d expect, holds these ID numbers as data, which means that we were able to add it to FixMyStreet. Now when you click on one of the dots, you’ll see the number displayed, like this:
The same functionality works for signs, Belisha beacons, bollards and traffic signals, as well as streetlights. Each of them has their own unique identifier.
So, if you’re in Bucks and you want to make a report about any of these things, note down the ID number and compare it when you click on the asset. This means the correct information is sent through the first time — which, in turn, makes for a quicker fix. Win/win!
This type of functionality is available to any council using FixMyStreet Pro: find out more here.
Header image: Luca Florio
When you consider that FixMyStreet has been running for over a decade, it’s not really surprising that the maps in some areas are a little over-crowded with pins.
That can be a problem for anyone trying to make a new report — even when you zoom right in, we were beginning to find that in some very congested areas, it was difficult to place a new pin without clicking on an existing one.
We’ve tried to remedy this in various ways in the past. For a while we only displayed newer reports by default, a decision which we discarded when we brought in pagination, allowing users to click through batches of reports rather than seeing them all in one long list on a single page.
For some time now we’ve also provided the option to hide the pins completely, via this button both on the desktop and app versions:
And there’s also a ‘hide pins’ option at the foot of the map:
But even so, arriving at a map absolutely covered in pins and having to look around for that button doesn’t exactly seem like a nice, smooth user journey, so we’ve revisited the matter.
Why not just delete the old reports?
We’ve always had a policy of keeping every report live on FixMyStreet (unless it’s reported to us as abusive, or its maker contacts us to ask us to remove it — and even in this latter case we’d prefer to retain the content of the report while anonymising it).
This is because the reports made to councils build up to create an invaluable archive of the issues that various regions of the country face, through time.
The historic collection of reports allows planners to understand recurring or seasonal problems; and researchers use this data as well, to get insights into all sorts of issues. For examples, see Réka Solymosi’s presentation at TICTeC on using FixMyStreet data to understand what counts as ‘disorder’ in the environment, or mySociety’s own research on why some areas of the country report on FixMyStreet more than others.
And so here’s what we’ve done
- When you visit a map page on the main FixMyStreet site, by default, you’ll again only see reports that are less than six months old, and that are still open.
A report remains ‘open’ until the council marks it as ‘closed’, or a user or the council marks it as ‘fixed’. ‘Closed’ means that the council doesn’t intend to do further work on the issue, which can be for reasons such as the issue not falling within their responsibilities or because it is part of their regular maintenance schedule and will be seen to in time.
- You can still opt to see closed and fixed reports by selecting from the dropdown at the top of the list:
- And you can also still see reports older than six months by clicking the checkbox:
- The two filters work together, giving you the options of displaying:
- Open reports less than six months old (the default)
- Open reports of any age
- All reports less than six months old
- All reports of any age
- Any combination of open/closed/fixed reports less than six months old
- Any combination of open/closed/fixed reports of any age
To keep things simpler for app users, the display there is set to only show newer, open reports, so if you want the full range of options, you’ll need to switch to viewing the site on a desktop.
Additionally, reports that have been closed for six months without any update being made will now no longer allow updates. If you need to update an issue that falls into this category, we recommend starting a new report (possibly linking to the old one for reference if it provides useful information for the council).
But you might not see this everywhere
Some councils use FixMyStreet Pro as their own fault-reporting software. These councils can opt whether or not to adopt these defaults, so your experience may be slightly different when visiting FixMyStreet via your local council’s own site.
We think that we’ve arrived at a more intuitive solution than those we tried before — and we hope that these options will suit everyone, whether you’re a user in a hurry coming to make a quick report, or someone who’d like to see a more in-depth history of the area. Give it a go, and then let us know your thoughts.
Jenna Corderoy, Alaveteli Professional Advocate, brings us an update on the project.
Since our last blog post on Alaveteli Professional — our Freedom of Information toolkit for journalists, campaigners and activists — there have been a few exciting developments.
The batch request feature is coming along nicely: this will allow users of the service to send one Freedom of Information request to multiple authorities and help them to easily manage large volumes of responses.
We’re going to be working with a small group of our beta testers to develop this feature and make sure we release it in a useful and responsible form (click here to apply as a beta tester and get a year’s free access to WhatDoTheyKnowPro, the UK version of the service).
We’ve been pleased to see the first news story to emerge as the result of a request made through WhatDoTheyKnowPro: a response to the Foreign and Commonwealth Office showed which are the countries where UK holidaymakers are most likely to get arrested. The full list was covered in the Birmingham Mail.
But we also have plans for this Freedom of Information toolkit to go international: Alaveteli Professional will be a bolt-on option for anyone already running an FOI site on our software platform Alaveteli.
In April, mySociety team members traveled to roll out the first such project, with Info Pro Všechny, the Czech Republic’s Alaveteli site.
We were able to introduce beta users to the features we’ve been developing, such as the ability to keep requests private until the story has been published.
While in the Czech Republic, we held a roundtable discussion with journalists and campaigners, swapping Freedom of Information battle stories and sharing tips and tricks for getting the best results when submitting requests for information, as well as experiences of filing requests to European Union institutions.
mySociety was also invited to give a talk to student journalists based in Olomouc about Info Pro Všechny and Alaveteli Professional in general, discussing success stories generated from our Freedom of Information sites from around the world.
We’re currently working on subscription options, which will allow us to officially launch WhatDoTheyKnowPro as a paid-for service in the UK, and later in the year, we plan to introduce the Pro toolkit to the Belgian Alaveteli site Transparencia.be, which has been making a splash in Belgian politics.
You may know Dr Ben Goldacre from his ‘Bad Science’ and ‘Bad Pharma’ campaigns, which fight misinformation around medicine. Ben has just launched his latest project, the AllTrials Transparency Index — and mySociety helped with the website side of things.
The AllTrials campaign focuses around the fact that a shockingly large proportion of clinical trials do not have their results publicly published.
Not only does this devalue the time, goodwill and even potential risk put in by participants, but there are also issues around bias. Those trials published tend to be the ones which show positive results: if that doesn’t sound like such a terrible situation to you, try playing this game from the Economist magazine, which graphically depicts the problems with skewed coverage. At worst, such selective publication can be dangerous, or lead to poor choices from bodies making medical purchasing decisions.
Transparency and data visualisation are two areas where mySociety has a long history, and so it came to be that we fashioned the deceptively simple AllTrials Transparency Index site, on which anyone can browse the transparency index of the world’s major drug companies, and dive in deeper to the data to see how it was compiled. The source data is free for others to download too, so anyone can integrate it into other projects.
AllTrials are also tracking whether companies register new trials:
That’s the part that anyone can understand — and now, notes for the more technically-inclined who may be wondering how we took the data on each drugs company and presented it in a way that can be quickly and easily taken in.
This is an ongoing campaign with a commitment to future audits, so we wanted to make it easy for the AllTrials team to update the site and republish the source data each time they do.
It’s a static Jekyll site. We wrote a custom plug-in to parse a CSV and produce a page for each company within that CSV, as well as creating some summary data that feeds into the graphs on the front page.
This data is then pulled from the CSV, and D3 is employed to build the graphs and insert them into the generated pages.
The end result is a site that looks good and which can automatically update whenever the underlying data changes. We hope we’ll have played a small part in helping to ensure that it does — and for the better.