It’s something we wanted to build, and something we believed there was a need for: but wanting and believing do not make a sound business case, and that’s why we spent the first few weeks of the project in a ‘discovery’ phase.
Our plan was to find out as much as we could about journalists, our prospective users — and particularly just how they go about using FOI in their work. Ultimately, though, we were seeking to understand whether journalists really would want, or need, the product as we were imagining it.
So we went and talked to people at both ends of the FOI process: on the one hand, journalists who make requests, and on the other, the information officers who respond to them.
Since we’re planning on making Alaveteli Professional available to partners around the world, it also made sense to conduct similar interviews outside the UK. Thanks to links with our Czech partner, running Informace Pro Všechny on Alaveteli, that was a simple matter. A recent event at the Times building in London also allowed us to present and discuss our findings, and listen to a couple of interesting expert presentations: Matt Burgess of Buzzfeed talked about some brilliant use of FOI to expose criminal landlords, and listed FOI officers’ biggest complaints about journalists. Josh Boswell of the Sunday Times was equally insightful as he ran through the ways that he uses FOI when developing stories.
These conversations have all helped.
The life of an investigative journalist is never simple
The insights our interviewees gave us were turned by Mike Thompson (formerly of mySociety, and brought back in for this phase) into a simple process model showing how journalists work when they’re pursuing an investigation using FOI.
After conceiving of a story that requires input from one or more FOI request, every journalist will go through three broad phases: research; request and response; and the final data analysis and writing. The more complicated cases can also involve refused requests and the appeals process.
For a busy working journalist, there are challenges at every step. Each of these adds time and complexity to the process of writing a story, which is an anathema to the normal daily news cycle. FOI-based stories can be slow, and timing unpredictable — editors do not particularly like being told that you’re working on a story, but can’t say when it will be ready, or how much value it will have.
During the research phase diligent journalists will make a time-consuming trawl through resources like authorities’ own disclosure logs and our own site WhatDoTheyKnow (or its equivalents in other countries), to see if the data they need has already been released.
Where a ‘round robin’ request is planned, asking for information from multiple authorities — sometimes hundreds — for information, further research is needed to ensure that only relevant bodies are included. In our two-tired council system, where different levels of authority deal with different responsibilities, and not always according to a consistent pattern, that can be a real challenge.
Wording a request also takes some expertise: get that wrong and the authorities will be coming back for clarification, which adds even more time to the process.
Once the request has been made it’s hard to keep on top of correspondence, especially for a large round robin request. Imagine sending a request to every council in the country, as might well be done for a UK-wide story, and then dealing with each body’s acknowledgements, requests for clarifications and refusals.
When the responses are in journalists often find that interpretation is a challenge. Different authorities might store data or measure metrics differently from one another; and pulling out a meaningful story means having the insight to, for example, adjust figures to account for the fact that different authorities are different sizes and cater for differently-dispersed populations.
Sadly, it’s often at this stage that journalists realise that they’ve asked the wrong question to start with, or wish that they’d included an additional dimension to the data they’ve requested.
What journalists need
As we talked through all these difficulties with journalists, we gained a pretty good understanding of their needs. Some of these had been on our list from the start, and others were a surprise, showing the value of this kind of exploration before you sit down to write a single line of code.
Here’s what our final list of the most desirable features looks like:
An embargo We already knew, anecdotally, that journalists tend not to use WhatDoTheyKnow to make requests, because of its public nature. It was slightly sobering to have this confirmed via first person accounts from journalists who had had their stories ‘stolen’… and those who admitted to having appropriated stories themselves! Every journalist we spoke to agreed that any FOI tool for their profession would need to include a way of keeping requests hidden until after publication of their story.
However, this adds a slight dilemma. Using Alaveteli Professional and going through the embargo-setting process introduces an extra hurdle into the journalist’s process, when our aim is, of course, to make the FOI procedure quicker and smoother. Can we ensure that everything else is so beneficial that this one additional task is worthwhile for the user?
Talking to journalists, we discovered that almost all are keen to share their data once their story has gone live. Not only does it give concrete corroboration of the piece, but it was felt that an active profile on an Alaveteli site, bursting with successful investigations, could add to a journalist’s reputation — a very important consideration in the industry.
Request management tools Any service that could put order into the myriad responses that can quickly descend into chaos would be welcome for journalists who typically have several FOI requests on the go at any one time.
Alaveteli Professional’s dashboard interface would allow for a snapshot view of request statuses. Related requests could be bundled together, and there would be the ability to quickly tag and classify new correspondence.
Round-robin tools Rather than send a notification every time a body responds (often with no more than an acknowledgement), the system could hold back, alerting you only when a request appears to need attention, or send you status updates for the entire project at predefined intervals.
Refusal advice Many journalists abandon a request once it’s been refused, whether from a lack of time or a lack of knowledge about the appeals process. WhatDoTheyKnow Professional would be able to offer in-context advice on refusals, helping journalists take the next step.
Insight tools Can we save journalists’ time in the research phase, by giving an easy representation of what sort of information is already available on Alaveteli sites, and by breaking down what kind of information each authority holds? That could help with terminology, too: if a request refers to data in the same language that is used internally within the council, then their understanding of the request and their response is likely to be quicker and easier.
Onwards to Alpha
We’re currently working on the next part of the build — the alpha phase.
In this, we’re building quick, minimally-functional prototypes that will clearly show how Alaveteli Professional will work, but without investing time into a fully-refined product. After all, what we discover may mean that we change our plans, and it’s better not to have gone too far down the line at that point.
If you are a journalist and you would like to get involved with testing during this stage and the next — beta — then please do get in touch at firstname.lastname@example.org.
We’d love to bring our Alaveteli Professional project to Kenyan journalism.
As of this year, Kenyan citizens are enjoying a new right to know, thanks to their Freedom Of Information Act, pending since 2007 and finally passed this year.
Alaveteli Professional will provide Kenyan journalists with a toolset and training to help them make full use of FOI legislation, so they can raise, manage and interpret requests more easily, in order to generate high-impact public interest stories.
But the project will also bring benefits to all Kenyans. By helping journalists and citizen reporters to make full use of the Act, it will ultimately make it easier for everyone to hold power to account.
How you can help
Now here’s the bit you need to know about: please tweet using the hashtag #innovateAFRICA explaining why you think Alaveteli Professional in Kenya is an important digital solution.
This will demonstrate that you agree that Alaveteli Professional is worthy of innovateAFRICA’s support — every tweet helps to give our application more traction.
Tweets from everyone are welcome, but yours will have extra leverage if you’re a mySociety partner, a Kenyan journalist or activist who would use the project, a funder or a digital innovator yourself.
Please use your 140 characters to help us bring better FOI capabilities to Kenya! And don’t forget that hashtag: #innovateAFRICA.
Image: Innovate Africa
Lucie Stephens has sent Freedom of Information requests to every Local Education Authority in the country — over 200 of them.
She’s requesting information about asbestos in schools, and for a reason that’s very close to her heart. Earlier this year, Lucie’s mother, a teacher, died of mesothelioma, a cancer that is almost exclusively caused by exposure to that substance.
Now Lucie is acting on the promise she made before her mum passed away: “To do my best to make sure no-one else has to suffer like she has.”
We were really interested to hear how Lucie came to use Freedom of Information as tool in her campaign, so we got in touch to ask her a few questions.
After Mum’s diagnosis I wanted to know how safe my daughter’s school was. I wrote to them and got a positive response: her school does contain asbestos and they were willing to share this information with me.
I then wrote to my local council to request the same information for my borough. This took a lot longer, and involved frequently having to chase and remind them of deadlines missed.
I thought all parents and teachers should know if their school contains asbestos. I didn’t want them to have to struggle to access the information from their local council in the way that I did.
A friend suggested that FOI would be a good way of getting hold of consistent information, and at this point it also became clear that it would be a useful way to analyse how well asbestos is being managed in our schools.
Had you heard of the Freedom of Information Act, or used it previously?
I had heard about FOI before but hadn’t used it. It had always seemed a complicated and challenging process, so I was nervous of it. I knew what questions to ask but was still worried about getting the format right.
I had made contact with the National Union of Teachers (NUT), as they are also campaigning to get asbestos removed from schools. I asked them to advise on the best wording to use in order to get the information that I required, and they were really helpful.
Once you have all the data for every school in the country, what will you do with it?
Every parent and teacher should know if their school contains asbestos.
I am planning to use the data collected to produce an easy-to-access national map that parents and teachers can use to find out if their school is affected. By telling parents which schools have asbestos we are enabling them to hold their school, local education authority and ultimately (we hope) the Department for Education to account. The map will also be useful for campaigners and journalists concerned with the issue of asbestos in schools.
What would be the ideal outcome of your campaign?
I want policy on asbestos in schools to change.
Each school should be expected to produce an annual report on the extent of asbestos and how it is being managed. This will increase transparency and accountability until we get to the point where all asbestos in schools has been removed.
At the same time the Department for Education needs to change its approach. It needs to commit to the phased removal from all schools by 2028.
To achieve this they need to earmark funds specifically for asbestos removal and set annual targets for the number of schools that will be asbestos free. The phased removal should start with the most dangerous schools first but seek to remove all asbestos by 2028.
By revealing which schools contain asbestos, the campaign will make it easier for parents to hold the DFE to account over their plans for the removal of asbestos.
Have any of your FOI requests been turned down?
I made my requests to local authorities, asking them to provide the figures for every school in their jurisdiction. Most have not provided information for academies and free schools as they do not have a statutory responsibility for them.
There were various refusals and partial responses, from authorities which just issued generic statements that ‘all local schools built before 2000 contain asbestos’ with a link to their online schools directory, to those claiming that the information sought would take too long to collect, and so refusing to issue it, or requesting payment before the information could be released.*
Some areas withheld data on the number of claims that they had received from teachers, school staff and ex-pupils, stating that the figures were low enough to make it possible to identify the individuals concerned. Some areas refused to release details of the amount of money paid out in claims, stating that this would breach data protection. This rule was not applied consistently, though, as some areas did release figures relating to only one claim and settlement.
I was very irritated to find some authorities who, because they claimed to be unable to answer one of my seven questions, then refused to answer any of them. To date we only have data from 135 local councils of 152 approached. Some are very overdue in providing the data.
In gathering the data it was clear that there is huge variety in the ways in which asbestos is being managed in schools. It was also apparent that there is a wide range of ways in which FOI requests are handled.
What benefits did you get from using WhatDoTheyKnow?
Without WhatDoTheyKnow I wouldn’t have been able to collect the data.
The site was really easy to use. It was very helpful in managing the data as it appeared, and in reminding me when authorities were overdue in providing information.
In some cases I used my right to a review to challenge the handling of a request, and this led to further information being released. I definitely wouldn’t have made a challenge without the site making the process straightforward in the way it does.
It has also been great to have the data held on a public site like this, so that I’ve been able to direct journalists and campaigners to the source of the data directly.
The only challenge with the site was that the number of requests was capped. I can totally appreciate why this was, and in fact when I did contact the team and ask for this cap to be removed so that I could complete the requests in a shorter timeframe, they were very helpful. Having ascertained that I was sending bulk requests for a valid reason, they acted quickly to remove the cap.
You can read more about Lucie’s campaign, and lend your support, on her petition page.
* Note: Making an FOI request is almost always free, but authorities can charge under certain circumstances, and they can refuse to respond to a request outright if it will cost more than a certain amount — currently £450 — to do so.
This week, the 2016 Open Government Partnership summit is being held in Paris, France.
The OGP was launched in 2011 to provide an international platform for domestic reformers committed to making their governments more open, accountable, and responsive to citizens. Its goals are, therefore, pretty much in line with the objectives of civic tech — the field we at mySociety been ploughing in for over a decade. So we’ve been supporters of the OGP since it started, as have many of our partners and collaborators.
Today, there are 70 participating countries. Inevitably, some member countries have governments whose enthusiasm for the OGP goals is, perhaps, greater than their willingness to actually implement the changes that attaining such goals require. But even where that is the case, the OGP is a catalyst for getting accountability and transparency onto their agenda, and we’re all for that.
All of mySociety’s projects fall somewhere within the remit of the OGP. For example, our Freedom of Information platform Alaveteli encourages citizens to make requests, and our work with parliamentary monitoring organisations is all about opening up the activity of parliaments in a way that OGP very much promotes.
This year, mySociety will be represented at OGP by Bec (from our research team), Jen (who manages our international partners) and Dave (who works on the EveryPolitician project). If you’re in Paris for the OGP, please do seek us out. We’re around for the main events, but also:
- We’re hoping EveryPolitician data can help some of the projects in the OGP toolbox hackathon
- Bec will be on the Hands-on Accessing and Re-using Parliamentary Data panel at 4:15pm Thursday (lecture theatre, Conseil Economique)
- then at 5pm in the “pitch corner” Bec is reminding people about about TICTeC (that’s our research conference in Florence next year)
- Jen and Dave are running a session on EveryPolitician on Friday at 12:30 (Room 5, Iéna)
You might already be enjoying one of the usability improvements that FixMyStreet version 2.0 has brought, though it’s possible that you haven’t given it much thought.
But for FixMyStreet, we hadn’t given much thought to letting you filter reports by more than one dimension, until Oxfordshire County Council suggested that it would be a useful feature.
For quite some time, you’d been able to filter by category and status (“Show me all pothole reports” or “Show me all ‘unfixed’ reports”), but this new functionality is more flexible.
You can now select multiple categories and multiple statuses simultaneously (“show me all pothole and graffiti reports that are unfixed or in progress”) — and all through the power of tickboxes.
If you’re a non-technical person, that’s all you need to know: just enjoy the additional flexibility next time you visit FixMyStreet. But if you are a coder, you might like to read more about how we achieved this feature: for you, Matthew has written about it over on the FixMyStreet Platform blog.
Today, and for the next three days, a landmark appeal takes place. The UK’s most senior judges are in the Supreme Court, deciding whether Parliament or the government has the authority to trigger Brexit via Article 50.
If you’d like to read along (or search for a specific term), you can now do so on this site, quickly put together by mySociety senior developer Matthew, today. For apparently this is what he likes to do in his free time.
Like most Supreme Court hearings, this one is being broadcast live, and daily digital transcripts are available. It was this last fact, perhaps, that prompted a friendly tweet:
— Chris Adams (@mrchrisadams) December 5, 2016
That was enough to send Matthew scurrying to see whether the scrapers we’d put in place for the Leveson Inquiry would also work in this instance. In large part they did, although there is still a small bit of polishing up of the final product to do.
Meanwhile, sit back and enjoy the show. That is, as the Guardian puts it, if you have an appetite for “intricate legal argument, arcane vocabulary and historical precedents”. Well, come on, who doesn’t?
If you’ve used FixMyStreet recently — either to make a report, or as a member of a council who receives the reports — you might have noticed that the site’s automated emails are looking a lot more swish.
Where previously those emails were plain text, we’ve now upgraded to HTML, with all the design possibilities that this implies.
It’s all part of the improvements ushered in by FixMyStreet Version 2.0, which we listed, in full, in our recent blog post. If you’d like a little more technical detail about some of the thought and solutions that went into this switch to HTML, Matthew has obliged with in a blog post over on FixMyStreet.org.
I’m just a few weeks into my position of Research Associate at mySociety and one of the things I’m really enjoying is the really, really interesting datasets I get to play with.
Take FixMyStreet, the site that allows you to report street issues anywhere in the UK. Councils themselves will only hold data for the issues reported within their own boundaries, but FixMyStreet covers all local authorities, so we’ve ended up with probably the most comprehensive database in the country. We have 20,000 reports about dog poop alone.
Now if you’re me, what to do with all that data? Obviously, you’d want to do something with the dog poop data. But you’d try something a bit more worthy first: that way people won’t ask too many questions about your fascination there. Misdirection.
How does it compare?
So, starting with worthy uses for that massive pile of data, I’ve tried to see how the number of reports in an area compares against other statistics we know about the UK. Grouping reports into ONS-defined areas of around 1,500 people, we can match the number of reports within an area each year against other datasets.
To start with I’m just looking at English data (Scotland, Wales and Northern Ireland have slightly different sets of official statistics that can’t be combined) for the years 2011-2015. I used population density information, how many companies registered in the area, if there’s a railway station, OFCOM stats on broadband and mobile-internet speeds, and components from the indices of multiple deprivation (various measures of how ‘deprived’ an area is, such as poor health, poor education prospects, poor air quality, etc) to try and build a model that predicts how many reports an area will get.
The good news: statistically we can definitely say that some of those things have an effect! Some measures of deprivation make reports go up, others make it go down. Broadband and mobile access makes them go up! Population density and health deprivation makes them go down.
The bad news: my model only explains 10% of the actual reports we received, and most of this isn’t explained by the social factors above but aspects of the platform itself. Just telling the model that the platform has got more successful over time, which councils use FixMyStreet for Councils for their official reporting platform (and so gather more reports) and where our most active users are (who submit a disproportionate amount of the total reports) accounts for 7-8% of what the model explains.
What that means is that most reasons people are and aren’t making reports is unexplained by those factors. So for the moment this model is useful for building a theory, but is far from a comprehensive account of why people report problems.
Here’s my rough model for understanding what drives areas to submit a significantly higher number of reports to FixMyStreet:
- An area must have a problem
Measures of deprivation like the ‘wider barriers to housing deprivation’ metric (this includes indicators on overcrowding and homelessness) as well as crime are associated with an increase in the number of reports. The more problems there are, the more likely a report is — so deprivation indicators we’d imagine would go alongside other problems are a good proxy for this.
- A citizen must be willing or able to report the problem
Areas with worse levels of health deprivation and adult skills deprivation are correlated with lower levels of reports. These indicators might suggest citizens less able to engage with official structures, hence fewer reports in these areas.
People also need to be aware of a problem. The number of companies in an area, or the presence of a railway station both increase the number of reports. I use these as a proxy for foot-traffic – where more people might encounter a problem and report it.
Population density is correlated with decreased reports which might suggest a “someone else’s problem” effect – a slightly decreased willingness to report in built-up areas where you think someone else might well make a report.
- A citizen must be able to use the website
As an online platform, FixMyStreet requires people to have access to the website before they can make a report. The less friction in this experience makes it more likely a report will be made.
This is consistent with the fact that an increased number of slow and fast home broadband connections (and fast more than slow ones) increases reports. This is also consistent with the fact that increased 3G signal in premises is correlated with increased requests.
Reporting problems on mobile will sometimes be easier than turning on the computer, and we’d expect areas where people more habitually use mobiles for internet access to have a higher number of reports than broadband access alone would suggest. If it’s slightly easier, we’d expect slightly more – which is what this weak correlation suggests.
Not all variables my model includes are significant or fit neatly into this model. These are likely working as proxy indicators for currently unaccounted for, but related factors.
I struggle, for instance, to come up with a good theory why measures of education deprivation for young people are associated with an increase in reports. I looked to see if there was a connection between an area having a school and having more reports on the basis of foot-traffic and parents feeling protective over an area – but I didn’t find an effect for schools like I did for registered companies.
So at the moment, these results are a mix of “a-hah, that makes sense” and “hmm, that doesn’t”. But given that we started with a dataset of people reporting dog poop, that’s not a terrible ratio at this point. Expanding the analysis into Scotland and Wales, analysing larger areas, or focusing on specific categories of reports might produce models that explain a bit more about what’s going on when people report what’s going wrong.
I’ll let you know how that goes.
You might have seen it in the Daily Mirror: the full extent of the Department of Work and Pensions’ legal costs, incurred while fighting the obligation to name the companies who participated in the Workfare scheme.
Workfare is a government program which required the unemployed to work for one of the participating organisations, in exchange for no pay other than their existing benefits — working out lower than the minimum wage.
It’s a story in which our site WhatDoTheyKnow is strongly involved. The original request for the list of companies participating in the Workfare scheme was made on the site back in January 2012 by user Frank Zola.
That request was refused, noting that the information was “being withheld under Section 43 of the FOI Act which relates to the commercial interests of both the Department and those delivering services on our behalf”.
As any WhatDoTheyKnow user is given the means to do, Zola referred the request to the Information Commissioner. They ruled in favour of the release.
The government were unforthcoming, however, and the matter was taken to tribunal and through the court of appeal. Zola continued to pursue the case doggedly as the government repeatedly questioned the ruling that the information must be released into the public domain. Their defence was that the companies and charities listed as participating in the Workfare scheme might suffer negative effects to their reputation and commercial viability, given the strong swell of public opinion against the scheme.
In July 2016, four and a half years after the request had first been made, the full list was finally disclosed, and can be seen on WhatDoTheyKnow here.
But the story doesn’t end there. More than one person, including the Mirror’s own reporters, wondered just how much had been spent by defendants on both sides of the legal tussle. In August another user lodged this request with the DWP and discovered that their costs amounted to £92,250.
Meanwhile, a similar request to the ICO reveals that their costs in defending the case used a further £7,931 from the public purse.
We highlight this story partly because it shows the value of persistence. WhatDoTheyKnow is designed to help users to understand their rights. If your request is refused, it makes it clear that you have the right to request an internal review, making that route less intimidating to those who don’t know the ropes. If you go on to the appeals process, we hope that having all previous correspondence online helps with that. Other users can also offer help and support via the annotations system.
In this case though, we think many would have been deterred once the matter had been referred to the higher courts, and we congratulate everyone concerned for sticking to their guns and getting this information out into the public domain.
In a further twist, it’s perhaps worth relating that a few weeks ago, the supermarket Sainsbury’s contacted the WhatDoTheyKnow admin team and asked us to remove their name from the list of organisations who took part in Workfare, since “a small number of our stores did participate in the government’s Work Experience programme but this was not company policy”. We decided not to comply with this request.
This year, Bristol Council did something unusual and admirable. As far as we’re aware, they’re the first UK council to have taken such a step.
Working with mySociety on custom Open311 ‘middleware’ while adopting FixMyStreet as their fault-reporting system, they now enjoy full flexibility, no matter what the future holds.
Thanks to this open approach, Bristol will extract more value from their existing systems and lower operating costs. With integrated, open solutions, and the raised quality of report formatting that Open311 brings, everyone will benefit.
Councils are increasingly understanding the value of flexibility when it comes to service providers.
Contracts that lock them into a single provider for many years mean that, often, there’s no opportunity to benefit when technology advances, and disproportionate costs can be charged for implementing the slightest changes.
This desire for flexibility was a strong factor in Bristol City Council’s decision to adopt FixMyStreet for Councils — and that opened the door for a conversation about Open311.
We’ve always advocated integration via Open311, to the extent that we offer free hook-up with FixMyStreet to any councils who support it.
Because Open311 is an open standard, it supports the entire landscape of providers like FixMyStreet. Right now, Bristol can accept street fault reports not just from us, but from a full range of services — in other words, any site or app that cares to connect with them can do so. No-one knows what the future will hold: if a game-changing system emerges in the future, it makes sense that you’d be able to accept its reports.
All well and good: but when Bristol City Council implemented FixMyStreet as their fault-reporting system, the concept was taken a little bit further. With our collaboration, Bristol created their own Open311 ‘middleware’, sitting between the two systems and talking to both.
Via this method, their existing CMS, Confirm, can hook up to reports coming through from FixMyStreet. That all works smoothly — but, just as importantly, if Bristol ever decide to replace their CRM provider, they’ll be able to do so with no knock-on effect to FixMyStreet reports. And if they ever decide to replace FixMyStreet with a different provider, or indeed to accept reports from a range of providers, they can do that too.
Bristol found us via the GCloud procurement system, and are the first metropolitan unitary authority to install FixMyStreet.
Bristol launched its FixMyStreet service to the public in the summer of 2016.
This autumn, they added asset-based reporting, meaning that known council properties such as streetlights, grit bins and gullies are all marked on FixMyStreet’s maps. Residents can pinpoint and report the location of faults with these assets far more accurately as a result.
There’ll be a phased rollout across departments, starting with Highways and moving across departments as Bristol extend their own middleware. We’ll be watching with great interest.