Private data, containing personal details of the general public, is accidentally released by public authorities at least once a fortnight, say mySociety.
The volunteer team behind WhatDoTheyKnow, mySociety’s freedom of information website, have dealt with 154 accidental data leaks made by bodies such as councils, government departments and other public authorities since 2009, and these are likely to represent only the tip of the iceberg.
On the basis of this evidence, we are again issuing an urgent call for public authorities everywhere to tighten up their procedures.
How WhatDoTheyKnow works
Under the Freedom of Information act, anyone in the UK may request information from a public body.
WhatDoTheyKnow makes the process of filing an FOI request very easy: users can do so online. The site publishes the requests and their responses, creating a public archive of information.
Public authorities operate under a code of conduct that requires personal information is removed or anonymised before data is released: for example, while a request for the number of people on a council housing waiting list may be calculated from a list including names, addresses and the reason for housing need, the information provided should not include those details.
Accidental data releases become particularly problematic when the data requested concerns the details of potentially vulnerable people.
Hidden data is not always hidden
When users request information through WhatDoTheyKnow, it’s often provided in the form of an Excel spreadsheet. But unfortunately, private data is sometimes included on those spreadsheets, usually because the staff member who provides it doesn’t understand how to anonymise it effectively.
For example, data which is in hidden tabs, or pivot tables, can be revealed by anyone who has basic spreadsheet knowledge, with just a couple of clicks.
By its very nature, data held by our public authorities can be extremely sensitive: imagine, for example, lists of people on a child protection register, lists of people who receive benefits, or as happened back in 2012, a list of all council housing applicants, including each person’s name and sexuality.
Our latest warning is triggered by an incident earlier this month, in which Northamptonshire County Council accidentally published data on over 1,400 children, including their names, addresses, religion and SEN status. Thanks to the exceptionally fast work of both the requester and the WhatDoTheyKnow volunteers, it was removed within just a few hours of publication, and the incident has been reported to the Information Commissioner’s Office. Concerned residents should contact the ICO or the council itself.
Advice for FOI officers
Back in June 2013, we set out the advice that we think every FOI officer should know. That advice still stands:
- Don’t release Excel pivot tables created from spreadsheets containing personal information, as the source data is likely to be still present in the Excel file.
- Ensure those within an organisation who are responsible for anonymising data for release have the technical competence to fulfil their roles.
- Check the file sizes. If a file is a lot bigger than it ought to be, it could be that there are thousands of rows of data still present in it that you don’t want to release.
- Consider preparing information in a plain text format, eg. CSV, so you can review the contents of the file before release.
Part of a larger picture
Not every FOI request is made through WhatDoTheyKnow—many people will send their requests directly to the public authority. Moreover, we can only react to the breaches that we are aware of: there are, in all probability, far more which remain undiscovered.
But because of WhatDoTheyKnow’s policy of making information accessible to all, by publishing it on the site, it’s now possible to see what an endemic problem this kind of treatment of personal data is.
When we come across incidents like these, we act very rapidly to remove the personal information. We then inform the public authority who provided the response. We encourage them to self-report to the Information Commissioner’s Office, and where the data loss is very serious, we may make an additional report ourselves.
Our FOI site WhatDoTheyKnow has a fancy new frontage.
Coming hot on the heels of TheyWorkforYou’s new homepage, the fresh look is part of our rolling process of design improvements. Out goes the rather sober grey and burgundy colour scheme, and in comes a fetching cobalt blue paired with banana yellow.
As you might have guessed, though, there’s more to this than a new palette. Yes, in the fast-changing world of web design, fashions change and dated sites can run the risk of looking irrelevant—but we are also keen to ensure that any new design works for its keep.
Not just a pretty face
It’s important, when we invest time and resources into a redesign, that there are tangible improvements. So, like almost everything we do these days, the changes will be subjected to scrutiny from our Research team.
They’ll be checking that we’ve:
- Improved the site’s usability, making it more obvious how to browse or file FOI requests;
- Encouraged users to take the step of making an FOI request, even if the concept is a new one for them;
- Enabled people to understand what the FOI Act is, and what rights it confers.
That’s a lot to expect from a simple redesign, so let’s take a look at how we hope to achieve it.
Of course, the first thing visitors see is the title text. It may seem pretty simple, but, as anyone who writes will know, the shorter the sentence, the harder it is to get right.
Take it from us, this deceptively simple piece of copy represents quite a bit of anguished brainstorming:
It tries to distill a complex idea into something that absolutely everyone can understand, even if they’ve never heard of FOI before. Meanwhile, the subtitle highlights your legal right to information.
Alaveteli, the software this and many other FOI sites around the world are built on, has always included two figures on its sites’ homepages: the number of requests that have been made through the site, and how many public authorities it has contact details for. The image below displays WhatDoTheyKnow’s stats at the time of writing:
It’s a nice way of showing that the site is both useful and used, but there’s something else, too: when users see that other people have taken an action online, they’re more likely to take the plunge themselves. It’s the same thinking that informed our byline on WriteToThem: “Over 200,000 messages sent last year.”
How it works
The homepage now includes a simple graphic to show the path you can expect to take if you go ahead and file an FOI request on the site:
Breaking the process down into just three steps makes it look manageable, and there’s a link deeper into our help pages for people who want to understand the FOI Act better.
For those who prefer to browse
Some content remains the same. We’ve still included links to the latest successful requests—albeit lower down the page, so as not to distract from the page’s main message, that you can make a request. These show, more graphically than any piece of copy could, that you can get results:
They’re also a great way into the site for people who just want to browse: they are a random assortment of requests that have recently been marked as successful, and can often throw up some surprising and interesting subject matter.
Sharing the benefits
Provided that we discover that the design has been effective in the areas mentioned above, we hope to roll it out as an option on the wider Alaveteli codebase, so it can be implemented by anyone running an Alaveteli site.
Meanwhile, the open source code can be accessed on Github by anyone who would like to use it.
WhatDoTheyKnow is our Freedom of Information website, through which you can send an FOI request to UK publicly-funded bodies. It is used in many different ways, by many different users.
Here’s a recent blog post by Doug Paulley which we think is worth highlighting. It uses a series of FOI requests across every council with social services responsibilities in England, Wales and Scotland, and every health and social care trust in Northern Ireland, to get to the truth of a simple question: whether or not a particular disability organisation, Leonard Cheshire, was honest when stating that they wanted to pay their carers a living wage.
We wanted to draw your attention to it because, as well as being a good read, it really highlights the innovative uses that can be made of the rights we enjoy under the FOI Act. It also shows that you don’t have to be a journalist to dig into a story like this. Perhaps it gives you ideas for something you’d like to investigate?
How many people visit mySociety’s websites?
That’s a question we don’t ask ourselves as much as many other organisations. Much of our current funding is dependent on transactions (that is, the number of people using the site to complete an action such as making an FOI request, writing to a politician, or signing up to receive emails when their MP speaks), and rightly so, since that is a better measure of the sites’ actual effectiveness.
All the same, visitor numbers* do tell us about things like how much public awareness there is of what we do, and which of our sites is more visible than the others, so it’s good to take a proper look now and again.
Which of our UK sites is most visited?
By far our most popular site in terms of visitor numbers is our Freedom of Information site WhatDoTheyKnow. With over 4.5 million visitors 2014-15, it’s had three times more users than its closest competitor, TheyWorkForYou.
As well as allowing users to submit FOI requests, WhatDoTheyKnow also puts the responses into the public domain, so that the information becomes openly available. Every request receives, on average, twenty readers, meaning that transactions do not show the whole picture for this site.
WhatDoTheyKnow’s user numbers are also rising steadily. It’s up 8% on last year, and March 2015 was its highest month for unique users since its launch in 2008, at 470,509.
Which is least visited?
This dubious honour goes to WriteToThem, which nonetheless welcomed 457,209 visitors during the year, either helping them to write to their representatives, or simply showing them who those representatives were.
This was still a decent 11% rise on the previous year, despite a real rollercoaster where some months dipped substantially from the previous year.
Which made the most gains in the last year?
FixMyStreet saw the biggest percentage change, with a 21% rise in visitor numbers compared to the previous year; we talked a bit more about that in a recent blog post. WhatDoTheyKnow had the highest rise in actual visitor numbers: over 360,000 up on 2013-14.
Which fell by the most in the last year?
TheyWorkForYou saw a 12% drop in visitor numbers year on year (and also the biggest drop in real terms)—disappointing, but something we hope to rectify with the new voting pages, an ongoing process of rolling redesign, and some grassroots outreach.
How much effect do external events have on visitor numbers?
We already know that, as you’d expect, when Parliament is on holiday, MPs, debates and legislation aren’t in the news, and TheyWorkForYou visitor numbers fall. There’s also a weekly pattern for all our sites, where far fewer people use them at the weekends, presumably indicating that lots of our users access them from work.
It’s too early to say exactly what effect the election has had on our sites: as I write, people are eagerly checking out the voting records of newly-appointed cabinet ministers on TheyWorkForYou.
One thing we know for sure is that fewer people will have been using WriteToThem, because there have been no MPs to write to for the last few weeks. We’ve removed the “write to your MP” links from TheyWorkForYou, which always drove a good deal of WriteToThem’s traffic.
FixMyStreet enjoyed a boost back in June, when it was featured on the Channel 4 programme ‘The Complainers’—and the nice thing is, user numbers never receded back to their previous levels after the programme was over. Maybe people just need to use FixMyStreet to see how useful it is.
How many people visit mySociety’s UK websites in total?
This is a difficult figure for us to produce with accuracy, because we don’t trace whether you’re the same person visiting a number of our different sites.
However, the aggregate total of visitors to all our UK sites (WriteToThem, TheyWorkForYou, FixMyStreet and WhatDoTheyKnow) for 2014-15 is 6,983,028. Thanks very much if you were one of them
How can I help?
Glad you asked! If you find mySociety sites useful, you can help us spread the word by telling friends, sharing the URLs with any groups you are a member of, posting on Facebook or Twitter, or writing to your local paper.
We have a number of materials for FixMyStreet which can be found here; we hope to create similar materials for our other sites too, and we’ll make sure we announce it on here when we do.
* Note: all references to ‘users’ refer to unique users within the period discussed. So, users in a year means individual people who may have visited any number of times over that year, but are only counted once; same with monthly users.
WhatDoTheyKnow, our website for submitting FOI requests, has listed Network Rail for some time—even though it was not subject to the Freedom of Information Act—for reasons which Richard explained in a 2012 blog post.
As of March 24, however, Network Rail, which had previously been handling requests on a voluntary basis (although perhaps without quite as much adherence as we’d prefer) became fully FOI compliant. So, if there’s anything you’ve been bursting to ask them, now is the time.
Back in January 2012, I wrote a blog post to mark a milestone: WhatDoTheyKnow, our Freedom of Information site, had processed 100,000 requests.
Just three years later, that number now stands at 250,000.
That represents a quarter of a million requests for information that have been processed through the site, and published for anyone to access.
Everything we said in that previous blog post still stands:
WhatDoTheyKnow was set up to give everyone, not just experts, access to information.
By publishing the requests and responses, it strives to create efficiencies for all.
And none of it would have been possible were it not for our wonderful, dedicated team of volunteers, who manage the site admin, help users with their queries, and diligently discuss and process any legal challenges that arise. Thank you, Ganesh, Alex, Alistair, Helen, John, Richard and Ben, and thank you, Francis for your legal advice.
As well as performing a service for the people of the UK, WhatDoTheyKnow also stands as an example of what’s possible. Much of our international activity focuses on helping partners use Alaveteli, our FOI software, to get Right To Know sites up and running in jurisdictions all over the world. It is great to be able to show them that an Alaveteli-based FOI site can thrive.
This is a problem we have been warning about for some time. Islington Council were fined £70,000 for a similar incident in 2012. In light of this fresh incident we again urge all public authorities to take care when preparing data for release.
As with the Islington incident, the information was in parts of an Excel spreadsheet that were not immediately visible. It was automatically published on 14th November when Hackney Council sent it in response to a Freedom of Information request, as part of the normal operation of the WhatDoTheyKnow website. All requests sent via the website make it clear that this will happen.
This particular breach involved a new kind of hidden information we hadn’t seen before – the released spreadsheet had previously been linked to another spreadsheet containing the private information, and the private information had been cached in the “Named Range” data in the released spreadsheet.
Although it was not straightforward to access the information directly using Excel, it was directly visible using other Windows programs such as Notepad. It had also been indexed by Google and some of it was displayed in their search previews.
The breach was first hit upon by one of the data subjects searching for their own name. When they contacted us on 25th November to ask about this, one of our volunteers, Richard, realised what had happened. He immediately hid the information from public view and notified the council.
We did not receive any substantive response from the council and therefore contacted them again on 3rd December. The council had investigated the original report but not understood the problem, and were in fact preparing to send a new copy of the information to the WhatDoTheyKnow site, which would have caused the breach to be repeated.
We reiterated what we had found and advised them to consult with IT experts within their organisation. The next day, 4th December, we sent them a further notification of what had happened, copying the Information Commissioner’s Office (ICO). As far as we are aware, this was the first time the ICO was informed of the breach.
From our point of view it is very disappointing that these incidents are still happening. Freedom of Information requests made via WhatDoTheyKnow are a small fraction of all requests, so it is very likely that this kind of error happens many more times in private responses to requesters, without the public authority ever becoming aware.
Our earlier blog post has several tips for avoiding this problem. These tips include using CSV format to release spreadsheets, and checking that file sizes are consistent with the intended release. Either of these approaches would have averted this particular breach.
We would also urge the ICO to do as much as possible to educate authorities about this issue.
In a recent blog post, we summarised the research we commissioned from the University of Manchester’s Rachel Gibson, Marta Cantijoch and Silvia Galandini, on whether or not our core UK websites have an impact.
The full research paper is now available, and you can download it here .
Professor Rachel Gibson says: “This research presents a unique and valuable insight into the users of online resources such as FixMyStreet and WhatDoTheyKnow.
“Through applying a highly original methodology that combines quantitative and in-depth qualitative data about people’s experience of mySociety sites over time, we provide a picture of how eDemocracy tools are contributing to activism at the local level.
“We thank all those that contributed to this important study and mySociety for their co-operation in developing this highly rewarding and academically rigorous project.”
Our thanks to Rachel, Marta and Silvia for conducting this research, which utilised methods not previously used in the civic tech field. We hope that it will prove a useful foundation to our own further research, and that of others.
Few of our users realise this, but hardly a week goes by without mySociety receiving a legal threat relating to our Freedom of Information website WhatDoTheyKnow.com.
These might refer to perceived libel in a request, or to material released in error, which an authority now wishes to retract. In the normal course of things, our team deal with legal issues quickly and diligently, occasionally consulting our lawyer – and generally speaking, they never need concern our users.
On Friday November 7th, at 2:17pm, we received a ‘letter before action’ from Enfield Council’s legal department, asking us to do two things: first, that we take down a certain request, and secondly that we provide them with information on the person who had raised it.
Well, that’s a quick turnaround even by the standards of our crack team of volunteers, even if it had been clear that Enfield had a good legal case. And, once we looked closely, we weren’t at all sure that they did.
The FOI request which had triggered this message seems like a fairly standard one: it asks for information about the closure of public libraries, and how much those closures would contribute towards the council’s stated target of making £65 million of savings over the next three years.
It is worth mentioning that the name this FOI request was filed under was clearly and demonstrably an impersonation – it claimed to be from the CEO of Enfield Council. In fact, we’d already been in correspondence with the council over this, and, as impersonation is against our site policies, it was a quick and easy decision for us to remove the name.
We will not disclose your email address to anyone unless we are obliged to by law, or you ask us to.
– and indeed, we have only done so once, when compelled to by a court order, in all the site’s long history (currently standing at over 200,000 FOI requests and over 71,000 users). The other point was slightly more tricky. We do our best to run WhatDoTheyKnow in the most responsible manner possible, for our users and for public authorities. We often have to tread a delicate line in order to do so.
Often there is a good reason that public bodies want information taken down, and the team routinely act rapidly to remove personal information, and other material that public bodies accidentally release, from our website. When we do take material down, wherever possible we do so transparently, leaving a note explaining what’s been removed and why.
But, where possible, we do not remove a request from the site, unless there is a very clear reason why its publication is breaking the law. Putting the mischievous name of the requester aside, this appeared to be a standard request about libraries and funding.
On occasions, like this, when requests to take material down appear unfounded or overzealous, we challenge them.
The notice before action stated that ‘the public availability of this information is or is likely to be highly damaging to Enfield Council’s ability to properly carry out those projects’. It also referred to ‘confidential and commercially sensitive’ material having been released, but we can find little within the request that is not publicly available elsewhere – for example, on the council’s own website one can find details of the Library Plan Development consultation document, containing very similar information – and nothing that seems obviously sensitive.
The council have recently been reported as saying:
“No decisions have been made yet on the type of library or the location of libraries. The final decision on the library service, location and different types of libraries will be made in February or March next year following the conclusion of this consultation.”
So – if a decision has not yet been made, the number of libraries to be closed cannot be a leak, as the information does not yet exist.
For those reasons, we responded to Enfield Council ask for clarification. We took down the request in question as a precaution, while we awaited this clarification. We gave them slightly longer than 43 minutes in which to do so — in fact, we contacted them on 10 November asking them to reply by 5pm on 14 November with clarification on their position.
For some reason it took them until 13 November to say they wouldn’t be able to reply substantively by then, so we asked them to respond instead by 5pm today — otherwise we would make the request public again.
No clarification has yet arrived. That being the case, we have made the request live.
What impact do mySociety sites actually have? We could lose a lot of sleep over this important question – or we could do something concrete, like conducting academic research to nail the answers down for once and for all.
As slumber enthusiasts, we went for the research option – and, to help us with this commitment we’ve recently taken on a new Head of Research, Rebecca Rumbul. Watch this space as she probes more deeply into whether our tools are making a difference, both in the UK and abroad.
Even before Rebecca came on board, though, we had set a couple of research projects in motion. One of those was in partnership with the University of Manchester, funded by the ESRC, which sought to understand what impact our core UK sites (FixMyStreet, WriteToThem, TheyWorkForYou and WhatDoTheyKnow) have on their users, and specifically on their level of political engagement.
Gateways to participation
It’s perhaps worth mentioning that, while our sites appear, on the face of it, to be nothing more than a handy set of tools for the general citizen, they were built with another purpose in mind. Simply put, each site aims to show people how easy it is to participate in democracy, to contact the people who make decisions on our behalf, and to make changes at the local and national levels.
Like any other online endeavour, we measure user numbers and transaction completions and time spent on site – all of that stuff. But one of the metrics we pay most attention to is whether users say they are contacting their council, their MP or a public body for the first time. Keeping track of this number ensures that we’re doing something to open democratic avenues up to people that haven’t used them before.
But there are plenty more questions we can ask about the impact we’re having. The University of Manchester study looked into one of them, by attempting to track whether there was a measurable change in people’s political activity and engagement after they’ve used one of our sites. On Monday, researchers Rachel Gibson, Marta Cantijoch and Silvia Galandini presented their findings to an attentive audience at King’s College London.
The project has taken a multi-pronged approach, asking our users to complete questionnaires, participate in online discussions, or keep a 12-week diary about political and community engagement (thanks very much to you, if you were one of the participants in this!). The result was a bunch of both qualitative and quantitative data which we’ll be able to come back to and slice multiple ways in the future – Gibson says that they haven’t as yet managed to analyse all of the free text diaries yet, for example.
In itself this study was interesting, because not much research has previously been conducted into the impact of digital civic tools – and yet, as we know from our own international activities, people (not least ourselves) are launching sites all over the world based on the premise that they work.
Some top-level conclusions
The research will be published in full at a future date, and it’s too complex to cover all of it within the confines of a short blog post, but here are just a few of the takeaway findings:
- A small but quantifiable uplift in ‘civic participation’ was noticed in the period after people had used our sites. This could include anything from working with others in the local community to make improvements, to volunteering for a charity.
- No change was found in the level of political influence or understanding that people judged themselves to have. This was a surprise to the researchers, who had thought that users would feel more empowered and knowledgeable after contacting those in power, or checking up on their parliamentary activity.
- As with our research back in 2011, the ‘average’ user of mySociety sites was found to be white, above middle-aged, and educated to at least degree level. Clearly this is a userbase which we desperately need to expand, and we’ll be looking carefully – with more research and some concentrated outreach efforts – at how we can do that.
- Users tended to identify themselves as people who already had an interest in politics. Again, here is an area in which we can improve. Of course, we’re happy to serve such users, but we also want to be accessible to those who have less of a baseline interest.
- Many users spoke of community action as bringing great satisfaction. In some cases, that was getting together in real life to make improvements, but others saw something as simple as reporting graffiti on FixMyStreet as an action that improved the local area for everyone.
Thanks to the University of Manchester researchers for these insights and for presenting them so engagingly. We’ll update when the full research is available.