This is a problem we have been warning about for some time. Islington Council were fined £70,000 for a similar incident in 2012. In light of this fresh incident we again urge all public authorities to take care when preparing data for release.
As with the Islington incident, the information was in parts of an Excel spreadsheet that were not immediately visible. It was automatically published on 14th November when Hackney Council sent it in response to a Freedom of Information request, as part of the normal operation of the WhatDoTheyKnow website. All requests sent via the website make it clear that this will happen.
This particular breach involved a new kind of hidden information we hadn’t seen before – the released spreadsheet had previously been linked to another spreadsheet containing the private information, and the private information had been cached in the “Named Range” data in the released spreadsheet.
Although it was not straightforward to access the information directly using Excel, it was directly visible using other Windows programs such as Notepad. It had also been indexed by Google and some of it was displayed in their search previews.
The breach was first hit upon by one of the data subjects searching for their own name. When they contacted us on 25th November to ask about this, one of our volunteers, Richard, realised what had happened. He immediately hid the information from public view and notified the council.
We did not receive any substantive response from the council and therefore contacted them again on 3rd December. The council had investigated the original report but not understood the problem, and were in fact preparing to send a new copy of the information to the WhatDoTheyKnow site, which would have caused the breach to be repeated.
We reiterated what we had found and advised them to consult with IT experts within their organisation. The next day, 4th December, we sent them a further notification of what had happened, copying the Information Commissioner’s Office (ICO). As far as we are aware, this was the first time the ICO was informed of the breach.
From our point of view it is very disappointing that these incidents are still happening. Freedom of Information requests made via WhatDoTheyKnow are a small fraction of all requests, so it is very likely that this kind of error happens many more times in private responses to requesters, without the public authority ever becoming aware.
Our earlier blog post has several tips for avoiding this problem. These tips include using CSV format to release spreadsheets, and checking that file sizes are consistent with the intended release. Either of these approaches would have averted this particular breach.
We would also urge the ICO to do as much as possible to educate authorities about this issue.
Today, we’re sharing research conducted on the impact of online Freedom of Information technology, including our own platform Alaveteli.
Researchers Savita Bailur and Tom Longley spent three months gathering first-person experiences, analysing data and assessing existing literature to answer this question:
“In what circumstances, if any, can the Freedom of Information tools mySociety builds be shown to have measurable impacts on the ability of citizens to exert power over underperforming institutions?”
You can read their findings here:
1. Literature review [PDF]
The research was conducted in three parts: first, Tom and Savita reviewed existing literature on the impact of FOI, particularly FOI online, to form a baseline of existing knowledge in the area.
They went on to interview people who run, or ran, FOI sites in 27 different countries. They used the resulting transcripts for qualitative research, pulling out common themes to help them draw conclusions.
Finally, they were able to use these insights to create a list of critical success factors for those implementing FOI (especially Alaveteli) websites.
Why did we conduct this research, and why now?
Alaveteli has had a period of intense growth over the last three years – but it would be irresponsible of us to continue its promotion without assessing its true worth and impact.
This is best learned from the people who are at the coalface – the implementers (as Tom and Savita mention in the final research, a fuller study would have allowed them to include government workers and the sites’ end users, too, but that’s perhaps something for the future).
Alaveteli was created with the best intentions – to allow anyone, anywhere to put questions to the people and institutions in power – but it is important to assess whether those intentions have been realised.
We need to ensure that we have spent our efforts and our funders’ money responsibly, and that we are not wasting resources by making poor decisions.
mySociety’s Head of Research, Dr Rebecca Rumbul, says, “This report confirms that the basic model does work, with the UK site WhatDoTheyKnow.com operating as a well-used civic resource with thousands of users per month.
“Whilst the research shows that our partners implementing Alaveteli in their own countries are demonstrably up to the technical challenge of running these sites, it identifies the importance of governmental relations and receiving the right support in the early stages of implementation.
“We now hope to build on this research to better understand how to maximise the use and effectiveness of our platforms around the world in empowering citizens to engage with governments and decision-makers.”
The research was made possible by a grant from the Open Society Foundations.
As the literature review confirms, this type of study has never been done before – and with practitioners speaking to the researchers from within many different cultural backgrounds and political regimes (they interviewed implementers of 20 Alaveteli instances, from Australia to Uruguay), we are in a unique position to take a global view on the subject. For a fully-rounded picture, the study also spoke to implementers of seven sites running non-Alaveteli FOI software.
Of the experience, Savita and Tom say “We were so impressed by the dedication and determination of all the implementers in wanting to raise awareness of FOI and seeing Alaveteli as the platform to do this (even taking into account constructive criticism). The research experience was also great.”
The end result? Take a look for yourself – if you have the slightest interest in online democratic technologies or government-to-citizen information sharing on an international basis, it’s compelling reading.
What we’ll take away
There are learnings for us here, although it was great to hear such consistent praise for the Alaveteli platform and the community that has been created around it.
mySociety’s Director Tom Steinberg said, “We will certainly be looking carefully at the recommendations that have come from this report.
“This will include decisions about how to share best practice across the Alaveteli community, and not just in the technical areas.
“We’ll also be looking hard at the issue of how to ensure consistency in the analytics that are collected by different sites. And we very much hope to return to the subject in a couple of years’ time, when today’s new sites have become established, in order to conduct a follow-up piece of research.”
Few of our users realise this, but hardly a week goes by without mySociety receiving a legal threat relating to our Freedom of Information website WhatDoTheyKnow.com.
These might refer to perceived libel in a request, or to material released in error, which an authority now wishes to retract. In the normal course of things, our team deal with legal issues quickly and diligently, occasionally consulting our lawyer – and generally speaking, they never need concern our users.
On Friday November 7th, at 2:17pm, we received a ‘letter before action’ from Enfield Council’s legal department, asking us to do two things: first, that we take down a certain request, and secondly that we provide them with information on the person who had raised it.
Well, that’s a quick turnaround even by the standards of our crack team of volunteers, even if it had been clear that Enfield had a good legal case. And, once we looked closely, we weren’t at all sure that they did.
The FOI request which had triggered this message seems like a fairly standard one: it asks for information about the closure of public libraries, and how much those closures would contribute towards the council’s stated target of making £65 million of savings over the next three years.
It is worth mentioning that the name this FOI request was filed under was clearly and demonstrably an impersonation – it claimed to be from the CEO of Enfield Council. In fact, we’d already been in correspondence with the council over this, and, as impersonation is against our site policies, it was a quick and easy decision for us to remove the name.
We will not disclose your email address to anyone unless we are obliged to by law, or you ask us to.
– and indeed, we have only done so once, when compelled to by a court order, in all the site’s long history (currently standing at over 200,000 FOI requests and over 71,000 users). The other point was slightly more tricky. We do our best to run WhatDoTheyKnow in the most responsible manner possible, for our users and for public authorities. We often have to tread a delicate line in order to do so.
Often there is a good reason that public bodies want information taken down, and the team routinely act rapidly to remove personal information, and other material that public bodies accidentally release, from our website. When we do take material down, wherever possible we do so transparently, leaving a note explaining what’s been removed and why.
But, where possible, we do not remove a request from the site, unless there is a very clear reason why its publication is breaking the law. Putting the mischievous name of the requester aside, this appeared to be a standard request about libraries and funding.
On occasions, like this, when requests to take material down appear unfounded or overzealous, we challenge them.
The notice before action stated that ‘the public availability of this information is or is likely to be highly damaging to Enfield Council’s ability to properly carry out those projects’. It also referred to ‘confidential and commercially sensitive’ material having been released, but we can find little within the request that is not publicly available elsewhere – for example, on the council’s own website one can find details of the Library Plan Development consultation document, containing very similar information – and nothing that seems obviously sensitive.
The council have recently been reported as saying:
“No decisions have been made yet on the type of library or the location of libraries. The final decision on the library service, location and different types of libraries will be made in February or March next year following the conclusion of this consultation.”
So – if a decision has not yet been made, the number of libraries to be closed cannot be a leak, as the information does not yet exist.
For those reasons, we responded to Enfield Council ask for clarification. We took down the request in question as a precaution, while we awaited this clarification. We gave them slightly longer than 43 minutes in which to do so — in fact, we contacted them on 10 November asking them to reply by 5pm on 14 November with clarification on their position.
For some reason it took them until 13 November to say they wouldn’t be able to reply substantively by then, so we asked them to respond instead by 5pm today — otherwise we would make the request public again.
No clarification has yet arrived. That being the case, we have made the request live.
It’s surprising how many people know about our websites, but haven’t heard about one of their useful features: email alerts.
In previous blog posts, we’ve described how you can set up alerts so that you receive an email every time:
- Your chosen topic is mentioned in Parliament;
- A specific Peer or MP speaks in Parliament;
- Someone makes a FixMyStreet report within your chosen area.
What do most people subscribe to? Mentions of their own town or city; speeches by their own MP; and FixMyStreet reports within their own area. It makes sense – of course we are interested in the issues which affect our own community.
Now here’s another way to be the first to know about what’s going on in your local area: you can subscribe to alerts from WhatDoTheyKnow.com, and receive an email every time someone makes a request for information to your local council (or any public authority of your choice).
Alerts about Freedom of Information requests
WhatDoTheyKnow is our Freedom of Information site. It allows people to ask for information from public bodies such as councils, state schools, the NHS, et cetera – and it publishes both the requests and the responses.
If you ‘follow’ your own local authority, the site will automatically send you an email whenever anyone makes a request to it (condensed into a daily digest).
Because people use the Freedom of Information act to find out about things that really matter to them, these alerts can be a great way of keeping up with local concerns. If you’re a journalist, a councillor, a local activist or just an interested member of your own community, they can be both fascinating and invaluable.
If you’d like to ‘follow’ requests made to your own local council, here’s how:
1. Go to WhatDoTheyKnow.com
The homepage is at www.WhatDoTheyKnow.com.
2. Search for the authority you want to follow
As you can see from the homepage screenshot above, WhatDoTheyKnow currently covers more than 15,500 authorities – everything from local councils to Government departments, state schools and more. The easiest way to find the authority you need is to use the search box on the right of the page:
In this case, there is only one result for my search term, ‘Brighton Council’.
Below this result, I can also see previous FOI requests made to my council. Here’s where I get a taste of why this alert subscription might be so useful and interesting to me, as a local resident. There are requests about bus subsidies, allotment waiting lists, council salaries, school catchment areas… and lots more.
If you prefer, you can refine your search results by selecting “requests”, “users”, or “authorities” below the search box – in this exercise, we are looking for your local council, so you should click “authorities”.
On the right hand side of the page, you will see the title: “Follow this authority”, then the number of people who are already doing so, and a green ‘Follow’ button.
This button allows you to sign up for email alerts.
Below it, as you can see, there is also an option to access an RSS feed – this is useful if you use a “Reader” or “News Aggregator” to keep up with blogs and other feeds from a variety of sources.
But today, we’re signing up for an email alert, so click the green button.
4. Sign in or sign up
At this point, we ask you to sign up or sign in.
If you already have a WhatDoTheyKnow account, all you need to do is log in, and you’re done – your alert has been set up.
If you don’t have an account, it’s as simple as filling in your email address, name, and picking a password.
The site will then send you a confirmation email with a link in it – clicking on this helps to confirm that you are a real person, and that you have entered a genuine email address – which you’ll need, if you are going to receive alerts!
You are now registered, and you’ll receive an email once on every day that anyone makes a request to your local council, or an existing request is updated (eg the council responds, or someone leaves an annotation).
Every email alert has a link at its foot, which you can follow to ‘manage’ your alerts: if you want to stop receiving one or more of them, just click ‘unsubscribe’.
Of course, you don’t have to limit yourself to your own council. If you have a particular interest in any authority – perhaps your children’s school, a government department, or local public bodies- you can sign up to alerts in exactly the same way.
No matter how many alerts you subscribe to, they all arrive in just one email, so they won’t clog up your inbox.
In a forthcoming blog post, we’ll also be looking at how to subscribe to topics or keywords, and how to use operators to get a slightly more refined alert.
Or perhaps you’ve read a news story and wondered about the facts behind it: for example, just how many passport applications are made every month?
Maybe you have an interest in a piece of land near your house, or you’re trying to uncover facts about something that happened many years ago.
These are just a few examples of when the Freedom of Information Act could come in handy. Here in the UK, anyone has the right to ask for information from any public body.
You can ask central and local government departments, the NHS, the armed forces, state-funded schools and the regulators of bodies such as charities, businesses and other organisations for information – and if they hold it, in most cases, they must respond.
Not everyone is aware that they have this right, and if they do, they might not know where to begin. That’s why, back in 2008, we launched WhatDoTheyKnow, a site to promote the Freedom of Information process and make it as easy as possible to send requests.
Like many of our sites, it allows you to make contact with a public body, and also publishes your correspondence online so that others can benefit from your findings.
What can you ask?
Note the word ‘information’ in ‘Freedom of Information’ – this act strictly covers your right to request facts and figures, data and, well… recorded information.
It’s not for asking for data about yourself, and it’s not the place for woolly, indirect queries or requests for opinions. Plus, there’s no point in asking for stuff that the organisation doesn’t hold, or which is already publicly available on their website.
There are lots more details about this, and links to good sources of advice, on WhatDoTheyKnow’s Help pages.
How do you make a request?
We built WhatDoTheyKnow to make the Freedom of Information process really simple for everyone.
If you’d like to see exactly how to do it, we’ve put together a walk-through, below. Follow these steps, and you can make an FOI request too.
1. Go to WhatDoTheyKnow.com
WhatDoTheyKnow is our Freedom of Information request website: you’ll find it here.
2. Make sure no-one has already requested the information you want.
Before you make your request, we advise you to search (via Google or on WhatDoTheyKnow) to ensure that the information hasn’t already been published. If you’ve already done that, go straight to step 3.
You can use the search box on the right of the WhatDoTheyKnow homepage to check whether the information is already on our website.
Search for the name of the authority you want to make your request to.
I’m searching for my hometown of Brighton’s council, because I want to ask a question about a new development. Clicking through to the council’s page, I can see that 1,789 requests have already been made to them through WhatDoTheyKnow.com.
Do any of those requests cover the question I’m planning on asking? I can search them to make sure.
My request concerns Circus Street, so I search for that. It’s a fairly recent issue, so I’m going to restrict my results to the last couple of years – I’m not interested in queries about the same street from many years ago.
I can further refine my results so that I only see requests which have been successful – that is, where the council have provided the information asked for.
That narrows the results down quite nicely, and it’s easy for me to see that my question has not been answered before. It’s also worth clicking the ‘unresolved requests’ link to check that there isn’t a request awaiting a response.
3. Compose your request
Once you’re sure that no-one has made your request before, you can start your own. Click on the green button at the top of the page:
Type in a one-line summary of your question as the title. As you do so, the site will suggest similar requests which have already been made – another means of ensuring that you are not making a duplicate request.
None of these are relevant, so I’m going to go ahead and compose my request.
There are some handy tips on the right: keeping your message succinct and focused will get the best results.
It’s not a good idea to include opinions or complaints: the authority is only obliged to respond to requests for information.
Be sure to include your name below the “Yours faithfully” sign-off.
Take a moment to read the note at the foot:
Requests made on WhatDoTheyKnow are published on the site, as well as sent to the public authority – it’s all about making information available to everyone.
Remember how we searched to see if anyone had already asked this question? If they had, it would have saved me, and the council, some time – that’s why the site does everything in public.
One result of this is that, if you use the site, your name will be published too (if you prefer for that not to happen, you could make your request using a pseudonym, but if you’re thinking of that do read our advice on the subject first).
4. Check your request
Click on ‘preview your public request’ to check it over before you send it.
If you spot a mistake, or remember something you wanted to add, you can click ‘edit this request’.
Otherwise, if you’re ready, click ‘send request’.
5. Register or log in
If it’s the first time you have used the site, you will need to register.
Input your email address, name and a password on the right hand side of the page. Then check your email for a confirmation link before you can proceed. If you can’t find your email confirmation, check your spam folder.
You’ll only have to go through this process once. If you have an account, and are logged in to the site, your request is sent as soon as you click the ‘send’ button.
6. Await your response
Your request has been sent.
When the authority replies, WhatDoTheyKnow will send you an automated email. The reply is also published on the website; the email will contain the link to it.
You’ll have a chance to comment on the response, and send a follow-up request if needs be.
Notice the date on the screenshot above: the authority has 20 working days in which to respond. If you have not received a reply by this time, the site will automatically email you with information about the next steps you should take.
That’s it! If there’s something you need to know, we hope you’ll go ahead and give WhatDoTheyKnow.com a try.
A final thought
Many councils proactively publish information such as comments on planning – which means that a request like the one I make above would not be necessary. It’s always worth checking their website first, to save both your time and the council’s.
If your council doesn’t publish this kind of information as a matter of course, then the process of making a request can do two things.
First, it puts the council’s response in public, so that everyone can see it – and second, it goes towards showing that there is a demand for this kind of information. Your request just might prompt your council to start publishing more information.
This week, we received a request to add a new authority to WhatDoTheyKnow, our Freedom of Information site. Where appropriate, we are happy to do that, although it’s not always possible.
And this time, it really was not possible: the nomination was for the Klingon High Council.
On the face of it, one might think the High Council an ideal candidate for Freedom of Information requests – wouldn’t we all like to hold it to account for its long history of war, assassinations, and political intrigue – not to mention its miserable record on equality for female Klingons?
Sadly, the WhatDoTheyKnow team had to point out to our user that Do’Ha’ ‘oHbe’ yejquv subject to tuqjIjQa’ chut – which, as any native Klingon speaker will tell you, translates as: “Unfortunately, the Council is not subject to the laws of the United Kingdom”. That’s a pre-requisite for any authority that we include on the site.
On the other hand, of course, any Klingon who is interested in setting up an FOI site for their empire may wish to have a look at our Alaveteli platform.
Requesting new public authorities on WhatDoTheyKnow
You can use WhatDoTheyKnow to contact any public authority that we have details for.
If you want to make an FOI request to an authority that is not on the site, you can request it. We do our best to include any authority that is bound by the Freedom of Information Act (and some that are not) – but they are all UK bodies, subject to UK law.
Ideally, they should also be non-fictional, and located in the present rather than the far-distant future.
On our second monthly Alaveteli hangout, Henare Degan from the OpenAustralia Foundation led the discussion. The Australian team running Right To Know had some great experience to share from the Detention Logs group’s use of their site, and we can only thank them for volunteering to chair our chat.
This month we had groups running sites in Hungary, Ukraine, Czech Republic, Australia and the UK on the call. This allowed us to gather interesting perspectives on different site promotion techniques – from sites that have recently started and sites that have been running for a while.
Did you know, for example, that internet TV was so popular in Ukraine during the Winter Uprising that the promotional video for the Ukrainian Alaveteli instance was watched by over 20,000 people?
That’s the kind of thing we love learning from people running FOI sites in different countries. We’re lucky to be a part of a community where people are willing to talk candidly about their experiences. Especially when some might be as tricky as the ones Hungary are facing, with a series of increasingly obstructive laws being put into place.
We saw that Australia and Hungary have something in common: a number of requests come from journalists or activists, as opposed to in Ukraine where people make requests through the site because they’re simply glad to have a place to ask the authorities for help.
The Czech team told us that they’re pleased with the level of government responsiveness to questions through their site, which has recently topped the 2,500 request mark.
We heard about different methods groups are using to promote their sites; from promoting specific requests on Facebook in the Czech Republic, to linking up with journalists and FOI activists to seed the Right To Know site with requests before launch, to the Ukraine media coverage, both using internet TV and short film documentaries.
And a fun idea from a Czech NGO that the Informace pro všechny team are working with: an Open/Not Open competition with Awards for the most open government departments or public bodies. As they mentioned, there aren’t many awards in the public sector and recognition of public bodies’ efforts may encourage further public bodies to open up.
Finally we spoke about sharing the outcomes of some research work that mySociety is doing around Alaveteli and FOI. We also decided that sharing statistics on rates of requests and other such quantitative data might help the community, so we’ll look at the data mySociety is collecting and try and figure out the best way to share that.
The hackpad is here if you want to see our notes from the meeting!
We’ll have our next call in September, and will hopefully be putting information out on the two mailing lists about how to attend and what topics we might cover.
Alaveteli is our platform for anyone who wants to run their own Freedom of Information website. It’s what underpins our own site, WhatDoTheyKnow, and it’s open source software available for anyone, in any part of the world, to use.
Over the past few weeks, we’ve been making improvements to Alaveteli – the aim is to make it as easy as possible to install and use.
You’ll now find that – as well as a snazzy new look for the Alaveteli site – there’s much better documentation and installation instructions. We’ve also consolidated documentation into one place, and we’re working through it all, making sure that everything in there is completely up to date.
In short, if you’ve been thinking about running your own Right To Know site, there’s no better time to get started! The Alaveteli community are standing by to help you – and the documentation’s never been clearer.
Much of what we do here at mySociety relies on Open Data, so naturally we support Open Data Day. In case you haven’t come across this event before, here’s the low-down:
Open Data Day is a gathering of citizens in cities around the world to write applications, liberate data, create visualizations and publish analyses using open public data to show support for and encourage the adoption open data policies by the world’s local, regional and national governments.
If you’re planning on being a part of Open Data Day, you may find some of mySociety’s feeds, tools and APIs useful. This post attempts to put them all in one place. (more…)
You may be familiar with WhatDoTheyKnow, our website which simplifies the process of making a freedom of information request.
mySociety also provides the underlying software as a service for councils: it sits on the council website, templated and branded to fit their site’s style. When someone submits a request, it goes directly into the council’s own back-end processes.
Just like WhatDoTheyKnow, the system publishes all requests, and their answers, online. This helps the council show a commitment to transparency – it also has the effect of cutting down on duplicate requests, since users can browse previous responses.
Brighton and Hove Council are the first council to implement the software.
Now, ordinarily, when we sign off a new project for a client, we write up a case study for our blog. But this time, we were delighted to read an interview by Matt Burgess on FOI Directory, which has done all the hard work for us. With Matt’s permission, we are reproducing the piece in full.
The number of Freedom of Information requests public authorities receive is generally rising and central government dealt with more requests in 2012 than in any year since the Act was introduced. One council has decided to try and open up access to their requests using custom software from mySociety.
Brighton and Hove City Council have implemented a custom version of the popular WhatDoTheyKnow website where more than 190,000 requests have been made.
The council hope it will allow others to easily browse requests that have been made and make them more accountable.
We spoke to council leader Jason Kitcat about why the council decided to implement the new system – which was soft-launched at the beginning of November.
Why did you decide to implement the new system?
JK: I personally, and we collectively as a Green administration, believe passionately in openness and transparency. That’s the primary motivation. So digital tools to support making it easier for citizens to access council information I think are strongly in the interest of our city and local democracy.
We also were seeing an increase in the number of FOI requests, many of them similar. So using a system like this helps people to find the information that’s already published rather than submitting requests for it, when it’s actually already been published.
How does it work?
JK: It’s a customised version of the mySociety WhatDoTheyKnow site, delivered by mySociety for us in the council’s branding. It allows anyone to submit their FOI request in a structured way through the web and others can see the requests and any responses. The requests are linked in with the main WhatDoTheyKnow site to help further reduce duplication of requests and enable consistent commenting.
Behind the scenes it also offers workflow management to assist the council team who are responding to the requests.
What benefits will the system have to those answering and making FOI requests?
JK: It opens up the process, helps others to see what is going on even if they aren’t making requests themselves. Particularly important is that it by default puts requested information out there on the web without any more effort by the council or those making the requests.
Were there any obstacles in setting the system up and how much did it cost the council?
JK: Obstacles were mainly stretched resources within the council to prepare for the changed workflow, making sure our information governance was ready for this and that our web team could support the minor integration work needed.
Given this is a web-based ’software as a service’ offering it’s pretty straightforward to implement in the grand scheme of things. I don’t have the final costs yet as we’ve been doing some post-launch tweaks but, as is the way with nimble organisations like mySociety, I think pricing is very reasonable.
Do you think it will improve the council’s performance in responding to FOI requests and make the council more transparent to the public?
JK: Yes absolutely. Not only will the council’s FOI performance be more publicly accountable but I’m hoping we can reduce duplicate requests through this so that our resources are better focused.
Would you say it has been worth creating and why should other public authorities follow suit?
JK: Yes it’s worth it. I think we as councils have to be ever more open by default, use digital tools for transparency and relentlessly publish data. I believe this will result in better local democracy but also is one of the ways we can truly challenge cynicism in the whole political system.N.B.: The website current shows a large number of requests that appear to be unanswered. We asked about these and it includes the number of historic requests that were loaded into the site.————————————Many thanks to Matt of FOI Directory for allowing us to reproduce this interview in full.