Today we hear from Martin Rosenbaum, the BBC’s Freedom of Information specialist.
Since 2005 I and my colleagues in the BBC have used FOI as the foundation for certainly hundreds and hundreds, possibly thousands, of news stories and investigations at national and regional levels, across a wide range of topics — health, education, policing, environment, transport, foreign policy, and so on.
This has included revelations on important issues from staff shortages in A&E departments to how officials wrongly dismissed predictions about levels of Eastern European immigration, from which makes of cars are most likely to fail MOT tests to the numbers of parents withdrawing their children from schools, from the cost of policing football games to the identities of individuals who have turned down honours.
Journalism is based on asking people questions, but of course much of the time there’s no guarantee you will actually get them answered.
Freedom of information is a rare and valuable tool because it provides a legal right to some information — a right that can be enforced when necessary by independent bodies, the Information Commissioner and the Information Rights Tribunal. And that means FOI provides the power to obtain certain material in the public interest that otherwise could not be squeezed out of reluctant public authorities.
FOI has made a crucial difference to what the media can find out and what the public knows about what central and local government and the public sector is doing.
If you’re a journalist yourself, you might be interested in our latest project.
But don’t forget, FOI isn’t just for journalists: you can make your own requests for information at WhatDoTheyKnow.com.
Here in the UK, two names are particularly linked to FOI: Professor Heather Brooke, the investigative journalist who is responsible for the publication of MPs’ expenses, and Martin Rosenbaum, the BBC’s FOI correspondent.
Today we hear from Heather about the importance of FOI and how she’s used it, and tomorrow you can read Martin’s views.
I took two important FOI cases through the legal appeals process: one seeking the minutes to a BBC Board of Governors Meeting after the Hutton Inquiry1, and my notable legal victory against the House of Commons for details of MPs’ expenses2.
This victory in the UK High Court fundamentally changed law and policy, and for the first time in its history Parliament had to account to an outside body over how MPs’ claimed expenses. The court ruling and subsequent leak of the data led to a number of high-level political resignations as well as full-scale reform of the parliamentary expense regime and passage of the Recall of MPs Act 2015. A new government was elected in May 2010 on a mandate of transparency in part due to the scandal
I made extensive use of the UK’s Freedom of Information Act, filing about 500 FOIs and writing some 60 newspaper and magazine articles about the law and its impact on democracy from 2005-2010. I used the law to map and monitor public bodies for the first time in a citizen-friendly way in Your Right to Know. Through FOI I was able to flag up current and future problems such as secrecy in food safety regulation, the postcode lottery for criminal justice, the amounts police spend on public liability claims and propaganda.
Freedom of Information, rooted in Enlightenment values, contains within it a key principle of democracy that there must be access to information (and knowledge) for all equally. My approach in my 25-year journalistic career has been to use FOI as a means of testing the promise and practice of democracy. By their responses to FOI requests, we see how agencies truly think about citizens’ rights to access and participate in the political system.
Read the next installment to learn how Martin Rosenbaum’s use of FOI has underpinned hundreds, if not thousands, of news stories at the BBC.
If you’re a journalist yourself, you might be interested in our latest project.
But don’t forget, FOI isn’t just for journalists: you can make your own requests for information at WhatDoTheyKnow.com.
1Guardian Newspapers Ltd and Heather Brooke v IC and the BBC (2007) EA/2006/0011; EA/2006/0013
2Corporate Officer of the House of Commons v Information Commissioner & Heather Brooke, Ben Leapman, Jonathan Michael Ungoed-Thomas  EWHC 1084 (Admin) (16 May 2008)
— mysociety (@mysociety) June 27, 2016
That’s the tweet we put out on Monday, after a few days of the fastest-moving politics the UK had seen in years. Little did we know that there was plenty more to come.
And it’s true. Everyone is talking politics — in the street, in the pub, on Facebook. Everyone wants information; everyone wants to express their opinions: which means that TheyWorkForYou and WriteToThem are pretty useful right now.
This has been an interesting week for us here at mySociety. As well as engaging in the same scrolling through fast-changing news stories as the rest of the nation, we’ve been dashing to make a few changes to our sites.
In general, our working methods favour considered actions. We ticket ideas, we discuss them, we prioritise and schedule them, we peer review them, and then they go live. It’s an excellent system for ensuring that work is both necessary and robust. It’s not quite so ideal for working on a new feature you need, like, yesterday, so this has been a change of pace for us.
Information is key
Here’s our first significant addition. Before you email your MP on matters concerning Brexit, it’s useful to know where they stand, so we quickly created an infobox for MPs’ pages on TheyWorkForYou (based on data from the BBC):
Neither of our parliamentary sites needed structural changes: fortunately, they are built robustly and hosted on servers which cope with increased visitor numbers when they occur.
And they are occurring. In a week that has seen the referendum, the resignation of the Prime Minister, mass shadow cabinet resignations, and Conservative leadership nominees, you have had plenty to research and plenty to write to your representatives about.
Here’s what visitor numbers for TheyWorkForYou look like — five times the usual traffic:
And six times as much as usual for WriteToThem:
Increased traffic is no problem (quite the contrary; we love it!), but there were some things that needed our attention. More users means more user support, so we’ve spent more time than usual answering questions about who we are, how we generate our data, what to do if a confirmation email doesn’t arrive, and so on.
Oh, and about that data:
All these resignations- will nobody think of the people who maintain parliamentary monitoring websites?
— Myf Nixon (@mockduck) June 27, 2016
Lots of what’s published on TheyWorkForYou updates automatically, but not necessarily immediately. Parliamentary roles, for example, are only scheduled to update weekly.
That doesn’t allow for the rate of resignations and replacements that we’ve seen in the shadow cabinet this week, so our developers have had to go in and manually set the update code running.
We wanted to remind people that TheyWorkForYou is a great place to research the facts about those standing in a leadership contest. In particular, our voting record pages set out clearly and simply what each MP’s stance is on key issues.
So we’ve been tweeting and Facebooking reminders like this:
At times, the news moved too fast for us to keep up:
Oops! Meanwhile, we also have another useful source of information: the Conservative party speeches that were removed from the internet in 2013 and which we republished on our SayIt platform:
We’re working on improving the way that TheyWorkForYou pages look when you share them on Facebook and Twitter, which seems sensible given that they are being shared so much right now.
How many people visit mySociety’s websites?
That’s a question we don’t ask ourselves as much as many other organisations. Much of our current funding is dependent on transactions (that is, the number of people using the site to complete an action such as making an FOI request, writing to a politician, or signing up to receive emails when their MP speaks), and rightly so, since that is a better measure of the sites’ actual effectiveness.
All the same, visitor numbers* do tell us about things like how much public awareness there is of what we do, and which of our sites is more visible than the others, so it’s good to take a proper look now and again.
Which of our UK sites is most visited?
By far our most popular site in terms of visitor numbers is our Freedom of Information site WhatDoTheyKnow. With over 4.5 million visitors 2014-15, it’s had three times more users than its closest competitor, TheyWorkForYou.
As well as allowing users to submit FOI requests, WhatDoTheyKnow also puts the responses into the public domain, so that the information becomes openly available. Every request receives, on average, twenty readers, meaning that transactions do not show the whole picture for this site.
WhatDoTheyKnow’s user numbers are also rising steadily. It’s up 8% on last year, and March 2015 was its highest month for unique users since its launch in 2008, at 470,509.
Which is least visited?
This dubious honour goes to WriteToThem, which nonetheless welcomed 457,209 visitors during the year, either helping them to write to their representatives, or simply showing them who those representatives were.
This was still a decent 11% rise on the previous year, despite a real rollercoaster where some months dipped substantially from the previous year.
Which made the most gains in the last year?
FixMyStreet saw the biggest percentage change, with a 21% rise in visitor numbers compared to the previous year; we talked a bit more about that in a recent blog post. WhatDoTheyKnow had the highest rise in actual visitor numbers: over 360,000 up on 2013-14.
Which fell by the most in the last year?
TheyWorkForYou saw a 12% drop in visitor numbers year on year (and also the biggest drop in real terms)—disappointing, but something we hope to rectify with the new voting pages, an ongoing process of rolling redesign, and some grassroots outreach.
How much effect do external events have on visitor numbers?
We already know that, as you’d expect, when Parliament is on holiday, MPs, debates and legislation aren’t in the news, and TheyWorkForYou visitor numbers fall. There’s also a weekly pattern for all our sites, where far fewer people use them at the weekends, presumably indicating that lots of our users access them from work.
It’s too early to say exactly what effect the election has had on our sites: as I write, people are eagerly checking out the voting records of newly-appointed cabinet ministers on TheyWorkForYou.
One thing we know for sure is that fewer people will have been using WriteToThem, because there have been no MPs to write to for the last few weeks. We’ve removed the “write to your MP” links from TheyWorkForYou, which always drove a good deal of WriteToThem’s traffic.
FixMyStreet enjoyed a boost back in June, when it was featured on the Channel 4 programme ‘The Complainers’—and the nice thing is, user numbers never receded back to their previous levels after the programme was over. Maybe people just need to use FixMyStreet to see how useful it is.
How many people visit mySociety’s UK websites in total?
This is a difficult figure for us to produce with accuracy, because we don’t trace whether you’re the same person visiting a number of our different sites.
However, the aggregate total of visitors to all our UK sites (WriteToThem, TheyWorkForYou, FixMyStreet and WhatDoTheyKnow) for 2014-15 is 6,983,028. Thanks very much if you were one of them 🙂
How can I help?
Glad you asked! If you find mySociety sites useful, you can help us spread the word by telling friends, sharing the URLs with any groups you are a member of, posting on Facebook or Twitter, or writing to your local paper.
We have a number of materials for FixMyStreet which can be found here; we hope to create similar materials for our other sites too, and we’ll make sure we announce it on here when we do.
* Note: all references to ‘users’ refer to unique users within the period discussed. So, users in a year means individual people who may have visited any number of times over that year, but are only counted once; same with monthly users.
This is a problem we have been warning about for some time. Islington Council were fined £70,000 for a similar incident in 2012. In light of this fresh incident we again urge all public authorities to take care when preparing data for release.
As with the Islington incident, the information was in parts of an Excel spreadsheet that were not immediately visible. It was automatically published on 14th November when Hackney Council sent it in response to a Freedom of Information request, as part of the normal operation of the WhatDoTheyKnow website. All requests sent via the website make it clear that this will happen.
This particular breach involved a new kind of hidden information we hadn’t seen before – the released spreadsheet had previously been linked to another spreadsheet containing the private information, and the private information had been cached in the “Named Range” data in the released spreadsheet.
Although it was not straightforward to access the information directly using Excel, it was directly visible using other Windows programs such as Notepad. It had also been indexed by Google and some of it was displayed in their search previews.
The breach was first hit upon by one of the data subjects searching for their own name. When they contacted us on 25th November to ask about this, one of our volunteers, Richard, realised what had happened. He immediately hid the information from public view and notified the council.
We did not receive any substantive response from the council and therefore contacted them again on 3rd December. The council had investigated the original report but not understood the problem, and were in fact preparing to send a new copy of the information to the WhatDoTheyKnow site, which would have caused the breach to be repeated.
We reiterated what we had found and advised them to consult with IT experts within their organisation. The next day, 4th December, we sent them a further notification of what had happened, copying the Information Commissioner’s Office (ICO). As far as we are aware, this was the first time the ICO was informed of the breach.
From our point of view it is very disappointing that these incidents are still happening. Freedom of Information requests made via WhatDoTheyKnow are a small fraction of all requests, so it is very likely that this kind of error happens many more times in private responses to requesters, without the public authority ever becoming aware.
Our earlier blog post has several tips for avoiding this problem. These tips include using CSV format to release spreadsheets, and checking that file sizes are consistent with the intended release. Either of these approaches would have averted this particular breach.
We would also urge the ICO to do as much as possible to educate authorities about this issue.
Simple things are the most easily overlooked. Two examples: a magician taking a wand out of his pocket (see? so simple that maybe you’ve never thought about why it wasn’t on the table at the start), or the home page on www.fixmystreet.com.
The first thing FixMyStreet asks for is a location. That’s so simple most people don’t think about it; but it doesn’t need to be that way. In fact, a lot of services like this would begin with a login form (“who are you?”) or a problem form (“what’s the problem you want to report?”). Well, we do it this way because we’ve learned from years of experience, experiment and, yes, mistakes.
We start off by giving you, the user, an easy problem (“where are you?”) that doesn’t offer any barrier to entry. Obviously, we’re very generous as to how you can describe that location (although that’s a different topic for another blog post). The point is we’re not asking for accuracy, since as soon as we have the location we will show you a map, on which you can almost literally pinpoint the position of your problem (for example, a pothole). Pretty much everyone can get through that first stage — and this is important if we want people to use the service.
How important? Well, we know that when building a site like FixMyStreet, it’s easy to forget that nobody in the world really needs to report a pothole. They want to, certainly, but they don’t need to. If we make it hard for them, if we make it annoying, or difficult, or intrusive, then they’ll simply give up. Not only does that pothole not get reported, but those users probably won’t bother to try to use FixMyStreet ever again.
So, before you know it, by keeping it simple at the start, we’ve got your journey under way — you’re “in”, the site’s already helping you. It’s showing you a map (a pretty map, actually) of where your problem is. Of course we’ve made it as easy as possible for you to use that map. You see other problems, already reported so maybe you’ll notice that your pothole is already there and we won’t have wasted any of your time making you tell us about it. Meanwhile, behind the scenes, we now know which jurisdictions are responsible for the specific area, so the drop-down menu of categories you’re about to be invited to pick from will already be relevant for the council departments (for example) that your report will be going to.
And note that we still haven’t asked you who you are. We do need to know — we send your name and contact details to the council as part of your report — but you didn’t come to FixMyStreet to tell us who you are, you came first and foremost to report the problem. So we focus on the reporting, and when that is all done then, finally, we can do the identity checks.
Of course there’s a lot more to it than this, and it’s not just civic sites like ours that use such techniques (most modern e-commerce sites have realised the value of making it very easy to take your order before any other processing; many governmental websites have not). But we wanted to show you that if you want to build sites that people use, you should be as clever as a magician, and the secret to that is often keeping it simple — deceptively simple — on the outside.
EDITED TO ADD: Wow, we have reached capacity on this event for the 17th. If you’d like to come, please drop us a line at the address below, and we’ll let you know if a space becomes available. Alternatively, ask to be added to the list for the next week.
NB, If you have claimed a space, please let us know if you won’t make it, so we can offer your space to someone else.
Hey you! Yeah, you, the cool one, hacking on civic and democratic tech. Or thinking about it, at least.
How about dropping in to a mySociety hack night, hosted by the Open Data Institute? It’s a great space, and there’ll be food, drink, and friendly people to talk through your ideas with and actually get those projects on the go.
No agenda and no talks; it’s just a chance to spend some time showing what you’re working on, swapping ideas and getting advice from other people interested in the fields of civic technology and open data.
Where? At the Open Data Institute, 65, Clifton Street, London EC2A 4JE. We’re on the third floor. Map
When? Weekly from Wednesday 17th July, 6:00 – 9:00 pm
Who? Anyone with a bit of technical knowledge. We’ll be there to chat things through and maybe help with sticky problems.
Why? Get going on that website or tool you’ve always wanted to build; meet people who are interested in the same stuff as you; oh and did we mention the food?
If that all sounds good, drop us a line on email@example.com to let us know you’re coming to be added to the wait list.
Photo by Nathanael Boehm (CC)
You may remember our recent post announcing that we’d been nominated for an Emmy and a BAFTA award. Always the bridesmaid and never the bride, you might have thought.
As you’ll recall, we created the app and the website tools for the Channel 4 programme alongside production companies Tiger Aspect and the Project Factory, who share the accolade.
So: yays all round – and don’t forget, our award-winning skills are for hire.
We were really pleased by this report on BBC Oxfordshire this morning.
Oxfordshire County Council is one of the local authorities who have integrated FixMyStreet into their own website. We’re delighted to see what a success it’s been for them: over 15,000 potholes fixed since its installation in March.
We can’t take any credit for the actual repairs, of course, but we like to think that FixMyStreet’s easy interface has simplified the reporting process for the people of Oxfordshire. Read more about FixMyStreet For Councils here.
Photo by Tejvan Pettinger (CC)
Many mySociety projects rely on a team of volunteers to keep them going. FixMyTransport, WhatDoTheyKnow and Pledgebank may look like very simple sites that run themselves, but the truth is that there’s a lot of human intervention going on behind the scenes, keeping the wheels oiled.
Our volunteer teams deal with masses of site admin, they discuss policies and future development, and they give advice to our users. They may also go and talk about our projects in the wider community, and this is what WhatDoTheyKnow volunteer, Richard Taylor, did recently when he addressed the Association of Chief Police Officers at the “Transparency in UK Policing” event.
Richard has written about his experience here; I am linking to it because, as well as giving a good introduction to WhatDoTheyKnow within a policing context, it also explains exactly what sort of work the WhatDoTheyKnow volunteers do routinely, and the kind of issues that are discussed within the team. It might just make you value our volunteers more, or it might pique your interest in becoming one yourself.
If that latter applies, you can find out more about volunteering for WhatDoTheyKnow here, or about the ways you can help across all mySociety projects here. But either way, I encourage you to go and read Richard’s post.
Photo by Aaron van Dorn (CC)