In our previous post, we identified WhatDoTheyKnow’s current need for sources of funding.
But WhatDoTheyKnow also needs more volunteers to join the team. Since the site’s launch, it’s always depended on a highly-motivated, active group of administrators who work to keep it running.
At mySociety, we’re very grateful for the work the volunteers do; for their part, they tell us that they find the work rewarding and interesting — but we’re always aware that we can’t, and shouldn’t, demand too much from them. The more volunteers we can recruit, of course, the less the workload will be for everyone.
We’ve identified three general areas in which volunteer help would be very welcome, and if you think you’d fit in to any of these, we’d love to hear from you.
- interested in FOI and transparency
- happy to work remotely but as part of a team, communicating mainly via email
- able to dedicate a minimum of a few hours per week to helping run the site
Each of our volunteer administrators give their time freely and are the only reason we can run the service day to day at all.
Being a volunteer is both rewarding but also challenging, as each juggles their day jobs and home lives. So the more volunteers we have, the more we can spread the workload between them.
If you have a specific interest in FOI or transparency, or indeed you’d just like to help support a well used civic tech service then we’d love to hear from you. There is always a diverse range of jobs and tasks needing to be done, even if you can only help a couple of hours a week. We all work from home and communicate via email and other online tools.
If you can help us a volunteer the first thing to do is to write to the team introducing yourself and letting us know about your relevant skills, experience and interests.
- a law student or professional who can offer expertise in the day-to-day running of the site; or
- a legal firm or chambers who could offer legal advice on an ad hoc, pro bono basis
Volunteers with legal backgrounds We take our legal and moral responsibilities in running WhatDoTheyKnow very seriously and we always welcome volunteers with experience of legal matters. Some of the legal aspects of running the site are handled routinely on a day to day basis by the admin team.
They may, for example, remove correspondence which could give rise to claims of defamation, or where personal data is disclosed by an authority mistakenly and they consider continued publication to be unwarranted.
The legal challenges thrown up by operating our service are varied and interesting. Joining us could be an opportunity for someone to get some hands on experience of modern media law, or for a more experienced individual, to provide some occasional advice and guidance on more challenging matters.
We often find ourselves balancing claims that material published on our site could aid criminals or terrorists, or could cause harm in other ways, and we do our best to weigh, and balance, such claims against the public interest in making the material available.
As material published on our website may have been used to support news articles, academic research, questions from elected representatives, and actions by campaign groups or individuals it’s important we don’t remove correspondence lightly and that we’re in a position to stand up, where necessary, to powerful people and institutions.
Legal firms that can offer advice As from time to time there are cases which are more complicated, we would like to build a relationship with a legal firm or chambers that can advise us on an ad hoc basis on defamation, privacy (misuse of private information) and data protection.
The ability to advise on copyright law and harassment law would also be an advantage. And we also on very rare occasions may need help as to how to respond to the threat of litigation.
Could you offer help in this area? Please do get in touch to discuss getting involved.
- a committed, organised, empathetic person who could volunteer a few hours (working from home) a week
In our previous post we mentioned that we’d ideally secure funding for an administrator who could handle our user support mail and deal with routine but potentially complex and time-sensitive tasks such as GDPR-based requests.
While we seek funding for this role, would you be willing to fill it on a voluntary basis? Please get in touch.
Lots to help with
So in summary, what we need to keep WhatDoTheyKnow running is money, volunteer help, and legal support. If you can help with any of these, or have some ideas of leads we might be able to follow, please do get in touch. It also helps to share this post with your networks!
Alternatively, you can help out with a donation large or small — every little helps.Donate now
Image: CC0 Public Domain
I’m Richard Taylor, a member of the volunteer team which administers WhatDoTheyKnow.com on a day to day basis, and I spoke at the event highlighting the broad range of people who have collaborated to make WhatDoTheyKnow a success, and sharing some ideas for the future. Here’s what I said:
I’m someone who wants to see our representative democracy working; that’s why I support what mySociety does; I support giving tools to people to help people engage with our society, how we make decisions about running our society, how we run our public services, our health service, policing, how we organise our cities, how we plan development of new homes and design, or evolve, our transport systems.
I joined WhatDoTheyKnow as a user on the 22nd of July 2008, so almost exactly ten years ago. My first Freedom of Information requests were on policing, for the local Stop and Account policy – as you can see from those kinds of requests I’m keen on transparency and accountability of those we give powers over us. I looked up my early FOI requests and I was rapidly onto my local councillor allowances, details of which weren’t online, and as I’m from Cambridge and there were some very Cambridge requests in there too – on the running of the river – on the regulation of punting – a perennial local issue, and for the terms and charges for grazing on the city’s commons. One of the things I do is campaign for proportional police use of TASERs, I made requests on that subject too.
Within just a few days of joining the site I was sending in lists of public bodies to add to the system; and shortly after that I was invited on to the administration team so I didn’t have to bug developer Francis Irving, or the volunteers who’d already started to help running the site, including John Cross, Alex Skene, Tony Bowden to do things like add new bodies, but could make changes myself.
The volunteer team
Mine is the same route many of our volunteers took to joining the team running the site in the early years; those making lots of good proposals for bodies to add, or making other suggestions were invited to help out. The way we’ve found new volunteers has changed a little over time, and we have had to keep topping up the pool of volunteers as people have moved on. We started to approach users of the site who were making helpful annotations assisting other users, and who were making great use of the site themselves. We found Ganesh Sittampalam and Doug Paulley that way, both of whom have put huge amounts of time into developing the site, the service. Latterly we’ve moved to advertising for new team members and seeking applications from those who want to join us, and that’s brought us some of our current active volunteers, Michael Bimmler for example.
Volunteers have put in an enormous amount of time into running the site. If you put a cash value on that time I’m sure the volunteers would by far be the biggest donor to the site. The site probably wouldn’t exist, and certainly wouldn’t exist in its current form without volunteer input; so many good ideas for websites get built, often with funding to kick them off, but they don’t do what WhatDoTheyKnow has done, and survive, grow, and thrive. Volunteer input has enabled that.
The site certainly has grown and thrived, we now have around seven million users viewing the site per year; according to Google analytics, and 162,000 signed up users. There are approaching half a million request threads on the site now. An interesting aspect of those statistics is the viewing is not focused on a small handful of requests, but rather visitors are spread broadly across the long-tail of requests and released information. In 2016 17% of requests to central government monitored bodies went via our service; but the vast majority of requests, 88%, go to bodies where central government don’t track FOI request statististics.
The volunteers I’ve mentioned already, plus Helen Cross and Alastair Sloan, have put substantial chunks of time into running the site. There are many others too including Rob McDowell, Ben Harris, Gavin Chait and Peter Williams. The volunteers supporting the service have not just come from the volunteer team, the trustees who’re ultimately responsible for the site are volunteers too, ten years ago mySociety was more of a volunteer based organisation than it is now, trustee Amandeep Rehlon was dealing with the finances on a volunteer basis, we’ve had great moral and policy guidance from Manar Hussain and Owen Blacker, and the chair of the trustees, another volunteer, James Cronin.
We have been amazingly lucky with the volunteers we’ve attracted to the administration team. Doug Paulley is an incredible activist and campaigner on disability rights, and so many of the others are legal and information rights experts, activists and campaigners in their own rights.
Volunteers are only part of the story, we wouldn’t be able to do what we do, and what we want to do without the institutional support of mySociety, and the organisation’s brilliant staff. When the initial developer and project manager Francis Irving moved on he was succeeded by a series of great lead developers, Robin Houston, Seb Bacon, and now Louise Crow and other staff team members, currently Gareth Rees, Graeme Porteous, Liz Conlan …(See Github for the full list of contributors to the code!) the site is supported by the whole mySociety team, including designers Zarino Zappia and Martin Wright, Abi Broom, who runs the show, Gemma Moulder – events organiser from our perspective, who also works on spreading services based on WhatDoTheyKnow around the world, and mySociety’s communications person Myf Nixon. Thanks are also due to ten years’ of mySociety sysadmins including Sam Pearson,Ian Chard, and in the early days volunteers who’d keep things running, Adam McGreggor, and Alex Smith.
A key WhatDoTheyKnow volunteer was Francis Davey who was our volunteer legal advisor for many years. Francis Davey’s top piece of advice which I recall was to avoid court. We’ve pretty much succeeded to date-with that. One of the key roles of the volunteer team is to run what is a relatively legally risky site without getting sued and consequently, probably, taking down not just WhatDoTheyKnow but the rest of mySociety too.
We deal with a lot of defamation claims, personal information takedown requests, and an array of more obscure legal challenges.
As well as trying to avoid getting annihilated via legal processes a key aspect of our approach to running the site is we try our utmost to run it responsibly. What those involved didn’t do is find a legally friendly jurisdiction and anonymously just let the system loose to run unmanaged and unchecked. We’re real accountable people who respond to concerns from all comers, individuals, public bodies, our own users, about what’s published on our site.
What are we doing by running our site?
We’re doing a lot more than just helping users make a request for information to a public body. We’re activists, we’re promoting running our society in a transparent, inclusive, accountable, way, not just by lobbying, making speeches, writing articles, but by doing something, by running our site.
Running our service promotes Freedom of Information and other access to information laws; people come across our site when searching for information they’re seeking; we show what can be obtained by publishing requests and responses; others might find the information they’re seeking directly, or see that they can make a similar request, perhaps adapting a request that’s been made elsewhere to their local public bodies..
Anyone can make a Freedom of Information request by private email to a public body. I’d find that potentially a bit of a selfish action, incurring cost to the public for a response only I might see, but making a request via WhatDoTheyKnow to obtain information which should be accessible to the public automatically makes that information accessible to anyone who searches for it, anyone who Googles for the information. Even if a requester doesn’t themselves do something with the information released by making a request via WhatDoTheyKnow.com they’ve enabled others to do so. You’re often doing public good just by making a request via WhatDoTheyKnow.com (though do see our advice on making responsible and effective requests).
WhatDoTheyKnow makes something which would otherwise be quite challenging for many people – getting a FOI request and response online – easy. I’m sure only a fraction of users of our site would have taken the time to write a blog about their request, and update it with the response, if they had to do that manually.
A big benefit of making a request on WhatDoTheyKnow.com is many people are already using our site and watching for responses; if you make a request to a local council on WhatDoTheyKnow.com the chances are your local journalists are tracking requests to the local council and they’ll be alerted to any response.
At WhatDoTheyKnow we’re an independent third party, we’re not the requestor and we’re not the public body. This can be useful when there’s a dispute about a response to a request, if a public body denies receiving it for example. We’d love to work more closely with the regulator, the Information Commissioner’s office, we’d love them to use our service more to help them in their role in enforcing the law. Often just having a request on our site can help people get a response, good public bodies really care about the impression those visiting their pages on our site get. Lots of public bodies will get in touch with us if they don’t like the way a request has been classified by one of our users for example.
A really big advantage having information released via our service is people can cite it when they take action based on it, be that action a blog post, an article in the media, an academic publication, or a letter to an MP. You can show, again using WhatDoTheyKnow.com as an independent third party, where the information you are relying on has come from, giving more weight, more credibility, to whatever it is you’re doing, your lobbying, your journalism, your research. WhatDoTheyKnow, and mySociety more broadly, has been in the business of enabling better informed debate and higher quality journalism well before “fake news” entered our lexicon.
We’re always looking for new bodies to add to our site, the database of public bodies which is behind the site keeps growing, we’re now at over 23,000 public bodies. That compares to about 450 public bodies listed on the Gov.uk website, and just 305 in the latest “Public Bodies” report by the Cabinet Office. The big difference is made up by schools, GP surgeries and NHS dentists, all of which are subject to FOI; we also list groups of organisations like companies owned by local government – public bodies in terms of the Freedom of Information Act but all but invisible to central government.
I said we were in the business of activism; changing society by doing things. One big part of or Freedom of Information law related activism is listing bodies on our site which are not, or not yet, subject to access to information laws. We’ve listed many bodies before they became subject to the Freedom of Information Act, showing the demand for information, and showing the kind of information people want, but couldn’t access. One example was Network Rail which we listed before it became subject to FOI in March 2015, another was the Association of Chief Police Officers .. however that’s now become the National Police Chief’s Council and MPs failed to make that successor body subject to FOI – in that case it’s not a huge problem as they realise they need to be transparent and they voluntarily comply, but, significantly, the Information Commissioner can’t enforce a law which a body is not technically subject to.
There are always more public bodies to add, we list Housing Associations for example, they’re a another class of body which are not subject to FOI, even coroners aren’t subject to FOI which you might find surprising given their important public role in ensuring our society is safe, and more people don’t die in the future for the same, preventable, reasons people have died in the past. We list some coroners, and volunteer Kieran is working on making our coverage comprehensive. Local medical committees; committees of GPs are another set we’re hoping to add soon.
Maintaining the body database is a constant task. Government is constantly reorganising, we try to keep up with changes recently for example, recently, in research councils, and keeping track of NHS reorganisations is a challenge on its own. There have already been 17 requests to London North Eastern Railway Limited, the Government rail operator of last resort which we listed when it took over running trains on the East Coast mainline about ten days ago.
Seeking improvements to laws which impact our service, its users, and the accessibility of public information
As well as our activism we have a record of more traditional lobbying; sharing the experience running our service has given us experience of the operation of access to information law. We took part in the Post-Legislative Scrutiny of the Freedom of Information Act 2000 in 2012 for example, and just a few months ago we responded to a consultation by the Cabinet Office on the Code of Practice which bodies responding to FOI requests have take into account.
In terms of what we’re calling for, we’re not FOI fans specifically, we’d actually rather people didn’t have to make FOI requests, we’re in favour of proactively releasing information and running public services transparently, though that said FOI requests are requests for information people want to know; rather than information which public bodies want to publish so they will probably always have their place.
Why not make public bodies consider proactive publication of information of the sort requested, when dealing with a FOI request? That’s a provision which is in the specialised law on access to datasets but doesn’t apply to access to information requests more broadly.
Timeliness of responses, and timeliness of enforcement action from the Information Commissioner are other key things we campaign on. If you want a copy of a FOI response that’s been made to particular union, lobby group, or journalist and is the information behind the day’s news, surely you should be able to get a copy of it pretty much straight away, and there can be no excuse for a body dithering until the 20 working day deadline. The law requires a prompt response; that aspect of the law needs following and enforcing.
We also want to close loopholes in FOI; one terrible one, is if a public body can think of a class of information and list it on its website with a price for it, it becomes exempt from disclosure for free under FOI. This is clearly open to abuse, fortunately few bodies have misused it too-date, but there are examples – just look at your local council’s list of information they make available for a fee.
Running the Site
Some might be interested to know administration has changed as the site has grown. There’s been a constant improvement of the site’s software to make it easier to run, but that needs to continue so we can cope with it getting bigger without having to increase the volunteer effort exponentially in-line with the site’s growth. We’ve outgrown the team@ mailing list system the site started with; we now separate the support mail from discussions among volunteers, and on top of that there’s a separate discussion of legal matters; so people aren’t overwhelmed.
One challenge we have is the workload, and volunteer input, are both variable. Sometimes there’s a week where you really need someone full time running the site. Sometimes you could firefight the incoming issues in maybe an hour a day, or day a week.
Something we’d like to do is encourage past volunteers to join our monthly calls; join the legal discussion list, volunteers list, drop into the support mailbox and help out on occasion, every little helps, following what we’re doing for a week a couple of times a year might provide some outside, detached, input; help keep us on-track, challenge us, and assist us in spotting drifts in policy / practice.
Ideas for the future
We’re always keen to hear any ideas for what we could be doing better, or differently we welcome input from anyone and everyone who cares about the service in some way. Some of the things we could do improve:
- We could do even better at transparently running the site. We already try to run the site as transparently as we can; if we hide a message, or redact content from correspondence, we make clear where we’ve done so and explain why. We don’t though have a transparency report like Google and Reddit do, reporting on takedown requests, how many there have been, who they’ve come from – individuals, requesters, public bodies, public officials, regulated professionals, and how we responded. Requests for user data. One challenge is sometimes the moral thing to do is not shout about and draw attention to something we’ve taken down too quickly; don’t want to draw attention to taking down something that’s still in Google’s cache for example – if we really believe it shouldn’t be online any more.
- We should do more to highlight excellent, interesting and influential, uses of the site. It would be great to have ways within the system to note when responses have been used by others, cited in Parliament, resulted in a news story, or if someone has analysed responses from a range of public bodies around the country for example.
- We have volunteers, but there is no real community of users around the site, or around our lobbying activities, or, to the extent there has been in the past, a community – around mySociety any more. There’s an opportunity there..
- I think we have a duty to be careful with the way the WhatDoTheyKnow pro-service is used. Anyone can sign up for a Gmail account and make requests; but we are doing more than Gmail to encourage and enable FOI requests, and not least the pro system is built on a largely volunteer built and maintained database. Use to-date has apparently been good, and we have a general principle of not spending time discussing hypothetical situations, but, in running the free site as volunteers we’ve always been mindful of the impact of our actions on our reputation, and the reputation of Freedom of Information law itself. For example we ask those considering bulk requests if they’ve carefully selected the set of bodies to make their request to, if the request could be made to a central body rather than lots of local bodies, if a sampling exercise would suffice instead of asking perhaps hundreds of bodies, and we advise on making clear requests in the first instance to reduce the need for clarifications – saving public bodies and requesters time and effort. [Update: following the event we agreed to update our House Rules to include a reference to our advice on making responsible and effective requests|]).
- Lastly, on sustainable funding for the site, ideally I think this would be though a handful of media organisations, campaign groups, or other bodies paying for a pro-service; which would hopefully give them great value in terms of organising FOI requests, prompting them to chase up late requests, saving time finding contact details and easily making bulk requests. Perhaps as the number of individual users of the Pro service grows organisations will see the value of providing access to all their staff.
About six million people a year visit mySociety’s Freedom of Information website WhatDoTheyKnow.com; there are well over 100,000 registered users, and over 385,000 requests have been made via the service.
Of course, it’s fantastic that WhatDoTheyKnow is so well used, but the growth and popularity of the site brings its own challenges, not least the day-to-day admin that keeps the site running.
Many aspects of the site’s operation are run by volunteers, supported by mySociety’s staff and trustees — and due to the site’s success we’re looking to expand the volunteer team.
What does volunteering involve?
The work is pretty varied, but there are some frequent and recurring tasks:
Dealing appropriately with requests to remove material from the website
This is one interesting challenge which arises fairly often. Sometimes these requests are from public bodies who’ve released information they didn’t mean to; and they can also come from individuals and companies who are named in correspondence on the site.
These decisions are not always as black and white as you might expect. Some recent examples where we had to carefully consider the balance on both sides were:
- Material which Transport for London were concerned could be used to steal a tube train. We considered: was this a genuine risk? Was our publication really increasing the risk? Was the information already available elsewhere? What was the potential value of publishing the information to tube staff and their representatives, travellers and the wider public?
- Sainsbury’s were concerned that published material didn’t reflect their corporate policy on “workfare”. That may have been the case, but we asked ourselves whether that made it inappropriate to continue to publish the information that had been released. Additionally, where did the public interest lie? What legal risks were there arising from continued publication?
Responding promptly and accordingly to accidental releases
Thankfully, the frequency with which public bodies accidentally release personal information in bulk via Freedom of Information responses is decreasing, but the WhatDoTheyKnow team still have to act promptly when this does occur.
We often help users on both sides of the FOI process. For requesters, we can answer questions about FOI and how to use it, and we also work with the staff of public bodies who are at the receiving end of requests.
And all the rest
There’s always more that can be done to promote the service, draw attention to interesting correspondence on the site, and lobby for improvements to our access to information laws.
The wider team at mySociety help people around the world to establish and run their own online Freedom of Information services; and new features are being added to the UK site to make it more attractive to professional users such as journalists and campaign groups. Volunteers have the opportunity to get involved in these activities, helping steer the direction of new projects, based on their frontline experience of being a site administrator.
Keeping the database of thousands of public bodies up to date is another challenge, especially given the frequency of reorganisations in the UK’s public sector.
We work primarily by email, with regular video conferencing meetings, and occasionally meet up in person.
As a volunteer, you can decide how much time you put in, and what aspects of running the service you decide to take part in — but ideally we’re looking for people who can spare at least an hour or two, a couple of days a week.
We understand that people’s external commitments vary over time, and of course, there’s a flexible approach if a team member needs to step away for a stretch now and then.
What makes a good WhatDoTheyKnow volunteer?
There’s one characteristic that all the WhatDoTheyKnow volunteers have in common: a belief in the value of Freedom of Information, or, more widely, the expectation of transparency and accountability from the bodies which citizens fund.
As for practical skills: perhaps you’ve been involved in moderating discussions on the web, or have experience with access to information, defamation, or data protection law. Or perhaps you have, or would like to gain, experience dealing with “customers” by email.
Primarily we’re looking for people capable of making good judgements, and who can communicate clearly online.
Before joining the team, new volunteers will have to agree to follow our policies covering subjects such as security and data protection. That said, part of the role may be, if desired, taking a part in developing and refining these, and other, policies as the service grows and changes.
How to apply
If helping us run WhatDoTheyKnow sounds like the kind of thing you’d be interested in doing, then please do apply to join us.
We only have the capacity to bring on and train a few volunteers at a time, and it is important that those chosen to help administer the service are trustworthy and committed to its policies, direction and non-partisan stance. For these reasons, we are recruiting volunteers via a formal application process.
To apply please write to us before the 20th of March 2017, introducing yourself, and letting us know about any relevant interests or experience you have.
What do we offer in return?
As a volunteer, the main reward comes from the satisfaction of assisting users, making good decisions, and helping run what is fast becoming a key part of the country’s journalistic and democratic infrastructure.
Volunteers may be invited to mySociety events and meet-ups, providing a chance to take part in discussions about the future direction of the service and the organisation’s activities more generally. There have been a number of conferences held, where those running Freedom of Information sites around the world have got together to share experiences: one or more volunteers may be invited to join in, with travel expenses paid.
Other ways to help out
If volunteering to join the WhatDoTheyKnow team isn’t for you, perhaps there’s something on mySociety’s Get Involved page that is — or you could:
Image: MarkBuckawicki [CC0], via Wikimedia Commons
We were shocked and saddened to learn that Peter Williams, one of our WhatDoTheyKnow volunteers passed away earlier this week.
Peter only joined WhatDoTheyKnow as a volunteer in March of this year, however in that short time he had become a valued member of the team.
No stranger to Freedom of Information, he had been using the rights provided for in the FOI Act since shortly after it came into force here in the UK, and had been a user of WhatDoTheyKnow since 2012.
Education and senior executive pay and benefits were some of his particular areas of interest, and Peter was researching the reasons why some public bodies sometimes fail to respond to requests.
As a consequence, Peter had helpfully been collecting information on specialist colleges and schools, and proposing additions and edits to the site. Following the same route that has led to several of our keenest users becoming volunteer administrators, he was invited to join the team so that he could make the changes he was proposing himself.
By all accounts he made a strong impression during his short time on the team, both with his fellow volunteers and across the mySociety team.
“Many aspects of the site’s operation, including dealing with correspondence from users, considering requests to remove material from the site, and discussing our policies and the future development of the service, benefited from Peter’s input”, says one.
Others say: “He was a valued volunteer and a great person”; and “he was a funny, thoughtful and committed guy”.
We were saddened to learn of death of someone who shared our beliefs in the value of making information held by public bodies accessible, and who shared our passion for activism.
All are feeling the loss of a colleague who approached his role with such enthusiasm and diligence, and our team will be the poorer for his absence. On behalf of mySociety, our Trustees and the WhatDoTheyKnow volunteers our thoughts are with Peter’s family and friends.
As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
This is a problem we have been warning about for some time. Islington Council were fined £70,000 for a similar incident in 2012. In light of this fresh incident we again urge all public authorities to take care when preparing data for release.
As with the Islington incident, the information was in parts of an Excel spreadsheet that were not immediately visible. It was automatically published on 14th November when Hackney Council sent it in response to a Freedom of Information request, as part of the normal operation of the WhatDoTheyKnow website. All requests sent via the website make it clear that this will happen.
This particular breach involved a new kind of hidden information we hadn’t seen before – the released spreadsheet had previously been linked to another spreadsheet containing the private information, and the private information had been cached in the “Named Range” data in the released spreadsheet.
Although it was not straightforward to access the information directly using Excel, it was directly visible using other Windows programs such as Notepad. It had also been indexed by Google and some of it was displayed in their search previews.
The breach was first hit upon by one of the data subjects searching for their own name. When they contacted us on 25th November to ask about this, one of our volunteers, Richard, realised what had happened. He immediately hid the information from public view and notified the council.
We did not receive any substantive response from the council and therefore contacted them again on 3rd December. The council had investigated the original report but not understood the problem, and were in fact preparing to send a new copy of the information to the WhatDoTheyKnow site, which would have caused the breach to be repeated.
We reiterated what we had found and advised them to consult with IT experts within their organisation. The next day, 4th December, we sent them a further notification of what had happened, copying the Information Commissioner’s Office (ICO). As far as we are aware, this was the first time the ICO was informed of the breach.
From our point of view it is very disappointing that these incidents are still happening. Freedom of Information requests made via WhatDoTheyKnow are a small fraction of all requests, so it is very likely that this kind of error happens many more times in private responses to requesters, without the public authority ever becoming aware.
Our earlier blog post has several tips for avoiding this problem. These tips include using CSV format to release spreadsheets, and checking that file sizes are consistent with the intended release. Either of these approaches would have averted this particular breach.
We would also urge the ICO to do as much as possible to educate authorities about this issue.
This month we released a new version of FixMyStreet. Amongst the new features, fixes, and thingamajigs were some small improvements added by two volunteers, Andrew and Andy.
Even though these are not core pieces of functionality — in fact, precisely because they are not — we want to draw your attention to why they were included, and why this is a Very Good Thing. And perhaps, if you’re a coder who wants to put something into an Open Source project but hasn’t quite found a way in, Andrew and Andy’s work will nudge you into becoming a contributor too.
One of the axioms of Open Source is that, because anyone can read the source code, in theory anyone can contribute to it. In practice, though, it’s not really as simple as that. Both ends of the “anyone can contribute” idea require effort:
- Before contributing to a project of any complexity (as we hope you want to do), there’s often a lot to learn, or figure out, before any work can even begin;
- Before accepting contributions to such a project (keen as we are to do so), there’s an overhead of testing, checking, and managing the incoming code.
The ugly real world
The basic issue here is that software is complex — no matter how well-written, tested and documented program code is, if the problem it’s solving is in the real world, it’s not going to be simple.
This is especially true of anything used by the public, because often you can only make things seem simple at the front (such as a clean web interface or “user journey”) by working hard behind the scenes with data structures and processes that handle the underlying complexity. It’s inevitably true of any projects which have been developed over time — programmers like to use the term “legacy code” to describe anything that wasn’t written then way they’d choose to write it now.
Often the problems that software is solving are not quite as obvious as they first appear. At mySociety we’ve got a wealth of experience and actual usage data that ultimately changes the way we build, and develop, our platforms. We understand the fields we work in well (technically, the nerds like to call these the “problem domain”), whether it’s governmental practice or civic user behaviour, and that’s often knowledge that’s not encapsulated anywhere in the program code.
Furthermore, any established platform must protect against the risk that new changes break old behaviour — something that regression testing is designed to catch. This is especially important on platforms like FixMyStreet or Alaveteli where the software is already running in multiple installations.
This is why we have a team of full-time, experienced, and (thanks to our rigorous recruitment process) talented programmers who can invest the time and effort to be familiar with all these things when they set to work coding.
But this builds up to an impediment: sensitivity to any of these issues is enough to make anyone think twice about simply forking our code and starting to hack on it for us.
How it sometimes works
In practice, then, how does anything get contributed? How come it doesn’t all get written by our own coders?
The answer is, of course, we do work with major contributors outside our own team (have a look at the activity on our github repos to see them) — but it always requires a period of support and on-line discussion both before and during the process. There’s also the business of testing, and sometimes politely pushing back on, pull requests (which is how code contributions are submitted). But the fact of the matter is that this is only possible for people who are willing to spend time familiarising themselves with the specific code, technologies, and practices that we’re using on that project. These tend to be hard-code devs, and — here’s the point — they’re always experienced Open-Sourcers: this will never be the first time they’ve worked on such a project.
Which is where the little features come in.
The joy of small
We noticed this problem — that contributing code to our projects is simply not easy for us or for contributors. Importantly, it’s not just us: it’s Open Source everywhere. But we can’t simply dismiss the opportunity for contribution. We want to encourage coders to do this, because we believe that Open Source is intrinsically a good thing.
We do two things to make it easier to contribute:
- We identify small features that a coder can approach without a full understanding of the code and the problem domain;
- We help people (like you!) get started by opening up a laptop at our weekly meetups.
The first of these seems obvious now: when we add issues (an idea for a new feature, or maybe a bugfix) to our github repos, if we think they’re candidates for manageable, isolated work, we tag them with the label: Suitable for volunteers (like this).
Often these turn out to be “nice-to-haves” that one of our full-time devs can’t be pulled off more pressing problems to add just now. (Case in point: Andrew added a date-picker to the FixMyStreet admin stats page, and three of our own staff had stumbled upon and applauded the difference it had made within a week of it going live).
It means it’s much easier for you to get involved, because often it’s a little, isolated piece of code. And it’s much more manageable for us, because the change you’ll be submitting is also isolated.
So if you’re looking for something to tackle, pick one of those issues, and let us know (just to check nobody else has baggsied it already). Fork the repo, cut the code, write the tests, submit a pull request!
But wait — if that last paragraph made you gulp, here’s the second thing we do: meetups. Of course, this is less helpful if you can’t make it to London on Wednesdays, but the concept is sound. Put simply, if there is a barrier to entry to diving in, and if one-on-one time with a dev, and some pizza, is what it takes to overcome that, it’s time well spent for you to come and see us.
Not 100% confident with git? Not sure when
db/schema.sqlgets used or how we like to handle migrations? No problem: we’re happy to guide you.
If this has struck a chord with you — you’d love to be an Open Source contributor one day, and you think mySociety projects make the world a better place — perhaps you should take a poke in our repos, or come along to a meetup. Start small, but do start.
Oh, and Andrew and Andy — thanks guys 😉
Photo by Matt Katzenberger (CC)
For an organisation whose members normally work from home, we’ve been pretty sociable recently, with meet-ups, conferences, and our annual retreat. We’re glad to discover that we haven’t actually lost the ability to communicate face to face…
If you’d like to come and sample our sharply honed social skills for yourselves, there are a number of opportunities still to come.
Every Wednesday: London meet-ups
If you’re in London, do feel free to drop by and say hello, any Wednesday from 6:00 pm – 9:00 pm. We meet at the Mozilla London space – and there are often other interesting things going on too.
Meet-ups are not just for coders – they’re for anyone who would like to talk more about mySociety projects or the wider eDemocracy field. On October 30th, we’re tying in with the Open Government Partnership event; you’re welcome to attend then or any other week.
Fancy coming along? Add your name to our Lanyrd pages here.
[Above: Our meet-ups are not always this busy! On this night, we happened to coincide with a Mozfest planning event.. speaking of which, see below]
25th – 27th October: Mozfest
Mozfest in Ravensbourne, London, is Mozilla’s annual innovative open web event for ‘technologists and creators’.
The event kicks off on the night of Friday 25th with a Science Fair. We’ll be there, showing our wares – in this case, we’ll be hoping to meet many of the internatonal attendeees and let them know about our open source software. But if you’re not an international attendee, you should totally swing by and say hello too.
30th October: Edinburgh
The next non-London mySociety meet-up will be in Edinburgh – watch this blog, our Twitter stream and Facebook page for details of precisely where (it’ll be a nice, central pub that serves food… suggestions are welcome).
That’s in advance of our attendance at the Channel Shift conference – but you don’t have to be a council employee to drop in. Come and share a pint and have a chat, whichever aspect of our work interests you.
20th November: Online Information conference
mySociety’s Director Tom will be giving the keynote presentation at the Online Information conference, the theme of which is “adapting to disruptive technologies and creating value with people, platforms and information”. Feel free to grab Tom afterwards for a chat!
4th December: Manchester
As with Edinburgh, we’re pitching up in Manchester for a Channel Shift conference, and will be taking the opportunity to mingle with lovely locals the night before. Again, pub suggestions are more than welcome.
We hope to see you soon at one of these events. And, if you’re wondering what we look like, well, you’re in luck. At our recent retreat we took a photo of the entire team (plus a few guests). Here we are in all our glory – click to see a larger version, if you dare.
Many mySociety projects rely on a team of volunteers to keep them going. FixMyTransport, WhatDoTheyKnow and Pledgebank may look like very simple sites that run themselves, but the truth is that there’s a lot of human intervention going on behind the scenes, keeping the wheels oiled.
Our volunteer teams deal with masses of site admin, they discuss policies and future development, and they give advice to our users. They may also go and talk about our projects in the wider community, and this is what WhatDoTheyKnow volunteer, Richard Taylor, did recently when he addressed the Association of Chief Police Officers at the “Transparency in UK Policing” event.
Richard has written about his experience here; I am linking to it because, as well as giving a good introduction to WhatDoTheyKnow within a policing context, it also explains exactly what sort of work the WhatDoTheyKnow volunteers do routinely, and the kind of issues that are discussed within the team. It might just make you value our volunteers more, or it might pique your interest in becoming one yourself.
If that latter applies, you can find out more about volunteering for WhatDoTheyKnow here, or about the ways you can help across all mySociety projects here. But either way, I encourage you to go and read Richard’s post.
Photo by Aaron van Dorn (CC)
Alaveteli (the software that runs WhatDoTheyKnow) is capable of being translated into any language, and we’ve finally switched on the ability to use the website in Welsh today. Many apologies for the long wait as this has been on our to-do list for well over 2 years…
As you can see, we don’t yet have a complete Welsh translation, and it’s just a start: we’ve done the help pages, and around 6% of the rest. To take a look at what’s been done, just click the “Cymraeg” link at the top of any page.
We’d love it if you could help us get to 100% by adding translations (or correcting any mistakes we’ve made!) at Transifex. You can read more about working with translations for Alaveteli, here and here, or just get in touch if you need a helping hand getting started or have any further questions.
And finally, a massive thank you & diolch to the translators who have already helped us get this far!