About six million people a year visit mySociety’s Freedom of Information website WhatDoTheyKnow.com; there are well over 100,000 registered users, and over 385,000 requests have been made via the service.
Of course, it’s fantastic that WhatDoTheyKnow is so well used, but the growth and popularity of the site brings its own challenges, not least the day-to-day admin that keeps the site running.
Many aspects of the site’s operation are run by volunteers, supported by mySociety’s staff and trustees — and due to the site’s success we’re looking to expand the volunteer team.
What does volunteering involve?
The work is pretty varied, but there are some frequent and recurring tasks:
Dealing appropriately with requests to remove material from the website
This is one interesting challenge which arises fairly often. Sometimes these requests are from public bodies who’ve released information they didn’t mean to; and they can also come from individuals and companies who are named in correspondence on the site.
These decisions are not always as black and white as you might expect. Some recent examples where we had to carefully consider the balance on both sides were:
- Material which Transport for London were concerned could be used to steal a tube train. We considered: was this a genuine risk? Was our publication really increasing the risk? Was the information already available elsewhere? What was the potential value of publishing the information to tube staff and their representatives, travellers and the wider public?
- Sainsbury’s were concerned that published material didn’t reflect their corporate policy on “workfare”. That may have been the case, but we asked ourselves whether that made it inappropriate to continue to publish the information that had been released. Additionally, where did the public interest lie? What legal risks were there arising from continued publication?
Responding promptly and accordingly to accidental releases
Thankfully, the frequency with which public bodies accidentally release personal information in bulk via Freedom of Information responses is decreasing, but the WhatDoTheyKnow team still have to act promptly when this does occur.
We often help users on both sides of the FOI process. For requesters, we can answer questions about FOI and how to use it, and we also work with the staff of public bodies who are at the receiving end of requests.
And all the rest
There’s always more that can be done to promote the service, draw attention to interesting correspondence on the site, and lobby for improvements to our access to information laws.
The wider team at mySociety help people around the world to establish and run their own online Freedom of Information services; and new features are being added to the UK site to make it more attractive to professional users such as journalists and campaign groups. Volunteers have the opportunity to get involved in these activities, helping steer the direction of new projects, based on their frontline experience of being a site administrator.
Keeping the database of thousands of public bodies up to date is another challenge, especially given the frequency of reorganisations in the UK’s public sector.
We work primarily by email, with regular video conferencing meetings, and occasionally meet up in person.
As a volunteer, you can decide how much time you put in, and what aspects of running the service you decide to take part in — but ideally we’re looking for people who can spare at least an hour or two, a couple of days a week.
We understand that people’s external commitments vary over time, and of course, there’s a flexible approach if a team member needs to step away for a stretch now and then.
What makes a good WhatDoTheyKnow volunteer?
There’s one characteristic that all the WhatDoTheyKnow volunteers have in common: a belief in the value of Freedom of Information, or, more widely, the expectation of transparency and accountability from the bodies which citizens fund.
As for practical skills: perhaps you’ve been involved in moderating discussions on the web, or have experience with access to information, defamation, or data protection law. Or perhaps you have, or would like to gain, experience dealing with “customers” by email.
Primarily we’re looking for people capable of making good judgements, and who can communicate clearly online.
Before joining the team, new volunteers will have to agree to follow our policies covering subjects such as security and data protection. That said, part of the role may be, if desired, taking a part in developing and refining these, and other, policies as the service grows and changes.
How to apply
If helping us run WhatDoTheyKnow sounds like the kind of thing you’d be interested in doing, then please do apply to join us.
We only have the capacity to bring on and train a few volunteers at a time, and it is important that those chosen to help administer the service are trustworthy and committed to its policies, direction and non-partisan stance. For these reasons, we are recruiting volunteers via a formal application process.
To apply please write to us before the 20th of March 2017, introducing yourself, and letting us know about any relevant interests or experience you have.
What do we offer in return?
As a volunteer, the main reward comes from the satisfaction of assisting users, making good decisions, and helping run what is fast becoming a key part of the country’s journalistic and democratic infrastructure.
Volunteers may be invited to mySociety events and meet-ups, providing a chance to take part in discussions about the future direction of the service and the organisation’s activities more generally. There have been a number of conferences held, where those running Freedom of Information sites around the world have got together to share experiences: one or more volunteers may be invited to join in, with travel expenses paid.
Other ways to help out
If volunteering to join the WhatDoTheyKnow team isn’t for you, perhaps there’s something on mySociety’s Get Involved page that is — or you could:
Image: MarkBuckawicki [CC0], via Wikimedia Commons
We were shocked and saddened to learn that Peter Williams, one of our WhatDoTheyKnow volunteers passed away earlier this week.
Peter only joined WhatDoTheyKnow as a volunteer in March of this year, however in that short time he had become a valued member of the team.
No stranger to Freedom of Information, he had been using the rights provided for in the FOI Act since shortly after it came into force here in the UK, and had been a user of WhatDoTheyKnow since 2012.
Education and senior executive pay and benefits were some of his particular areas of interest, and Peter was researching the reasons why some public bodies sometimes fail to respond to requests.
As a consequence, Peter had helpfully been collecting information on specialist colleges and schools, and proposing additions and edits to the site. Following the same route that has led to several of our keenest users becoming volunteer administrators, he was invited to join the team so that he could make the changes he was proposing himself.
By all accounts he made a strong impression during his short time on the team, both with his fellow volunteers and across the mySociety team.
“Many aspects of the site’s operation, including dealing with correspondence from users, considering requests to remove material from the site, and discussing our policies and the future development of the service, benefited from Peter’s input”, says one.
Others say: “He was a valued volunteer and a great person”; and “he was a funny, thoughtful and committed guy”.
We were saddened to learn of death of someone who shared our beliefs in the value of making information held by public bodies accessible, and who shared our passion for activism.
All are feeling the loss of a colleague who approached his role with such enthusiasm and diligence, and our team will be the poorer for his absence. On behalf of mySociety, our Trustees and the WhatDoTheyKnow volunteers our thoughts are with Peter’s family and friends.
As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
This is a problem we have been warning about for some time. Islington Council were fined £70,000 for a similar incident in 2012. In light of this fresh incident we again urge all public authorities to take care when preparing data for release.
As with the Islington incident, the information was in parts of an Excel spreadsheet that were not immediately visible. It was automatically published on 14th November when Hackney Council sent it in response to a Freedom of Information request, as part of the normal operation of the WhatDoTheyKnow website. All requests sent via the website make it clear that this will happen.
This particular breach involved a new kind of hidden information we hadn’t seen before – the released spreadsheet had previously been linked to another spreadsheet containing the private information, and the private information had been cached in the “Named Range” data in the released spreadsheet.
Although it was not straightforward to access the information directly using Excel, it was directly visible using other Windows programs such as Notepad. It had also been indexed by Google and some of it was displayed in their search previews.
The breach was first hit upon by one of the data subjects searching for their own name. When they contacted us on 25th November to ask about this, one of our volunteers, Richard, realised what had happened. He immediately hid the information from public view and notified the council.
We did not receive any substantive response from the council and therefore contacted them again on 3rd December. The council had investigated the original report but not understood the problem, and were in fact preparing to send a new copy of the information to the WhatDoTheyKnow site, which would have caused the breach to be repeated.
We reiterated what we had found and advised them to consult with IT experts within their organisation. The next day, 4th December, we sent them a further notification of what had happened, copying the Information Commissioner’s Office (ICO). As far as we are aware, this was the first time the ICO was informed of the breach.
From our point of view it is very disappointing that these incidents are still happening. Freedom of Information requests made via WhatDoTheyKnow are a small fraction of all requests, so it is very likely that this kind of error happens many more times in private responses to requesters, without the public authority ever becoming aware.
Our earlier blog post has several tips for avoiding this problem. These tips include using CSV format to release spreadsheets, and checking that file sizes are consistent with the intended release. Either of these approaches would have averted this particular breach.
We would also urge the ICO to do as much as possible to educate authorities about this issue.
This month we released a new version of FixMyStreet. Amongst the new features, fixes, and thingamajigs were some small improvements added by two volunteers, Andrew and Andy.
Even though these are not core pieces of functionality — in fact, precisely because they are not — we want to draw your attention to why they were included, and why this is a Very Good Thing. And perhaps, if you’re a coder who wants to put something into an Open Source project but hasn’t quite found a way in, Andrew and Andy’s work will nudge you into becoming a contributor too.
One of the axioms of Open Source is that, because anyone can read the source code, in theory anyone can contribute to it. In practice, though, it’s not really as simple as that. Both ends of the “anyone can contribute” idea require effort:
- Before contributing to a project of any complexity (as we hope you want to do), there’s often a lot to learn, or figure out, before any work can even begin;
- Before accepting contributions to such a project (keen as we are to do so), there’s an overhead of testing, checking, and managing the incoming code.
The ugly real world
The basic issue here is that software is complex — no matter how well-written, tested and documented program code is, if the problem it’s solving is in the real world, it’s not going to be simple.
This is especially true of anything used by the public, because often you can only make things seem simple at the front (such as a clean web interface or “user journey”) by working hard behind the scenes with data structures and processes that handle the underlying complexity. It’s inevitably true of any projects which have been developed over time — programmers like to use the term “legacy code” to describe anything that wasn’t written then way they’d choose to write it now.
Often the problems that software is solving are not quite as obvious as they first appear. At mySociety we’ve got a wealth of experience and actual usage data that ultimately changes the way we build, and develop, our platforms. We understand the fields we work in well (technically, the nerds like to call these the “problem domain”), whether it’s governmental practice or civic user behaviour, and that’s often knowledge that’s not encapsulated anywhere in the program code.
Furthermore, any established platform must protect against the risk that new changes break old behaviour — something that regression testing is designed to catch. This is especially important on platforms like FixMyStreet or Alaveteli where the software is already running in multiple installations.
This is why we have a team of full-time, experienced, and (thanks to our rigorous recruitment process) talented programmers who can invest the time and effort to be familiar with all these things when they set to work coding.
But this builds up to an impediment: sensitivity to any of these issues is enough to make anyone think twice about simply forking our code and starting to hack on it for us.
How it sometimes works
In practice, then, how does anything get contributed? How come it doesn’t all get written by our own coders?
The answer is, of course, we do work with major contributors outside our own team (have a look at the activity on our github repos to see them) — but it always requires a period of support and on-line discussion both before and during the process. There’s also the business of testing, and sometimes politely pushing back on, pull requests (which is how code contributions are submitted). But the fact of the matter is that this is only possible for people who are willing to spend time familiarising themselves with the specific code, technologies, and practices that we’re using on that project. These tend to be hard-code devs, and — here’s the point — they’re always experienced Open-Sourcers: this will never be the first time they’ve worked on such a project.
Which is where the little features come in.
The joy of small
We noticed this problem — that contributing code to our projects is simply not easy for us or for contributors. Importantly, it’s not just us: it’s Open Source everywhere. But we can’t simply dismiss the opportunity for contribution. We want to encourage coders to do this, because we believe that Open Source is intrinsically a good thing.
We do two things to make it easier to contribute:
- We identify small features that a coder can approach without a full understanding of the code and the problem domain;
- We help people (like you!) get started by opening up a laptop at our weekly meetups.
The first of these seems obvious now: when we add issues (an idea for a new feature, or maybe a bugfix) to our github repos, if we think they’re candidates for manageable, isolated work, we tag them with the label: Suitable for volunteers (like this).
Often these turn out to be “nice-to-haves” that one of our full-time devs can’t be pulled off more pressing problems to add just now. (Case in point: Andrew added a date-picker to the FixMyStreet admin stats page, and three of our own staff had stumbled upon and applauded the difference it had made within a week of it going live).
It means it’s much easier for you to get involved, because often it’s a little, isolated piece of code. And it’s much more manageable for us, because the change you’ll be submitting is also isolated.
So if you’re looking for something to tackle, pick one of those issues, and let us know (just to check nobody else has baggsied it already). Fork the repo, cut the code, write the tests, submit a pull request!
But wait — if that last paragraph made you gulp, here’s the second thing we do: meetups. Of course, this is less helpful if you can’t make it to London on Wednesdays, but the concept is sound. Put simply, if there is a barrier to entry to diving in, and if one-on-one time with a dev, and some pizza, is what it takes to overcome that, it’s time well spent for you to come and see us.
Not 100% confident with git? Not sure when
db/schema.sqlgets used or how we like to handle migrations? No problem: we’re happy to guide you.
If this has struck a chord with you — you’d love to be an Open Source contributor one day, and you think mySociety projects make the world a better place — perhaps you should take a poke in our repos, or come along to a meetup. Start small, but do start.
Oh, and Andrew and Andy — thanks guys 😉
Photo by Matt Katzenberger (CC)
For an organisation whose members normally work from home, we’ve been pretty sociable recently, with meet-ups, conferences, and our annual retreat. We’re glad to discover that we haven’t actually lost the ability to communicate face to face…
If you’d like to come and sample our sharply honed social skills for yourselves, there are a number of opportunities still to come.
Every Wednesday: London meet-ups
If you’re in London, do feel free to drop by and say hello, any Wednesday from 6:00 pm – 9:00 pm. We meet at the Mozilla London space – and there are often other interesting things going on too.
Meet-ups are not just for coders – they’re for anyone who would like to talk more about mySociety projects or the wider eDemocracy field. On October 30th, we’re tying in with the Open Government Partnership event; you’re welcome to attend then or any other week.
Fancy coming along? Add your name to our Lanyrd pages here.
[Above: Our meet-ups are not always this busy! On this night, we happened to coincide with a Mozfest planning event.. speaking of which, see below]
25th – 27th October: Mozfest
Mozfest in Ravensbourne, London, is Mozilla’s annual innovative open web event for ‘technologists and creators’.
The event kicks off on the night of Friday 25th with a Science Fair. We’ll be there, showing our wares – in this case, we’ll be hoping to meet many of the internatonal attendeees and let them know about our open source software. But if you’re not an international attendee, you should totally swing by and say hello too.
30th October: Edinburgh
The next non-London mySociety meet-up will be in Edinburgh – watch this blog, our Twitter stream and Facebook page for details of precisely where (it’ll be a nice, central pub that serves food… suggestions are welcome).
That’s in advance of our attendance at the Channel Shift conference – but you don’t have to be a council employee to drop in. Come and share a pint and have a chat, whichever aspect of our work interests you.
20th November: Online Information conference
mySociety’s Director Tom will be giving the keynote presentation at the Online Information conference, the theme of which is “adapting to disruptive technologies and creating value with people, platforms and information”. Feel free to grab Tom afterwards for a chat!
4th December: Manchester
As with Edinburgh, we’re pitching up in Manchester for a Channel Shift conference, and will be taking the opportunity to mingle with lovely locals the night before. Again, pub suggestions are more than welcome.
We hope to see you soon at one of these events. And, if you’re wondering what we look like, well, you’re in luck. At our recent retreat we took a photo of the entire team (plus a few guests). Here we are in all our glory – click to see a larger version, if you dare.
Many mySociety projects rely on a team of volunteers to keep them going. FixMyTransport, WhatDoTheyKnow and Pledgebank may look like very simple sites that run themselves, but the truth is that there’s a lot of human intervention going on behind the scenes, keeping the wheels oiled.
Our volunteer teams deal with masses of site admin, they discuss policies and future development, and they give advice to our users. They may also go and talk about our projects in the wider community, and this is what WhatDoTheyKnow volunteer, Richard Taylor, did recently when he addressed the Association of Chief Police Officers at the “Transparency in UK Policing” event.
Richard has written about his experience here; I am linking to it because, as well as giving a good introduction to WhatDoTheyKnow within a policing context, it also explains exactly what sort of work the WhatDoTheyKnow volunteers do routinely, and the kind of issues that are discussed within the team. It might just make you value our volunteers more, or it might pique your interest in becoming one yourself.
If that latter applies, you can find out more about volunteering for WhatDoTheyKnow here, or about the ways you can help across all mySociety projects here. But either way, I encourage you to go and read Richard’s post.
Photo by Aaron van Dorn (CC)
Alaveteli (the software that runs WhatDoTheyKnow) is capable of being translated into any language, and we’ve finally switched on the ability to use the website in Welsh today. Many apologies for the long wait as this has been on our to-do list for well over 2 years…
As you can see, we don’t yet have a complete Welsh translation, and it’s just a start: we’ve done the help pages, and around 6% of the rest. To take a look at what’s been done, just click the “Cymraeg” link at the top of any page.
We’d love it if you could help us get to 100% by adding translations (or correcting any mistakes we’ve made!) at Transifex. You can read more about working with translations for Alaveteli, here and here, or just get in touch if you need a helping hand getting started or have any further questions.
And finally, a massive thank you & diolch to the translators who have already helped us get this far!
Today sees the official launch of FixMyStreet’s open source codebase as a proper tool that we hope people will want to deploy in cities and countries around the world. It is based on FixMyStreet.com which we believe is the most usable, most mature street problem reporting tool in the world, but which is only available to British users.
We’re shouting about this launch a bit because we need your help to make the service ever better. First, we need feedback from programmers about whether we’ve got the install process right – whether it’s as easy and clear as we want it to be. And for non-coders who want to get involved, we want to ask for help with the process of translating the site’s text into different languages.
Over the years there have been many copies of FixMyStreet set up in many countries, often using the site’s original name, but always written by developers from scratch. We’re delighted to have inspired people, but all too often the people trying to build copies have stumbled as they realise just how hard it is to build a tool like this with the polish that users expect. We think that people everywhere would be better off if they could have a local FixMyStreet that was really usable, and really connected to the right people.
So we’re very happy to be able to open up a codebase that has been extensively modified in the last year, to help users around the world manage easy, successful deployments. Steps we have taken include:
- Putting the translation text into Transifex, so that non-technical translators can get started whenever they feel like it
- Developing Amazon Machine Images so people who want to tinker can get started in the minimum possible time
- Rewriting the entire codebase in order to make it a less confusing installation
- Building a global version of our MapIt political boundaries web service, so you can get going without having to wrestle administrative data out of your government before you get started.
Plus with the help of the wonderful OpenStreetMap, you can get maps without licensing hassles too.
Calling it version 1.0 is our way of saying two things. First, that the tool still has a lot of evolution left to do, and a long way to go before it is as good as we want it to become. But more ambitiously, calling it 1.0 is also our way of saying that it’s no longer just a codebase dumped into Github. It’s a real open source project, which we plan to support, and which we hope will make a real difference in the lives of ordinary people. Check it out.
One of the key differences between the UK’s national parliament and its local governments is that Parliament produces a written record of what gets said – Hansard.
This practice – which has no actual legal power – still has a huge impact on successful functioning of Parliament. MPs share their own quotes, they quote things back to one-another, journalists cite questions and answers, and every day TheyWorkForYou sends tens of thousands of email alerts to people who want to know who said what yesterday in Parliament. Without freely available transcripts of Parliamentary debates, it is likely that Parliament would not be anything like as prominent an institution in British public life.
No Local Hansards
Councils, of course, are too poor to have transcribers, and so don’t produce transcripts. Plus, nobody wants to know what’s going on anyway. Those are the twin beliefs that ensure that verbatim transcripts are an exceptional rarity in the local government world.
At mySociety we think the time has come to actively challenge these beliefs. We are going to be building a set of technologies whose aim is to start making the production of written transcripts of local government meetings a normal practice.
We believe that being able to get sent some form of alert when a council meeting mentions your street is a gentle and psychologically realistic way of engaging regular people with the decisions being made in their local governments. We believe transcripts are worth producing because they show that local politics is actually carried out by humans.
The State of the Art Still Needs You
First, though – a reality check. No technology currently exists that can entirely remove human labour from the production of good quality transcripts of noisy, complicated public meetings. But technology is now at a point where it is possible to substantially collapse the energy and skills required to record, edit and publish transcripts of public meetings of all kinds.
We are planning to develop software that uses off-the-shelf voice recognition technologies to produce rough drafts of transcripts that can then be edited and published through a web browser. Our role will not be in working on the voice recognition itself, but rather on making the whole experience of setting out to record, transcribe and publish a speech or session as easy, fast and enjoyable as possible. And we will build tools to make browsing and sharing the data as nice as we know how. All this fits within our Components strategy.
But mySociety cannot ourselves go to all these meetings. And it appears exceptionally unlikely that councils will want to pay for official transcribers at this point in history. So what we’re asking today is for interest from individuals – inside or outside councils – willing to have a go at transcribing meetings as we develop the software.
It doesn’t have to be definitive to be valuable
Hansard is the record of pretty much everything that gets said in Parliament. This has led to the idea that if you don’t record everything said in every session, your project is a failure. But if Wikipedia has taught us anything, it is that starting small – producing little nuggets of value from the first day – is the right way to get started on hairy, ambitious projects. We’re not looking for people willing to give up their lives to transcribe endlessly and for free – we’re looking for people for whom having a transcript is useful to them anyway, people willing to transcribe at least partly out of self interest. We’re looking for these initial enthusiasts to start building up transcripts that slowly shift the idea of what ‘normal’ conduct in local government is.
Unlike Wikipedia we’re not really talking about a single mega database with community rules. Our current plans are to let you set up a database which you would own – just as you own your blog on Blogger or WordPress, perhaps with collaborators. Maybe you just want to record each annual address of the Lord Mayor – that’s fine. We just want to build something that suits many different people’s needs, and which lifts the veil on so much hidden decision making in this country.
Get in touch
The main purpose of this post is to tell people that mySociety is heading in this direction, and that we’d like you along for the ride. We won’t have a beta to play with for a good few months yet, but we are keen to hear from anyone who thinks they might be an early adopter, or who knows of other people who might want to be involved.
And we’re just as keen to hear from people inside councils as outside, although we know your hands are more tied. Wherever you sit – drop us a line and tell us what sort of use you might want to make of the new technology, and what sort of features you’d like to see. We’ll get back in touch when we’ve something to share.