When we started building WhatDoTheyKnow Pro, our toolkit for professional users of FOI, we knew that there was one feature which would be a game-changer for such users: the ability to send a request to multiple authorities at once.
In this blog post, we examine what we wanted the tool to do, how we are guarding against abuse, and finally we’ll give a step by step walkthrough of the interface.
Investigative news stories or in-depth pieces of research often require information from a multitude of different sources. By gathering statistics or information from multiple authorities, journalists, activists and researchers can build up a previously-unseen picture, for example of how widespread a particular problem is, or where there are inequities in medical provision across the country.
It’s something that many professional users of FOI are doing already, usually with the aid of their own homemade spreadsheets on which they keep track of requests made, dates by which replies should be expected, which bodies have responded, which need chasing, and of course the information held in the responses themselves.
The standard WhatDoTheyKnow website already provides several helpful features that you just don’t get with a DIY system: it has all the right email addresses for authorities, for example; it guides you through the FOI process; and it will send you an email reminder when the deadline for response arrives — even taking bank holidays into account.
But we knew that in order for our batch request feature to woo people away from their spreadsheets, it needed to do more than those homebuilt systems, some of which have been refined over several years and work well, even if a bit clunkily, for their owners.
Power and responsibility
One important consideration was uppermost in our minds when it came to batch requests: it costs authorities time and money to respond to each request, and of course that multiplies with batch requests. We are keen to promote responsible use of FOI, so we want to fold appropriate safeguards and guidance into whatever system we build.
As mentioned, with WhatDoTheyKnow Pro we’re focusing on features that are genuinely useful for professional users of FOI, but we also want to help those users make better, more focused requests — ones that are more likely to get useful responses and see the light of day as news stories. So it was important that, in making it simple to send multiple requests, we also help users find the most suitable authorities to send their requests to.
With that in mind, here are some balances we’ve put in place:
- Users are limited in how many batch requests they can send within any one month — so there’s no chance to go too wild.
- There’s a limit to the number of authorities that can be added to a single batch: we set this to be the number of local authorities in the UK, which is a logical sector to survey in this way.
- Before users do a batch mailout, we encourage the sending of an initial request that goes to just a few authorities. This safeguard can reveal where a request is flawed, so for example, if the data you get back is not what you need or in the wrong format, you don’t have to send to the full list all over again.
- We provide advice on cost limits to encourage succinct batch requests.
- Authorities have the facility to report a request which is unsuitable for review by our administrators.
- We’re rolling out the batch request functionality gradually to vetted WhatDoTheyKnow Pro users so that we can gradually learn how people use it in practice, and course-correct as necessary..
Testing and improvements
So far, the batch feature is only available to a select group of test users, who are giving us feedback on how they’re finding it. There’s certainly nothing like having your code being used by real people to help you see where improvements might be made!
That said, it’s been a very gratifying process. With the help of our test users, we’ve seen that the batch request functionality has the potential to be immensely helpful to professional users of FOI; even genuinely game-changing. We are certain that with the sending tools, we’ve created a service that really adds value for this sector.
We’re now in the next phase, and turning our attention to improving the functionality that helps users deal with incoming responses when they come in. This already exists in a basic form, and thanks to our testers, we’ve identified which improvements we need to make. We’re already working on incorporating them. But that is definitely material for the next update — for now, let’s take a look at just how the batch request function works.
Using batch request
There are three parts to making any request, whether you’re doing it yourself or using WhatDoTheyKnow Pro:
- Creating the request
- Managing the responses
- Analysing the results
The batch request functionality builds on our super-simple FOI workflow tools for WhatDoTheyKnow Pro, extending them to make larger investigations much easier.
Creating the Request
The first step is compiling a list of authorities to send the request to. From the compose screen, you might search on a keyword (for example, ‘hospital’, ‘Birmingham’, or ‘Birmingham hospitals’) and then add the authorities you’re interested in.
Each authority is added to a recipient list and WhatDoTheyKnow Pro creates a ‘mail merge’ setup. You’ll see how many authorities you’re writing to in the compose interface.
You can then draft your request. The special `Dear [Authority name]` salutation gets automatically replaced with each of the selected authorities when you send your batch.
Finally, before sending you can choose a privacy duration.
At this point you can either go straight ahead and send your request, or save the draft and come back to it later.
Once you’ve sent your batch request, you’re going to receive a lot of replies from authorities. This is where WhatDoTheyKnow Pro’s functionality really comes into its own, keeping all that clutter out of your email inbox.
Here’s what it looks like: the first thing you’ll see is a high-level progress bar showing you the overall progress of your batch. There are three main states that help you manage the requests in the batch:
- In progress (yellow): This means that there’s no action needed by you – you’re waiting on the authority to respond with an acknowledgement or the information you’ve requested.
- Action needed (red): When a request in the batch receives a response from the authority, you’ll need to check it out. We mark the response as “action needed” for you to review and decide what to do next.
- Complete (green): Once there’s no further action needed – either you’ve got the information you asked for, the authority didn’t have the information, or they’ve refused and you don’t want to challenge them – the request moves to the ‘complete’ state, so you know you don’t need to think about it until you come to analyse the data.
Clicking the title of the batch reveals the individual requests and their progress status. From there, you can click through, read the response and update the status.
Now you’ve got all your data, it’s time to compare the results from different authorities.
Sometimes authorities will reply in the main correspondence.
Other authorities respond with one or more attachments. You can view these inline or download them to your computer.
If you’re dealing with a batch sent to lots of authorities, sometimes it’s easier to just download everything. You can download a Zip file containing all the correspondence and attachments for each request via the “Actions” menu. From there you can pull out the attachments that contain the raw data and plug the numbers or answers in to your spreadsheet so that you can compare across authorities.
You can sign up to WhatDoTheyKnow Pro today and receive 1 month free with the voucher code BLOGMARCH18. Make some requests to try out the FOI workflow tools for professionals, and get in touch to request to join the waiting list for batch access.
If your FOI requests have made the news, let us know! Send us links to your published stories and we’ll throw in an extra month of WhatDoTheyKnow Pro for free. Your stories help us improve WhatDoTheyKnow Pro.
To help us mark WhatDoTheyKnow’s tenth anniversary, we asked volunteers, supporters and users to tell us which Freedom of Information requests from the site’s first ten years particularly stuck in their minds.
The result was an eclectic mix of stories that really show the breadth of how WhatDoTheyKnow has been used. They have very little in common — unless you count the imagination and tenacity of those using FOI to try to uncover significant information.
Doug Paulley, WhatDoTheyKnow volunteer
A exposé that helped bring in the living wage for carers
Doug is one of the team of volunteers who give up their time to keep WhatDoTheyKnow running, using their experience and knowledge of FOI to moderate the site, give users guidance, and help set policy. Doug is also an extremely active user of FOI, having used the act to uncover many examples of discrimination and malpractice over the years.
He highlights the story of a care home talking the talk, but very much failing to walk the walk when it came to paying its staff the living wage.
“The exposure brought about by FOI played a significant part in the campaign for Leonard Cheshire, care home operator with 2,100 residents, to significantly increase carers’ wages to (just short of) the voluntary living wage. Journalist Heather Mills covered the story in Private Eye.” Read the whole story here.
Owen Blacker, mySociety trustee
Missing historic information on Cold War targets
Owen co-founded FaxYourMP, the earliest version of mySociety’s WriteToThem, and has been an important part of the organisation ever since — he’s now one of our trustees and a non-executive director. He recalls the building and launch of WhatDoTheyKnow and indeed was one of its earliest registered users.
Owen particularly remembers a pass-the-parcel like series of FOI requests in which he was handed from one organisation to another:
“I went round in circles trying to find out some Cold War information that nobody claims to know any more. In 1980, the entire Civil Service, nationwide, ran a dry run of a Cold War nuclear attack on the United Kingdom, called Operation Square Leg. I’m slightly concerned that we spent a lot of money planning the civil contingencies of a Cold War attack — a sensible things to do, arguably — but no longer know where we were expecting to be hit or at what megatonnage.” Owen links to the requests from this blog post.
Will Perrin, Indigo Trust
Safer streets and better data handling
Will is not only a trustee at Indigo, supporting mySociety’s work with parliamentary monitoring organisations in sub-Saharan Africa, but he’s also a trustee of London’s King’s Cross Community Projects. Indeed Kings Cross — a locality in which Will has a personal stake, with a long record of community action — is the subject of two of the three FOI requests he singled out:
First was the Kings Cross Walkability audit which revealed just how hostile to pedestrians the area was back in 2008. At the time, Will wrote in his blog: “Crossing the road in Kings Cross is a nightmare and now we have an official report commissioned by TfL that sets it out in black and white.”
Today he recalls its impact: “This document underpinned the police taking a corporate manslaughter case against TfL to the Criminal Prosecution Service with regard to a cyclist’s death in 2011. The case did not proceed but was instrumental in changing TfL’s attitude to cyclists’ rights.
“Then this request revealed a massive overspend by Network Rail in refurbishing its own offices at Kings Cross”.
Finally, Will’s third choice of request had wider implications for the country as a whole:
“The National Police Chiefs’ Council revealing that there was no governance system in place for the Automatic Numberplate Recognition System (ANPR) and the existence of Met’s ‘Olympic Data Feed‘ led to a new governance system being instilled; some 2 billion records were deleted along with the introduction of a vastly reduced retention period.” Annotations at the foot of this request give a little more background.
Matthew Somerville, mySociety developer
A long-standing pillar of mySociety’s development team, Matthew wrote the core code behind many of mySociety’s most notable websites and tools, including FixMyStreet and TheyWorkForYou. He spends his working days coding for mySociety’s useful tools, and much of his free time coding his own useful tools, if his website is anything to go by. What was his most memorable FOI request?
“It was a request asking Royal Mail for information about all their postboxes, made by Tom Taylor. I had to write a crowd-sourcing tool to locate them, as the information provided included street name but no actual location; they then (from another FOI request a few years later) released the co-ordinates as well.”
The data is mapped here. Why is this request significant?
“I’m not sure it’s really significant, but I do get plenty of people telling me they’ve used the site, and it’s something Royal Mail never got around to providing (even though that was their reasoning for refusing to release it)…”
So there we are: a handful of the 458,219 requests that WhatDoTheyKnow has processed to date. There are so many stories around FOI requests: each of them represents someone’s burning question; many of them result in a response that’s important, or fascinating, or historic. And that’s what makes WhatDoTheyKnow so rewarding to work on.
We’ve talked a lot about our new service for journalists and other professional users of Freedom of Information — but it’s not always the professionals who uncover the news stories.
This week, we mark WhatDoTheyKnow’s tenth anniversary. As part of the celebrations, we thought we’d look back on the news stories that came about because of requests made through the site. Many of these began with an FOI request submitted by a user with no links to the press, and were picked up by news outlets because the response was of public interest.
From the restrictions on what names can be given to a baby in this country, to an accidental torpedo release, and via a geographically-accurate Tube map, it makes for fascinating reading. You can see them all here.
On 22 February 2008, we posted an announcement on this very blog: “the new mySociety Freedom of Information site is now live”.
More than 450,000 requests later, WhatDoTheyKnow is marking its tenth anniversary: as part of the celebrations, we’ve put together a timeline showing how WhatDoTheyKnow has intersected with the history of FOI in the UK since we first gained our right to information in 2005. If nothing else, you may enjoy looking at the site’s rather more primitive design back in its early days.
The past decade has seen legal challenges, contributions to Parliamentary inquiries, and the development of our code for use in other countries (26 and counting). It has proved that an ambitious project can be kept going thanks to the efforts of unpaid but skilled and dedicated volunteers.
Most of all, though, it’s seen you, the general public, submitting requests for information, and sharing the responses you receive. That was always the idea, and, it turns out, it’s a pretty sound one.
An article in the current Private Eye Magazine has drawn our attention to the use that disability campaigner John Slater is making of our Freedom of Information service WhatDoTheyKnow.com.
In December 2016, Mr Slater asked the Department for Work and Pensions (DWP) to release the monthly “management information reports” received from contractors ATOS and Capita in relation to their work assessing eligibility for Personal Independence Payment benefits.
Mr Slater has pursued his request for over a year, and wasn’t put off by an initial response which stated that the information requested wasn’t held, nor a subsequent response refusing to release the material citing the contractors’ “commercial interests”.
In December 2017, a year after Mr Slater made his request, the Information Commissioner ordered the DWP to release the material, stating “The Commissioner has not been satisfied that disclosing the withheld information would be likely to damage the commercial standing of ATOS and Capita”. The Information Commissioner dismissed the DWP’s concerns that the information requested could be “misinterpreted in ways that could lead to reputational damage to both the Department and the PIP Providers as well as prejudice the efficient conduct of public affairs”.
The Information Commissioner’s decision notice was highly critical of the way the DWP had handled the case, noting the use of “standard paragraphs” rather than a discussion of the public interest tailored to the material in question, and DWP failing to engage promptly with the Information Commissioner, thus causing further delay.
The DWP have not yet complied with the Information Commissioner’s decision; they have appealed and a tribunal hearing is scheduled for April 2018.
This request is far from the only one showing Mr Slater’s persistence in pursuing the release of information held by the Department for Work and Pensions.
A request for Project Assessment Review Reports for the Universal Credit Programme that Mr Slater made in April 2016 was initially accepted and the department said they were considering it. Mr Slater chased up the lack of a response in June, and again in August and September, but when, six months after his original request, Mr Slater chased them again in October they deemed his persistence to be vexatious and rejected the request.
That request has now been further rejected by the DWP, who say that the information “if released would, or would be likely to, prejudice the free and frank provision of advice or which would otherwise, or would be likely otherwise to, prejudice the effective conduct of public affairs.”
Mr Slater has referred that decision to the Information Commissioner too.
On the 5th of December 2017, Debbie Abrahams MP, the Shadow Secretary of State for Work and Pensions, deployed the Parliamentary procedure of a motion for “an humble Address be presented to Her Majesty” to seek the release of the documents to the Work and Pensions Committee. MPs agreed the motion unanimously.
The committee chair, Frank Field MP, has suggested that:
A couple of copies would be made. These copies will be kept securely and members would be invited to come to the Committee office to read them. No-one else, other than the committee members, will be invited to make this journey to our Committee office and members will not be able to make copies, or take notes, about the documents.
– so despite the decision by the House of Commons the public still might not get to see the material via that route.
Mr Slater has been in touch with us and told us he finds the service provided by WhatDoTheyKnow extremely helpful when submitting and managing FOI requests.
He said that the ease of submitting requests and built in workflow that keeps track of time, reminding users that a response should have been issued, is invaluable. He also likes that a single platform exists where information obtained by its users is made available for everyone, as that embodies the spirit of the Freedom of Information Act.
We created FixMyStreet Pro to help councils and city governments better manage inbound street reports and issues from their residents.
In the past few months we’ve rolled out the FixMyStreet Pro service to new customers including Bath & North East Somerset, Buckinghamshire and Rutland councils; each of whom are taking the opportunity to get rid of legacy software, simplify their operations and make use of a much simpler and intuitive way for their residents and staff to make and manage reports.
We’re now looking for input from councils to help us guide the next phase of our service development on FixMyStreet Pro.
Having spoken to dozens of councils we think we can help them save more money by extending FixMyStreet Pro to other areas like waste and environment services and we would like to explore how much development work that might entail.
Not just for streets
As FixMyStreet’s name would suggest our focus so far has been on handling issues related to highways like potholes, lighting and gullies (drains to me and you), but FixMyStreet Pro already handles reports for a whole range of issues beyond streets.
Typically council users of FixMyStreet Pro have around 13 to 15 different self-selected categories that they accept reports on – each of which can be directed to different teams or departments. Tree reports can be sent directly to the parks department, graffiti or abandoned cars can be passed along to the just the right team in street cleansing.
These ‘front end reports’ all have one thing in common: all we need to make the report is a location and description, plus a contact for the reporter, which could be as simple as an email address or phone number.
But once you get deeper into the glamorous world of bins and waste services for individual residents the situation gets a little more complicated.
Missed bin collections, requests for recycling bags, bulky waste collection – these all require the resident to be identified, the particular property to be checked with the UPRN (Unique Property Reference Number), and in some cases payments levied and collected.
FixMyStreet Pro doesn’t currently offer these additional waste services, although it doesn’t require a huge leap of imagination to see how we could add these adjacent features to the service, not least because we already do a lot of the pieces across our other commercial services.
Fortunately there has already been a lot of work done to define common standards, such as the Local Waste Service Standards Project from 2016 and more recent work by individual councils to apply some of this work – we also have a lot of our own research and experience to draw upon with numerous specific feature requests from our current local authority clients.
To make this happen we’d like to recruit at least two or three friendly councils available for interviews and possibly a workshop or two, to help us determine specific requirements and test out some of our early prototypes and hypotheses. From here we’d aim to develop these features into fully working aspects of FixMyStreet Pro over the summer.
If this is of interest to you, if you’re already grappling with this in your own council, or you’d just like to find out more, please get in touch with firstname.lastname@example.org and we can have a chat.
In the meantime you can always find out more about what FixMyStreet Pro can do on one of our regular Friday Webinars.
We’re in the process of conducting a discovery and prototyping exercise to understand how Hackney Council currently respond to FOI requests — and also Subject Access Requests (SAR) — ahead of the new General Data Protection Regulation (GDPR), coming into force in May 2018.
The aim is to explore how we can help members of the public find the information they are looking for when attempting to submit an FOI or SAR request and subsequently, when a request is complete, making it easier to publish the non-personal responses to requests through a searchable public disclosure log.
Information should be free
When someone makes a Freedom of Information request to a public body, we like to think that the information provided, often at a not insignificant cost, should be available freely to everyone, in public.
That’s the basis of our Alaveteli software which runs in at least 28 countries, and WhatDoTheyKnow in the UK which has grown to become a vast database approaching half a million FOI requests and responses over the last 10 years, from almost 19,000 public bodies.
From our own research we know that at least 15% of all FOI requests made in the UK pass through WhatDoTheyKnow, and that rises to over 30% of all requests to some central government departments. That still leaves somewhere over 70% of all requests that feasibly could be made available to the public.
What usually happens instead is that these individual requests remain hidden away in private mailboxes and probably won’t ever see the light of day.
Our FOI strategy
In response to this we’re on a mission to expand the share of FOI requests that are catalogued and released in public through WhatDoTheyKnow. This requires us to have an understanding of the nature and source of these other requests.
Broadly speaking around one third of these remaining requests are made by commercial businesses seeking contractual information from councils, NHS trusts and the like. About a fifth are from journalists researching for stories, another slice come from students or academics, and the remainder from individuals who are often making just a single request.
Our overall strategy is pretty simple: expand the scope of WhatDoTheyKnow where we can to capture more requests directly, and create new services to cater to specific user needs.
This thinking is what led to our service for journalists and campaigners, WhatDoTheyKnow Pro. (So far we’ve been reluctant to directly cater to FOI requests made by commercial businesses, although on balance it would be better if these requests did eventually make it into the public domain.)
Working with Hackney
Through WhatDoTheyKnow we’ve been pretty firmly focused on helping citizens make good FOI requests. Some readers may remember our previous forays into this area, with WhatDoTheyKnow for Councils (since retired). We found issues with that: specifically, the assumption of immediate publication in particular placed us in position as both poacher and gamekeeper, creating a conflict of interest we weren’t comfortable with.
However when we consider the full lifecycle of creating a response to an FOI request we still believe that we can use our experience of FOI to help public bodies support better drafting of initial requests and aid the management of responding to these requests.
Which brings us back to our new collaboration with Hackney.
More specifically, we are working with Hackney to explore how we can:
- help users better submit clear and valid requests
- integrate this request form with other sources of information (including with a disclosure log) to try and help users find what they need more quickly and conveniently
- integrate with case management services so that queries are answered quickly and information published openly wherever possible
- use information from previous requests to speed the allocation of a particular request to a specific council service
- support compliance with current legislation, and pre-empt forthcoming changes (GDPR).
We’re just at the beginning of this process but we’ll be blogging and sharing more updates over the next couple of months. We will also be speaking to other public sector bodies, councils in particular, about how they manage the process of responding to FOI requests, the challenges they face, and the opportunities this offers them to proactively release more publicly available data.
Hackney are a great partner to work with. As you might be already aware they have been very active in adopting user-centred, agile methods to develop new services, they are comfortable and vocal talking about their work in public (check out the HackIT blog here) and they are especially focused on bringing their staff along with them as they evolve their approach.
We’ve got a lot to learn from them, and hopefully they can benefit from some of our experience representing the needs of citizens.
If you’re responsible for managing FOI requests or data protection in your own public sector body and you’d like to follow this project in more detail — or if you’d like to participate in some of the discovery work — then please get in touch at email@example.com.
Update: See part 2 of this blog post here.
The Freedom of Information Code of Practice is a set of guidelines for the public authorities that are liable to respond to requests for information under the FOI Act. It advises these bodies on how to adhere to the law and what counts as best practice.
The Cabinet Office recently ran a consultation on proposed revisions to the Code of Practice. Since this Code directly relates to the activities of the website WhatDoTheyKnow, and the services it provides for our users, we put in a response, which you can view here.
The response was submitted under the joint names of WhatDoTheyKnow, our FOI codebase project Alaveteli, and mySociety itself, having been worked on by the WhatDoTheyKnow volunteer team, those working on the Alaveteli project, and mySociety’s researchers. Between them there is a substantial amount of experience and knowledge on FOI in the UK: much of our response is based on our experience in helping users to obtain information from public bodies.
Indeed, our response commented on points which we felt particularly affect our users; among other issues, we responded on:
- Timeliness of responses, including the introduction of time limits for internal review and public interest test extensions, and the importance of prompt responses to requests which inform current public debate.
- The use of pseudonyms by those making requests: what counts as a pseudonym; whether this should be one of the indications that can be used to label a request as vexatious, and whether authorities might, at their own discretion, process a request even if pseudonymous.
- Proactive publication, including the point that routine publishing of data may be more efficient and cheaper than responding to individual repeated requests. One suggestion is that every Freedom of Information request should prompt a consideration by the public body of whether the kind of information requested could practically be routinely published.
- The application of fees to a request: the desirability of pointing out that most FOI requests do not incur a charge and that the requester will never be charged without notice. People can be deterred by the prospect of fees, and bodies’ responses often contain worrying notices about them in their emails and on Freedom of Information web-pages, when in reality they are rarely applied.
- The means of communication: that requests made by email, unless the requester specifies otherwise, should be taken as a preference for a response by email; the ease of making FOI requests; and the ease of using data in the format provided in any response.
We replied on several other points too, including the status of the Code of Practice itself. It was issued in 2004, and has not been updated since, and in fact it’s not a document that we use regularly when we’re advising users or corresponding with public bodies about the application of Freedom of Information law.
The high quality guidance which we, and our users, do use on a day-to-day basis comes from the Information Commissioner, so we suggested the Government consider whether, and if so how, the Code of Practice could incorporate, or endorse that documentation.
One other important point is that the Code of Practice constitutes guidance rather than law, so any welcome shifts in policy that it endorses should ideally be reflected in the law too.
As a case in point, while the Freedom of Information Act has always covered information “held on behalf” of a public body, the proposed Code of Practice sought to make information held by contractors working for public bodies more accessible in practice: we welcome this but we do caution that issuing a new Code of Practice is not a substitute for amending the law, if that’s what’s required.
If you are interested, do read our submitted document in full.You may also like to see responses from the Campaign for Freedom of Information and the Open Government Network: as we three organisations’ submissions share several common themes (without our having consulted one another), we hope that there’s a good chance of the Government taking them into account.
Last week TheyWorkForYou received criticism from some MPs following requests from Emma Reynolds MP to include a note on her voting record that acknowledged time she spent off on parental leave. Our initial response was not sufficient and we’re sorry.
You can see our more nuanced follow up response on Twitter here:
The @mySociety service @TheyWorkForYou is currently receiving criticism from some MPs following requests from @EmmaReynoldsMP to include a note on her voting record that acknowledged time she spent off on parental leave.
— Mark Cridge (@markcridge) February 2, 2018
In summary we’ve committed to doing two things:
- We’ll speak to Parliament to see if a feed of absences could be made available to update the relevant section of TheyWorkForYou.com
- In the meantime we have made some changes so we can manually append a note on long-term absences due to paternal leave or ill-health on request from MPs offices
TheyWorkForYou.com allows citizens to understand how their MPs are voting on issues on their behalf. We’re able to do this because we take the official record of what’s been discussed in Parliament, Hansard, and we represent it in a more ordered form that gathers together all of the votes from a particular issue together in one place.
We can only work with what is provided via the official feeds from Parliament – we don’t actively try to gather additional data; we do however manually categorise each vote to allow them to be grouped together, but everything else is automated.
So whilst we are reliant on what we’re able to source from the official Parliament feed, there is an extent to which we are re-presenting the original data in a more transparent way. Arguably that will change how people see it. As such we want to ensure that we properly represent as true a picture as possible.
MPs, like anyone else, often have to spend time away from their jobs for extended periods, either on parental leave, or due to illness. As this is not reflected in voting records from Parliament and thus not displayed on TheyWorkForYou it can paint an inaccurate picture of an individual MP’s commitment – this is an issue that we have been aware of for a couple of years.
This is particularly relevant in the case of women who take time off after having a child; the current practice in Parliament is that there is no provision for parental leave or ability for MPs to appoint a proxy to vote on their behalf, and that’s the issue that MPs were debating on Thursday when Emma Reynolds made her observation about TheyWorkForYou.
It’s a situation we agree is unfair and in need of urgent reform. We completely support any initiatives to stamp out practices that disproportionately discriminate against women in Parliament.
The list of things that Parliament needs to address in order to improve its working conditions is long and and deep-seated, that’s not something that mySociety can fix – the only people who can do that are MPs and staff in Parliament themselves and we’ll continue to support these changes where we can.
We know that records of attendance aren’t kept for MPs and we blogged about it previously. We also know that this should in principle be possible as they do publish absences of leave for Lords.
So what we have at least done in the meantime is put in place a workaround for TheyWorkforYou.
If we can get the aforementioned list of absences from an official Parliament feed then we’ll look to include that alongside relevant sections of voting records on TheyWorkForYou. This would be our preferred solution.
If, as we suspect, this just is not available or may be some time in coming, then for the moment we will manually append a note to an MPs voting record on request from their office.
This will at least allow us take care of the most clear cut cases.
However as a solution this is far from ideal as it will mean that we are entirely reliant on being notified when an MP is away and when they return, which leaves a lot of opportunity for inaccurate record keeping.
With the best will in the world, we all know that human error can creep in to manual systems — of course we’d never suggest that an MP would lie about taking a leave of absence, that’d be ridiculous; but it would be easy for those about to go on maternity leave to forget to engage in a piece of admin that isn’t even required by Parliament. TheyWorkForYou is familiar to a lesser or greater extent by different MPs and they regard it as significant to a greater or lesser degree. This being so, we’d never be entirely confident that we were presenting a completely consistent and accurate record.
It puts us in a position where we are inadvertently going to be held responsible for keeping track of each MP’s attendance without the means of actually carrying out this role to an acceptable standard. It also raises the issue of where we draw the line – there are many reasons an MP may not be able to attend Parliament other than long-term illness or parental leave; having received such requests over the years from MPs, we can be sure this is going to come up again and again, so we suspect that this won’t be the end of the discussion.
That being said we agree that applying a short term patch to support the cause of parental leave in Parliament is a price worth paying and we’ll deal with the follow ups as best we can in the meantime.
@TheyWorkForYou – @EmmaReynoldsmp is saying that you refused to note that she was on maternity leave next to her voting record? Is that true? Is that something that is under active discussion? it seems to penalise female MPs as it stands https://t.co/7FfPFLY2EP
— Simon Burall (@sburall) February 1, 2018
TheyWorkForYou refusing to note that an MP was on maternity leave? Wait, that doesn’t sound like us…
TheyWorkForYou has one main aim: to make it extremely easy to see what’s going on in Parliament. To that end, we publish debates, voting records, and all sorts of details about MPs such as their job titles, expenses, and even which words they use most often.
Sometimes, interpreting all of these facts needs a little context. Case in point: when an MP is off on extended sickness or maternity leave, the number of speeches and votes they make will, of course, go down. There are many little exceptions like this, in fact: for example, my own MP was, for a while, a teller, meaning that he counted votes and was not normally allowed to vote himself. As you’d expect, this had quite an effect on his voting tally.
Now, the trouble with these exceptions is that they’re not easy to code. Most of TheyWorkForYou’s data actually comes from Parliament itself: they provide all the day’s debates, for example, as XML code, which our automated scripts pick up and publish out in the nice, readable format you see each morning on TheyWorkForYou. That’s how we’re able to publish such a large quantity of content on so many MPs: if TheyWorkForYou was compiled editorially, it’d require a far larger staff than we have.
So in fact, when Emma Reynolds got in touch to ask that we note her maternity leave on TheyWorkForYou, we didn’t refuse. Rather, we told her the truth: that it was a tricky issue that would require a manual bit of coding, but we’d add it to our development list and hopefully get to it.
And that’s what we did. The trouble is, our development list is long, and we’re constantly having to make decisions about what to prioritise. This ticket is now a few years old (Ms Reynolds was not the first to ask for a note on her record to explain special circumstances) and it hasn’t yet risen to the top of the list above bug fixes and other more urgent additions. TheyWorkForYou is currently unfunded, so of course, projects which have funding and expectations/deadlines attached to that funding take priority.
Note: In retrospect, we recognise that the advice below is not strictly relevant to this post. While we do very much need funding, and also do very much encourage anyone with coding skills to come and help out with our backlog, these two solutions would not alleviate the main obstacle to the issue above, which is that the required data isn’t output by Parliament. So, feel free to read on if you like, but with that in mind. mySociety CEO Mark Cridge put out a series on tweets on Friday which clarify our thinking; you can see those here.
Until we’re able to prioritise this piece of work (or any other that our users/MPs would like to see), there are a couple of solutions.
Ask for Parliament to add such information to their output
As mentioned above, most of TheyWorkForYou’s content is automated, so if there were a data source to show that an MP was on a leave of absence, we could easily pick it up and include it on their page. We’ve asked an MP’s office about this but they replied:
We are not aware of any official source of information about an MP taking leave of absence.
From our point of view, this would be far preferable to a manual solution, which would rely on MPs getting in touch themselves to let us know when exceptions applied. This would almost certainly lead to a situation where some did, and some didn’t, meaning the information could look more accurate than it really was. Many researchers use our outputs, so we wouldn’t want this to become the foundation of a study on MPs’ leaves of absence!
Be the change you want to see
OK, that’s a bit of a hippy-esque maxim, but in this case it’s quite apt.
A small job like this would not take very long or cost very much — the reason we haven’t yet managed it isn’t because it’s a massive piece of work, but because there are so many other pressing tasks.
An MP (or anyone) who wanted to see a new feature could help by making a donation. If you specify that it is for a particular addition to the site, we’ll get back to you to discuss how viable that is, and how we can make it happen.
Or do it yourself! Like most mySociety projects, TheyWorkForYou runs on Open Source code. That means that, if you have development skills, you are very welcome to fork the code and make a pull request for whatever improvements or additions you like. We’ll gratefully merge in any that fit with the site (have a chat with us first to make sure everyone’s on the same page).
If you’re not a developer (say, for example, you’re an MP), you could even contract one to do this for you.
We hope that’s cleared things up a bit. We’re not out to demonise MPs who take maternity leave, honest. And we’ve lodged an official request for a correction from the Times.
Image: Erik Lundqvist