One goal of our Climate Action Plans Explorer is to make it easier for good ideas around cutting carbon to be shared and replicated between local areas. For this to happen, the service should be good at helping people in one area identify other areas that are dealing with similar situations or problems.
Currently the climate plans website shows the physical neighbours on a council’s page, but there’s every chance that councils are geographically close while being very different in other ways. We have been exploring an approach that identifies which authorities have similar causes of emissions, with the goal that this leads to better discovery of common approaches to reducing those emissions.
The idea of automatically grouping councils using data is not a new one. The CIPFA nearest neighbours dataset suggests a set of councils that are similar to an input authority (based on “41 metrics using a wide range of socio-economic indicators”). However, this dataset is not open, only covers councils in England rather than across the UK, and is not directly focused on the emissions problem.
This blog post explores our experiment in using the BEIS dataset of carbon dioxide emissions to identify councils with similar emissions profiles. A demo of this approach can be found here (it may take a minute to load).
Using the ‘subset’ dataset in the BEIS data (which excludes emissions local authorities cannot influence), we calculated the per person emissions in each local authority for the five groupings of emissions (Industry, Commercial, Domestic, Public Sector and Transport). We calculated the ‘distance’ between all local authorities based on how far they differed within each of these five areas. For each authority, we can now identify which other authorities have the most similar profile of emissions.
We also wanted to use this data to tell more of a story about why authorities are and are not similar. We’ve done this in two ways.
The first is converting the difficult to parse ‘emissions per type per person’ number into relative deciles, where all authorities are ranked from highest to lowest and assigned a decile from one to ten (where ten is the highest level of emissions). This makes it easy to see at a glance how a council’s emissions relate to other authorities. For instance, the following table shows the emissions deciles for Leeds City Council. This shows a relatively high set of emissions for the Commercial and Public Sector, while being just below average for Industry, Domestic and Transport.
Emissions type Decile for Leeds City Council Industry Emissions Decile 4 Commercial Emissions Decile 7 Domestic Emissions Decile 4 Public Sector Emissions Decile 9 Transport Emissions Decile 4
The second story-telling approach is to put easy-to-understand labels on groups of councils to make the similarities more obvious. We’ve used k-means clustering to try and identify groups of councils that are more similar to each other than to other groups of councils. Given the way that the data is arranged, there seemed to be a sweet spot at six and nine clusters, and as an experiment we looked at what the six clusters looked like.
How ‘Urban Mainstream’ Industrial emissions differ from other local authorities using a raincloud plot.
Using tools demonstrated in this jupyter notebook, we looked at the features of the six clusters and grouped these into three “Mainstream” clusters (which were generally similar to each other but with some difference in features), and three “Outlier” clusters, which tended to be smaller, and much further outside the mainstream. Reviewing the properties of these labels, these were relabelled into six categories that at a glance gets the broad feature of an area across.
Label Description Authority count Lower Tier Land Area % Lower Tier Population % Urban – Mainstream Below average commercial/industry/transport/ domestic emissions. High density. 165 14% 45% Rural – Mainstream Above average industry/transport/domestic emissions. Low density. 122 44% 26% Urban – Commercial Above average commercial/public sector, below average domestic/transport. High density. 66 4% 22% Rural – Industrial Above average transport/ industry/ domestic emissions. Low density. 43 37% 7% Urban – High Commercial Very high commercial/public sector emissions. 7 1% 2% High Domestic Counties Very above average domestic and transport emissions in county councils. 2 – –
This data is not tightly clustered, and the number of clusters could be expanded or contracted, but six seemed to hit a good spot before there were more clusters that only had a small number of authorities. The map below shows how these clusters are spread across the country. This map uses an exploded cartogram approach, where authorities with larger populations appear bigger. The authority is then positioned broadly close to their original position (so the blank space has no meaning).
Joining these different approaches allowed us to build a demo where for any local authority, you can get a short description of the emissions profile and cluster, and identify councils that are similar. This demo can be explored here (it may take a minute to load). The description for Croydon looks like this:
This is not the only possible way of crunching the numbers.
For example, the first thing we did was adjust the emissions data to be per person. This helps simplify comparison between areas of different sizes, but how many people are in an area is information that is relevant in helping councils find similar councils.
The BEIS data also breaks down those five categories by more variables (which might better separate agriculture from other kinds of industry for instance): an alternative approach could make more use of these.
We are considering using multiple different measures to help councils explore similar areas. This could include situation features like flood risk, deprivation scores and EPC data on household energy efficiency, but could also include, for example, that some councils are more politically similar to each other, and may find it easier to transfer ideas.
The datasets and processing steps are available on GitHub.
Image: Max Böttinger
Under the Freedom of Information Act, you have the right to ask public authorities for information. If they hold the information you’ve requested, in most cases they must release it.
‘If they hold it’ is a key point: of course, if the body doesn’t have the data you’re asking for, they can’t provide it, and accordingly, as permitted by Section 1(1) of the Act, they may issue an ‘Information not held’ response.
For the person requesting the information, this can feel like a blow, but we suggest stopping and thinking… is it surprising, newsworthy even, that the public body didn’t hold the information expected? Sometimes, it’s as interesting to know that an authority doesn’t collect or store a certain category of information as it would be to obtain it.
Often, though, an ‘information not held’ response may simply be a sign that you should re-request the information from a different body. Given the complexity of government it’s not surprising requests are sometimes misdirected to bodies that don’t actually hold the information requested. Fortunately for request-makers, in such cases Section 16 of the Act requires public bodies to provide advice and assistance and point requesters in the right direction.
Let’s take a look at a few recent examples where the fact that information isn’t held is at best surprising and at worst a matter of potential concern.
An NHS Trust doesn’t have data on hospital ward deep cleans
This request to Wirral University Teaching Hospital asked how many times the wards in Arrowe Park Hospital (one of this NHS Trust’s locations) had been deep cleaned in the months March to May 2020.
The response states that the Trust does not hold this information:
“Wirral University Teaching Hospital (WUTH) does not electronically collate this data, ward cleanliness activity is not measured or audited in this specific manor [sic].”
We think this is interesting information in itself, but the Trust could also have gone into more detail about why such data isn’t collected: is it that deep cleaning doesn’t happen, or is it just not noted? Looking more closely at the wording of the response, perhaps it is recorded, but on paper rather than digitally.
The request-maker didn’t actually specify that they wanted electronic data, so it is a little surprising that this has been assumed.
A curious citizen could issue a follow-up request to find out more, or to ask for copies of paper records if they do exist. Equally, if they felt it worth further probing, they might draw this response to the attention to the local Healthwatch, or their local councillors who could look more deeply into the matter.
DEFRA doesn’t have data on badger cull zone boundaries
A citizen requested maps from DEFRA to show the boundaries of ‘badger cull zones’ from last year and the proposed ones for 2021. As this was a request for environmental information, it was handled under the EIR.
DEFRA’s response quotes the exception at regulation 12(4)(a) of the EIR “which relates to information which is not held at the time when an applicant’s request is received”.
As required by both the FOI and EIR Acts, DEFRA points the user toward the body that it believes will have such information, Natural England, giving them a good idea of what to do next in their pursuit of this data.
In cases like these, where a request is repeated to another authority, we recommend the addition of annotations, linking each WhatDoTheyKnow request page to the other. Anyone can add an annotation, and it helps people discovering the information through, say, a search engine, to follow the request.
The DWP doesn’t have data on frozen pensions
This request seeks information from the Department of Work and Pensions about frozen state pensions, the UK practice of not uprating pensions according to the ‘triple lock’ system if the recipient lives in certain countries abroad.
It would appear that the request-maker is wondering whether it would actually cost the authority less not to administer such freezes.
The DWP state that they don’t hold information on the number of staff working specifically in this area:
“As of March 2021, the DWP has over 90,000 employees spread across different professions (e.g. policy, legal, finance, and operational delivery).
Many employees deal with a range of issues and do not exclusively focus on one aspect of the DWP’s work. Accordingly, we have no recorded information on how many employees are ‘required to deal with Frozen British Pensions’.”
They did not provide similar detail on the other questions posed by the request-maker, who has now requested an internal review, a recourse when you believe your request has been handled wrongly.
“I cannot believe that the DWP does not know the answers to my questions”, says the user in this request for a review. They might have been more precise about where the response has fallen short, but we’ll see whether this avenue is successful, presumably within the 20 working day limit advised by the ICO for internal reviews.
DCMS doesn’t have data on differing COVID rules for football spectators
A request-maker wonders why the rules around watching football differed depending on where the match is taking place — apparently grassroots football matches were being denied spectators because they were held on private grounds, while matches on public grounds could welcome a crowd:
“Please provide evidence / reasoning on what the difference to the threat of Covid there is between private and public football pitches.”
The Department for Digital, Culture, Media and Sport responds that it:
“does not have information within scope of your request. This is because DCMS did not make the decisions on whether spectators may attend sporting events, but followed advice given by the Cabinet Office. This would have been a Cabinet Office decision based on evidence that they were privy to.”
The response then provides an email address for that authority — though we are surprised that the Cabinet Office would not have consulted DCMS or at least provided them with their rationale for this policy.
Have you had a notable ‘information not held’ refusal, or can you spot one in responses recently classified “not held” on WhatDoTheyKnow? If so, let us know, and we might cover it in a future post.
Image: Nicholas Bartos
We were glad to see this recent tweet from Andy Mabbett:
— Andy Mabbett (@pigsonthewing) May 5, 2020
Andy has imported the IDs of every authority listed on our FOI site WhatDoTheyKnow into Mix’n’match, a tool for helping to link a dataset with existing Wikidata entities. Once a match has been made, the URL of the body’s WhatDoTheyKnow page is available as one of its identifiers (specifically, P8167).
This means that anyone running a project that utilises Wikidata will have the option to include WhatDoTheyKnow data in their site or app.
Andy says, “Wikidata acts as a hub for all sorts of databases and identifier systems. For example, it can be the only way of linking (programmatically, in the linked data sense) an MP’s official parliamentary record to their IMDb entry. I do a lot of work making that happen. As a regular and satisfied user of WhatDoTheyKnow, it appealed to me to add that site’s 24.5K listings of UK public bodies to the mix.”
The best-known site relying on Wikidata is of course Wikipedia, so in theory it would now be feasible, say, to include a template that automatically pulled the relevant WhatDoTheyKnow link into Wikipedia articles about authorities, or to build a browser extension that provided those links when the user visited such articles.
It would also be possible for us to pull information back the other way, so for example we might consider importing the first paragraph of a Wikipedia page for a body and using it within the introduction, as a way of providing context.
The matching of WhatDoTheyKnow authorities confirms which Wikidata URI (Uniform Resource Identifier) relates to each, meaning that these can now be used in “sameAs” metadata headers, scehma.org markup, etc. We think this might have a beneficial effect on the way search engines treat our pages in the future — something we’ll be keeping an eye on to check if that’s true.
Additionally, this works as a nice proof of concept that we can potentially recommend to other Alaveteli sites around the world, given that the Wikidata project is, of course, international.
But first, the bodies need to be checked with the Mix’n’match tool. At the time of writing, 1,302 bodies have been resolved, and can be seen here. Anyone is welcome to help by confirming more matches: just log in with a Wikimedia account.
Thanks to Andy for this initiative — it’s great to see the potential of our data being widened in one fell swoop.
There has already been a mutual benefit to this linking. WhatDoTheyKnow volunteer Matt has been able to use examples of failed matches to find cases where our database needed to be brought up to date with name changes. At the same time, Andy says it has helped him and his fellow Wikidata volunteers to create new items about councils and other bodies that were in WhatDoTheyKnow but not Wikidata.
Richard, also one of WhatDoTheyKnow’s volunteer team, says, “I’ve often thought there’s a lot of overlap between what we do on WhatDoTheyKnow and what Wikipedia volunteers are doing — we’re both maintaining lists of public bodies — so any tools for closer collaboration are great.”
Image: Carl Nenzen Loven
We’ve added a new functionality to the Alaveteli Pro codebase, allowing you to download a zip file containing all correspondence and attachments from a batch, and a spreadsheet (csv) to show the progress status of every request.
Alaveteli Pro is our tool for professional users of Freedom of Information. If you’re UK-based, you’re probably most familiar with our local iteration WhatDoTheyKnow Pro — but don’t worry: when we talk about improvements to Alaveteli Pro, you can be sure they’re also part of the WhatDoTheyKnow toolkit.
How to export
You’ll find these tools at the foot of the batch container in the requests list.
Why data exports?
Of course, we like to think Alaveteli Pro is a useful tool in its own right: there’s a lot you can do within the Pro interface, and it was built specifically to help you keep track of all your FOI activity in one place.
But sometimes users want to use external tools – either because they’re just more familiar with them, or because they want to do something beyond the functionality we offer.
Now there’s a simple way to get data out of Alaveteli, allowing you to analyse it with the tools of your choice, or perhaps send a progress report to a supervisor or editor.
It’s part of a programme of work to support cross border journalism between European organisations, supported by Adessium Foundation, allowing us to refine and improve the codebase for the benefit of all Pro users.
The technical bit
Those with a bit of coding knowledge may be interested to hear how we approached the zip download functionality. mySociety developer Graeme explains:
“With batch requests potentially going to as many as 500 different authorities, each request can receive several responses and attachments in return.
“All these emails and files mean that compiling the zip for download could be a lengthy job and would normally cause the request to time out. So for this new feature we’re utilising file streaming to send chunks of the zip as they become available.
“This means that the zip starts downloading immediately and you don’t have to sit watching and wondering whether anything is happening – you can see more and more data being transmitted.”
We hope you find this new feature useful. Please do let us know how you’re using it and any feedback you may have.
Image: Startup Stock Photos
If you’re looking for a quick and simple thing you can do from home to support meaningful action on climate change, help us make a list of councils’ Climate Action plans.
In the past 18 months, there’s been a spate of climate emergency declarations from local councils, in which they recognise the seriousness of the climate situation and commit to taking action. 65% of District, County, Unitary & Metropolitan Councils and eight Combined Authorities/City Regions have now declared a climate emergency. Many of these declarations commit to a date for getting to net zero, ranging from 2025 to 2050.
These declarations, and the commitment from central government to reach net zero by 2050, represent much needed progress. Commitments are good. But what we really need to address the climate emergency, both at a national and local level, are concrete plans.
As councils develop their plans for addressing the crisis, many individuals and groups need to be able to easily access, discuss and contribute to them to make sure they’re ambitious and high quality.
Councils can also learn and draw encouragement from each other’s efforts. At the moment, we think that’s harder than it should be.
Lots of people who want to take action on the climate locally are having to do the same work of finding their council’s plan, or finding out where they are in developing it, or finding other plans to compare it to. There’s no central place to find all the Climate Action plans that have been developed, or to track the process of developing them (or not!)
The climateemergency.uk site has been collecting those climate action plans they can find, but we think we can help them get a fuller picture, and create a resource that will help us all — and we’d like your help!
In the spirit of ‘start where you are’, we’ve made an open spreadsheet for collecting council climate action plans, and kicked it off with the ones from climateemergency.uk, to see if we can help improve what’s available. At the very least we’ll maintain this as a simple open resource, and share it wherever we think it might be useful. If you have thoughts about people who ought to know it exists, to use it or contribute to it, please do share them in the comments or drop us an email.
The key piece of information we want to collect at this point for each council is the URL where their Climate Action Plan can be found. But we’ve added some extra columns for anyone who wants to start looking at the details.
So, if you have five minutes, please have a look for a council’s Climate Action Plan and add it to the sheet.
If this works out well and seems useful, we’ve got some ideas about how to extend it and start to turn it into a more detailed and useful dataset or service. For example, tracking how the plans develop over time, how councils make progress against them, or breaking them down into a more detailed and comparative dataset — there seem to be key questions that would be useful to answer, for example around things like whether the plans only address emissions under councils’ direct control, or whether they’re focused on the area as a whole. So if you’d like to partner with us or support us to turn these ideas into reality, we’d love to hear from you! Email us at firstname.lastname@example.org.
You may remember our recent post on the surveillance techniques in use by police forces, as investigated by the campaign group Privacy International.
Several of you tweeted or commented that you were concerned to read of these new technologies. Well, here’s a way that you can get involved in finding out more.
Sounds fluffy? The reality is a bit more chilling. Cloud extraction technology allows the police to gain access from a citizen’s mobile phone to cloud based services such as their email, browser activity, and social media. So, if you are stopped and your phone is examined then handed back, surveillance might not stop there. Even after the phone is returned, using this tech police can monitor your online activity on an ongoing basis, seeing what you search for, trawling through your social media posts, and even accessing your location data.
Whether or not you’ve ever been detained by the police, you might like to know whether this sort of surveillance is in action in your own local neighbourhood. And that’s where FOI comes in.
To make everything as easy as possible for you, Privacy International have used pre-filled FOI requests* and provided the wording you should include. You can also see which forces have already been contacted, so as not to waste time making duplicate requests. Here’s where to get started.
Camilla Graham Wood, a Legal Officer at Privacy International, is clear about the benefits WhatDoTheyKnow has brought to their campaigning: “Using WhatDoTheyKnow we have created a way for members of the public to quickly and easily contact their local police force and ask them about intrusive surveillance tech. We were able to embed this on our own website and to pre-fill certain boxes as well as adding a tag so we can follow the progress of the campaign.
“Engaging the public in this way shows the level of public interest in policing technologies and introduces those who might not have used Freedom of Information request before to this valuable transparency tool”.
*If you’re running a campaign and you’d like to know how to set up something similar, take a look at this blog post where Gemma explained it all, back in 2016.
Image: Gilles Lambert
When you woke up this morning to check the election results, you may have visited TheyWorkForYou.
And you’d have found it bang up to date, thanks to the new MP data that was added through the night, as the election results came in. More than a fifth of you have a new MP, and whether you voted for them or not we know you’ll want to keep them accountable.
We’ve just now added one final MP — for St Ives, since weather conditions prevented ballot boxes coming over from the Isles of Scilly earlier.
We’ll be helping you hold all MPs, new and returning, to account over the next few years, as we publish their debates and votes, expenses, interests and contact details.
We make it as simple as possible for everyone to understand what’s going on in Parliament, and how you can play a part in your own democracy.
Right now, you can get a headstart:
If you’re a developer, researcher or just a good old data junkie, you might additionally like to:
- Use the API in the knowledge that it’s delivering the current MPs
- Download a spreadsheet of the current MPs
Now we need you to help us
We’re determined to carry on providing these services, but we still need your help to do so.
There are seven days left to run on our crowdfunder. Thanks to the generosity of hundreds of donors, we’ve already raised almost £10,000, for which we are enormously grateful.
But we still need to raise another £15,000 so that we can continue providing these services, as well as adding new features that will improve the site and make Parliament easier for everyone to follow.
Image: ©UK Parliament/Jessica Taylor/ Stephen Pike (CC by-nc/2.0)
Generally speaking, the sites just work. Sure, there are a bunch of tasks we’re managing on a daily basis behind the scenes, but none of those need bother you, the user. To employ a tired old metaphor, the sites glide swanlike, while under the water there’s some busy paddling to ensure that the latest debates, votes and representatives’ contact details are all present and correct.
During an election, though, that paddling becomes a bit more visible, and some services may be interrupted.
You want to contact your MP? Here’s the thing: officially, you don’t have one at the moment.
Parliament has dissolved. The representatives formerly known as MPs are no longer allowed to refer to themselves as such, and their parliamentary email addresses have been withdrawn.
So when you visit WriteToThem, you’ll see this message where we normally provide the link for writing to your MP:
Note that you can still use WriteToThem to contact all your other representatives, from local councillors to MSPs, Assembly members, MEPs, etc — provided that your issue is relevant to them (you’ll see a short list of the types of issue each representative deals with, on the site).
If you’ve got something to say about the current political situation or a matter that you’d like your MP to vote on, though, you’ll just have to wait. Even if your former MP is standing for re-election, they’re most likely dedicating a lot of their time to canvassing, and of course they won’t be taking any issues into the debating chamber just now because Parliament is not in session.
Where it becomes a little more tricky is if you have a constituency issue you want an MP to help with. Perhaps consider if it’s something your local councillor/s may be able to help with instead — it’s always worth asking them, anyway. If not, and if it’s an urgent matter, it may be worth calling your former MP’s office, as some (especially those standing for re-election) will still be running a bare bones service.
If your issue is not urgent, then wait until a couple of weeks after the election. In particular, if you find yourself with a brand new MP they’ll be finding their feet, setting up staff and office equipment, etc.
You’ll see the word ‘former’ used a lot, if you visit TheyWorkForYou over the next few weeks. For example, the homepage generally has a prominent link to direct you towards your own MP’s page. These days, it looks like this:
And if you do click through to any MP’s page, you’ll see that they now have this below their name:
On the page where we list all MPs, you’ll see this factually accurate message at the top:
If you want a list of who the MPs were, it’s still there, you just have to click the link.
And then there’s one more thing: of course, as there are no debates taking place in Parliament, we’re not sending out Westminster email alerts (you’ll still get those from Scottish Parliament and the London Assembly, though).
When will everything be back to normal?
Our friends at Democracy Club collate the election results as they come in, producing data that we can then import. Thanks to them we’re generally able to update TheyWorkForYou pretty much in real time. So, when you wake up in the morning you’ll hopefully be able to:
- Check who your MP is;
- If it’s someone new, sign up for alerts so you get an email when they speak.
For a little while, of course, new MPs will have very little content on their pages: you’ll see a message to say that data will start to appear once they’ve done a bit more.
WriteToThem takes a little longer to get back up to speed: that’s because we need to import all the MPs’ email addresses, and these can take a while to come through. If we’re using an official parliamentary email address, experience shows that they may not even be set up by Parliament for a short while.
So please be patient — as we mentioned earlier, it’s probably best to wait a couple of weeks before contacting your brand new MP in any case.
While mySociety sites are fully operational in the periods between elections, there are other organisations who swing into action and do their best work during this time.
So here are a few things you can do, thanks to those other orgs, while you wait for mySociety’s democracy services to return to normal.
- Visit WhoCanIVoteFor and WhereDoIVote from Democracy Club to discover who your local candidates for the General Election are, what they stand for, and where to find your nearest polling station.
- Upload scans of the political mailouts coming through your door to ElectionLeaflets, and help build a permanent archive of promises that elected representatives can be held to account for further down the line.
- Get the Who Targets Me extension on your browser to see clearly who is behind the political ads you’re being served on Facebook.
And finally: if you have questions about the whole electoral process, read the beginner’s guide to the UK General Elections we put together in 2017. While the names and dates have changed since then, the facts are still the same.
It’s a more common problem than you might think: given a list of postcodes, how can you match them to the administrative and electoral areas, such as wards or constituencies, that they sit within?
MapIt’s data mapping tool gives a quick, easy and cheap solution: just upload your spreadsheet of postcodes, tell it which type of area you want them matched to, and the data is returned to you — complete with a new column containing the information you need.
The tool can match your postcodes to every type of data that MapIt offers in its API, including council areas, Westminster constituencies, parish wards and even NHS Clinical Commissioning Groups (CCGs).
If that doesn’t sound like something you can imagine being useful, let’s look at a few hypothetical use cases (and if you have an actual case that you’d like to tell us about, please do let us know — we’re always keen to hear how our tools are being used).
Organisations, charities and campaigns sometimes need to match postcodes to administrative areas
Membership organisations, charities and campaigns usually collect the addresses of supporters, but don’t commonly ask them who their MP is (even if they did ask, most people in the UK don’t actually know the name of their MP).
But when a campaign asks followers to contact their MPs, it’s helpful to be able to suggest an angle based on whether the MP is known to be sympathetic to their cause, or not — indeed, there’s arguably no point in contacting MPs who are already known to be firmly on board.
So: input a spreadsheet of supporters’ postcodes, and get them matched to the associated Westminster constituencies.
For more advanced usages, organisations might match the MapIt tool’s output of postcodes with other datasets to discover the answers to questions like:
- Which members in a disability group have fewest GPs in their area, and might be finding it difficult to get help for their condition?
- Which supporters of a transport charity live in regions less served by public transport, and would be likely to take action to campaign for improved bus and train services?
- Which people affiliated to an ecological organisation live in predominantly rural areas and could help with a wildlife count?
Researchers sometimes need to match postcodes to administrative areas
Researchers often need to correlate people, institutions or locations with the boundaries they fall within.
They might have a list of postcodes for, say, underperforming schools, and want to find out whether they are clustered within authorities that have similar characteristics, like cuts to their funding or an administration that has a political majority one way or the other.
Teamed with other datasets, MapIt can help towards answering important questions like the number of people each CCG serves, how unemployment rates vary in different European regions, or average house prices within parliamentary constituencies.
Journalists sometimes need to match postcodes to administrative areas
Investigative or data journalists may obtain long spreadsheets full of postcodes in the course of their work, perhaps as a result of having submitted Freedom of Information requests to one or more authorities.
Perhaps they have the address of every university in the country, and there’s an election coming up — during the summer holidays. Knowing that students will mostly be in their home constituencies, they might be able to make informed predictions about how votes in the university towns will be affected.
Or let’s say that a journalist has gathered, from local councils, an address for every library scheduled to close. This could be compared with another dataset — perhaps literacy or crime rates — to draw conclusions over what impact the closures would have.
Part of a wider service
The one-off data mapping tool is just one service from mySociety’s MapIt, which is best known for its API.
This provides an ongoing service, typically for those running websites that ask users to input geographical points such as postcodes or lat/longs, and return tailored results depending on the boundaries those points fall within.
MapIt powers most mySociety sites, for example:
- When you drop a pin on the map while using FixMyStreet, MapIt provides the site with the administrative boundaries it falls within, so that the site can then match your report with the authority responsible for fixing it.
- When you type your postcode into WriteToThem, Mapit gives the site the information it needs to to display a list of every representative, from local councillor up to MEP, who represents your area.
- If you search for your postcode on TheyWorkForYou, MapIt tells the site what your Westminster constituency is and the site matches that to your MP. You can then be taken to their page with a record of how they have voted and everything they’ve said in Parliament.
Give it a try
Image: Thor Alvis
Freedom of Information forms the basis of many a campaign that seeks to expose hidden facts, or stories which should be in the public eye.
We spoke to Jen Persson, Director of defenddigitalme, about that organisation’s tireless campaign to get to the truth on the collection, handling and re-use of schoolchildren’s personal data in England.
What emerged was a timeline of requests and responses — sometimes hard fought for — which when pieced together reveal secrecy, bad practice and some outright falsehoods from the authorities to whom we entrust our children’s data. Perhaps most striking of the findings was the sharing of data with the Home Office in support of their Hostile Environment policy.
As Jen describes defenddigitalme’s campaign, “It began with trying to understand how my daughter’s personal information is used by the Department for Education; it became a campaign to get the use of 23 million records made safe”.
It’s a long tale, but definitely worth the read.
December 2012: consultations and changes
The story begins here, although it would still be a couple of years before Jen became aware of the issues around children’s data, “despite — or perhaps because of — having three young children in school at the time”.
Why did no one at all seem to know where millions of children’s personal data was being sent out to, or why, or for how long?
As Jen explains, “During the Christmas holidays, the Department for Education (DfE) announced a consultation about changing data laws on how nationally stored school pupil records could be used, proposing that individual pupil-level records could be given away to third parties, including commercial companies, journalists, charities, and researchers. Campaigners raised alarm bells, pointing out that the personal data would be highly identifying, sensitive, and insecure — but the changes went through nonetheless.”
2014: discovering the power of FOI
Jen came across that change in law for herself when reading about a later, similar data issue in the press: there were plans to also make available medical records from GP practices. This prompted her first foray into FOI, “to answer some of the questions I had about the plans, which weren’t being published”.
I feel strongly that if I am going to ask for information which has a cost in time and capacity in the public sector, then it should mean the answers become available to everyone.
And that first step got her thinking:
“At around the same time I asked the DfE a simple question, albeit through a Subject Access rather than FOI request: What personal data do you hold about my own child?
“My Subject Access request was refused. The Department for Education would not tell me what data they held about my children, and as importantly, could not tell me who they had given it to.
“There was nothing at all in the public domain about this database the DfE held, beyond what the campaigners in 2012 had exposed. It wasn’t even clear how big it was. How was it governed? Who decided where data could be sent out to and why? How was it audited and what were the accountability mechanisms? And why was the DfE refusing its lawful obligations to tell me what they held about my daughter, let me correct errors, and know where it had gone? Why did no one at all seem to know where millions of children’s personal data was being sent out to, or why, or for how long?
“Prior to all this, I’d never even heard of Freedom of Information. But I knew that there was something wrong and unjust about commercial companies and journalists being able to access more personal data about our children than we could ourselves.
I worded some questions badly. I learned how to write them better. And I’m still learning.
“I needed to understand how the database operated in order to challenge it. I needed to be able to offer an evidenced and alternative view of what could be better, and why. FOI was the only way to start to obtain information that was in the public interest.
“I believed it should be published in the public domain. WhatDoTheyKnow is brilliant at that. I feel strongly that if I am going to ask for information which has a cost in time and capacity in the public sector, then it should mean the answers become available to everyone.”
“I tried to ask for information I knew existed or should exist, that would support the reasons for the changes we needed in data handling. I worded some questions badly. I learned how to write them better. And I’m still learning.”
2015: sharing children’s personal data with newspapers
That was just the beginning: at the time of writing, Jen has made over 80 FOI requests in public via WhatDoTheyKnow.com .
Through FOI, defenddigitalme has discovered who has had access to the data about millions of individuals, and under what precepts, finding such astonishing rationales as: “The Daily Telegraph requested pupil-level data and so suppression was not applicable.” The publication “wished to factor in the different types of pupil” attending different schools.
Jen explains: “This covered information on pupil characteristics related to prior attainment: gender, ethnic group, language group, free school meal eligibility (often used as a proxy for poverty indicators) and SEN (Special Educational Needs and disability) status, which were deemed by the Department to be appropriate as these are seen as important factors in levels of pupil attainment.”
But with such granular detail, anonymity would be lost and the DfE were relying only on “cast iron assurances” that the Telegraph would not use the data to identify individuals.
2016: sharing children’s nationality data with the Home Office
In a Written Question put by Caroline Lucas in Parliament in July 2016, the Minister for Education was asked whether the Home Office would access this newly collected nationality data. He stated: “the data will be collected solely for the Department’s internal use […]. There are currently no plans to share the data with other government departments unless we are legally required to do so.”
But on the contrary: defenddigitalme’s subsequent requests would disclose that there was already a data sharing agreement to hand over data on nationality to the Home Office, for the purposes of immigration enforcement and to support the Hostile Environment policy.
Jen says: “As part of our ongoing questions about the types of users of the school census data, we’d asked whether the Home Office or police were recipients of pupil data, because it wasn’t recorded in the public registry of data recipients.
The Home Office had requested data about dependents of parents or guardians suspected of being in the country without leave to remain.
“In August 2016, a FOI response did confirm that the Home Office was indeed accessing national pupil data; but to get to the full extent of the issue, we had to ask follow up questions. They had said that “since April 2012, the Home Office has submitted 20 requests for information to the National Pupil Database. Of these 18 were granted and 2 were refused as the NPD did not contain the information requested.”
“But the reply did not indicate how many people each request was for. And sure enough, when we asked for the detail, we found the requests were for hundreds of people at a time. Only later again, did we get told that each request could be for a maximum agreed 1,500 individuals, a policy set out in an agreement between the Departments which had started in 2015, in secret.
“In the October afternoon of the very same day as the school census was collecting nationality data for the first time, this response confirmed that the Home Office had access to previously collected school census pupil data including name, home and school address: “The nature of all requests from the Police and the Home Office is to search for specific individuals in the National Pupil Database and to return the latest address and/or school information held where a match is found in the NPD.”
The Home Office had requested data about dependents of parents or guardians suspected of being in the country without leave to remain.
“In December 2016, after much intervention by MPs, including leaked letters, and FOI requests by both us and — we later learned — by journalists at Schools Week, the government published the data sharing agreement that they had in place and that was being used”.It had been amended in October 2016 to remove the line on nationality data, and allowed the data to be matched with Home Office information. It had also been planned to deprioritise the children of those without leave to remain when allocating school places, shocking opposition MPs who described the plan variously as “a grubby little idea” and, simply, “disgusting”.
Other campaigners joined the efforts as facts started to come into the public domain. A coalition of charities and child rights advocates formed under the umbrella organisation of Against Borders for Children, and Liberty would go on to support them in preparing a judicial review. ABC organised a successful public boycott, and parents and teachers supplied samples of forms that schools were using, some asking for only non-white British pupils to provide information.
Overall, nationality was not returned for more than a quarter of pupils.
2017: behind the policy making
Through further requests defenddigitalme learned that the highly controversial decision to collect nationality and country of birth from children in schools — which came into effect from the autumn of 2016 — had been made in 2015. Furthermore, it had been signed off by a little known board which, crucially, had been kept in the dark.
“I’d been told by attendees of the Star Chamber Scrutiny Board meeting that they had not been informed that the Home Office was already getting access to pupil data when they were asked to sign off the new nationality data collection, and they were not told that this new data would be passed on for Home Office purposes, either. That matters in my opinion, because law-making relies on accountability to ensure that decisions are just. It can’t be built on lies”, says Jen.
The process of getting hold of the minutes from that significant meeting took a year.
Jen says, “We went all the way through the appeals process, from the first Internal Review, then a complaint to the Information Commissioner. The ICO had issued a Decision Notice that meant the DfE should provide the information, but when they still refused the next step was the Information Rights First Tier Tribunal.
“Two weeks before the court hearing due, the DfE eventually withdrew its appeal and provided some of the information in November 2017. Volunteers helped us with preparation of the paperwork, including folk from the Campaign for Freedom for Information. It was important that the ICO’s decision was respected.”
2018: raised awareness
In April last year, the Department confirmed that Nationality and Country of Birth must no longer be collected for school census purposes.
However, Jen says, “Children’s data, collected for the purposes of education, are still being shared monthly for the purposes of the Hostile Environment. There’s a verbal promise that the nationality data won’t be passed over, but since the government’s recent introduction of the Immigration Bill 2018 and immigration exemption in the Data Protection Act, I have little trust in the department’s ability in the face of Home Office pressure, to be able to keep those promises.
“Disappointingly”, says Jen, “the government has decided instead of respecting human rights to data protection and privacy on this, to create new laws to work around them.
The direction of travel for change to manage data for good, is the right one.
“It’s wrong to misuse data collected for one purpose and on one legal basis entrusted for children’s education, for something punitive. We need children in education, it’s in their best interests and those of our wider society. Everyone needs to be able to trust the system.
“That’s why we support Against Borders for Children’s call to delete the nationality data.
“A positive overall outcome, however”, she continues, “is that in May 2018, the Department for Education put the sharing of all pupil level data on hold while they moved towards a new Secure Access model, based on the so-called ‘5-Safes’. The intention is distribute access to data with third parties, not distribute the data itself. The Department resumed data sharing in September but with new policies on data governance, working hard to make pupil data safer and meet ‘user needs’. The direction of travel for change to manage data for good, is the right one.”
2019: Defenddigitalme continues to campaign
Defenddigitalme has come a long way, but they won’t stop campaigning yet.
People working with FOI is really important, even and perhaps especially when it doesn’t make the press, but provides better facts, knowledge, and understanding.
Jen says, “Raw data is still distributed to third parties, and Subject Access, where I started, is still a real challenge.
“The Department is handing out sensitive data, but can’t easily let you see all of it, or make corrections, or tell you which bodies for sure it was given to. Still, that shouldn’t put people off asking about their own or their child’s record, or opting out of the use of their individual record for over 14s and adult learners, and demand respect for their rights, and better policy and practice. The biggest change needed is that people should be told where their data goes, who uses it for how long, and why.
“Access to how government functions and the freedom of the press to be able to reveal and report on that is vital to keep the checks and balances on systems we cannot see. We rely on a strong civil service to work in the best interests of the country and all its people and uphold human rights and the rule of law, regardless of the colour of government or their own beliefs. People working with FOI is really important, even and perhaps especially when it doesn’t make the press, but provides better facts, knowledge, and understanding.
“FOI can bring about greater transparency and accountability of policy and decision making. It’s then up to all of us to decide how to use that information, and act on it if the public are being misled, if decisions are unjust, or policy and practice that are hidden will be harmful to the public, not only those deciding what the public interest is.
“WhatDoTheyKnow is a really useful tool in that. Long may it flourish.”