Yesterday John Edwards, the government’s preferred candidate to be the next Information Commissioner, was questioned by the House of Commons’ DCMS committee.
During this session, Mr Edwards made several comments that raised alarm bells for us. Generally, he seemed more concerned with hypothetical abuse of the system by the public, rather than the demonstrable behaviour of the authorities that he would be charged with regulating. He also expressed support for the idea of charging requesters (contrary to the ICO’s current position and rejected by the 2016 review).
While it is not good that those were his instinctive reactions, the general sense is that he has not yet had a chance to become fully acquainted with Freedom of Information, despite it being a significant part of his future role. The bigger problem is that any potential candidate for the job (now or in the future) is likely to be in a similar position.
The UK Information Commissioner is two roles joined into one. In other jurisdictions these would separately be called the Privacy Commissioner (who oversees data protection) and Information Commissioner (who oversees access to information laws). When the role was created, there was more of a balance between the two aspects, but the growing significance of data protection in a digital world has meant that this side of the role has increased in focus and funding. Currently, less than 10% of the funding the ICO receives is for Freedom of Information related work.
While it is possible that some people exist who could handle both sides of the job equally well, recruiting with an emphasis on data protection minimises any focus on the FOI role.
John Edwards is currently the Privacy Commissioner in New Zealand, where there is an Ombudsman who separately deals with the equivalent of Freedom of Information complaints. While Elizabeth Denham, the current UK Information Commissioner had prior experience of a joint role in British Columbia, her role at the national level in Canada was similarly on the privacy side.
Here in the UK, this is likely to be a recurring issue for Freedom of Information. It has been positioned as a sideline to the ICO’s main priorities and something that new commissioners have to catch up on, rather than hitting the ground running with a clear sense of the problems inherent in the field.
In our recent report, we argued for a separate and independent FOI Commissioner, mirroring the arrangement in Scotland. We dug into the historical reasons why the roles were joined, and found that it had resulted from uncertainty about different offices applying different ideas about what ‘personal data’ is. The solidification and internationalisation of data protection rules (which also helps create an international recruiting pool for the role) makes this far less of a concern now, while the same trend has reduced the prominence of FOI in the joint role.
It is also our belief that the FOI Commissioner should be independent of the government. By this we mean that as a constitutional watchdog, the Commissioner should operate under the Officer of Parliament model, and receive oversight and funding from Parliament rather than the government. The current situation presents an obvious conflict of interest, where the government controls funding for its own regulator.
Freedom of Information has survived direct challenges in this country and across the world, but the danger (and default trajectory) for our Right To Know is not that it is abolished, but that it becomes less effective as time goes on. State activity moves to private contractors and the act is not extended to follow them. The lack of effective and well-funded enforcement from the ICO means increasingly open non-compliance, as public bodies discover loopholes that allow them to drag out responding to requests. The solutions to these problems are not difficult (and in many cases can borrow from processes in Scottish Freedom of Information legislation), but require the political will not just to defend the FOI act as it exists, but to argue for its role into the future.
As we approach the UK’s 20 year anniversary of Freedom of Information in 2025, we believe that change is needed to reclaim the ground that has been lost, and to strengthen Freedom of Information as a continuing practical and effective tool of government accountability. To this end, we are actively seeking to work with interested organisations and stakeholders to safeguard our right to information and place it on a more sustainable footing. In the run up to the next election, expected in 2024, we hope to see parties from across the political spectrum include provisions to defend and improve FOI operation in their manifestos. We also hope to explore opportunities for FOI to flourish in the devolved nations. To keep up with our work, sign up to our mailing list.
Hear more about our work
Subscribe to our newsletter.
One goal of our Climate Action Plans Explorer is to make it easier for good ideas around cutting carbon to be shared and replicated between local areas. For this to happen, the service should be good at helping people in one area identify other areas that are dealing with similar situations or problems.
Currently the climate plans website shows the physical neighbours on a council’s page, but there’s every chance that councils are geographically close while being very different in other ways. We have been exploring an approach that identifies which authorities have similar causes of emissions, with the goal that this leads to better discovery of common approaches to reducing those emissions.
The idea of automatically grouping councils using data is not a new one. The CIPFA nearest neighbours dataset suggests a set of councils that are similar to an input authority (based on “41 metrics using a wide range of socio-economic indicators”). However, this dataset is not open, only covers councils in England rather than across the UK, and is not directly focused on the emissions problem.
This blog post explores our experiment in using the BEIS dataset of carbon dioxide emissions to identify councils with similar emissions profiles. A demo of this approach can be found here (it may take a minute to load).
Using the ‘subset’ dataset in the BEIS data (which excludes emissions local authorities cannot influence), we calculated the per person emissions in each local authority for the five groupings of emissions (Industry, Commercial, Domestic, Public Sector and Transport). We calculated the ‘distance’ between all local authorities based on how far they differed within each of these five areas. For each authority, we can now identify which other authorities have the most similar profile of emissions.
We also wanted to use this data to tell more of a story about why authorities are and are not similar. We’ve done this in two ways.
The first is converting the difficult to parse ‘emissions per type per person’ number into relative deciles, where all authorities are ranked from highest to lowest and assigned a decile from one to ten (where ten is the highest level of emissions). This makes it easy to see at a glance how a council’s emissions relate to other authorities. For instance, the following table shows the emissions deciles for Leeds City Council. This shows a relatively high set of emissions for the Commercial and Public Sector, while being just below average for Industry, Domestic and Transport.
Emissions type Decile for Leeds City Council Industry Emissions Decile 4 Commercial Emissions Decile 7 Domestic Emissions Decile 4 Public Sector Emissions Decile 9 Transport Emissions Decile 4
The second story-telling approach is to put easy-to-understand labels on groups of councils to make the similarities more obvious. We’ve used k-means clustering to try and identify groups of councils that are more similar to each other than to other groups of councils. Given the way that the data is arranged, there seemed to be a sweet spot at six and nine clusters, and as an experiment we looked at what the six clusters looked like.
How ‘Urban Mainstream’ Industrial emissions differ from other local authorities using a raincloud plot.
Using tools demonstrated in this jupyter notebook, we looked at the features of the six clusters and grouped these into three “Mainstream” clusters (which were generally similar to each other but with some difference in features), and three “Outlier” clusters, which tended to be smaller, and much further outside the mainstream. Reviewing the properties of these labels, these were relabelled into six categories that at a glance gets the broad feature of an area across.
Label Description Authority count Lower Tier Land Area % Lower Tier Population % Urban – Mainstream Below average commercial/industry/transport/ domestic emissions. High density. 165 14% 45% Rural – Mainstream Above average industry/transport/domestic emissions. Low density. 122 44% 26% Urban – Commercial Above average commercial/public sector, below average domestic/transport. High density. 66 4% 22% Rural – Industrial Above average transport/ industry/ domestic emissions. Low density. 43 37% 7% Urban – High Commercial Very high commercial/public sector emissions. 7 1% 2% High Domestic Counties Very above average domestic and transport emissions in county councils. 2 – –
This data is not tightly clustered, and the number of clusters could be expanded or contracted, but six seemed to hit a good spot before there were more clusters that only had a small number of authorities. The map below shows how these clusters are spread across the country. This map uses an exploded cartogram approach, where authorities with larger populations appear bigger. The authority is then positioned broadly close to their original position (so the blank space has no meaning).
Joining these different approaches allowed us to build a demo where for any local authority, you can get a short description of the emissions profile and cluster, and identify councils that are similar. This demo can be explored here (it may take a minute to load). The description for Croydon looks like this:
This is not the only possible way of crunching the numbers.
For example, the first thing we did was adjust the emissions data to be per person. This helps simplify comparison between areas of different sizes, but how many people are in an area is information that is relevant in helping councils find similar councils.
The BEIS data also breaks down those five categories by more variables (which might better separate agriculture from other kinds of industry for instance): an alternative approach could make more use of these.
We are considering using multiple different measures to help councils explore similar areas. This could include situation features like flood risk, deprivation scores and EPC data on household energy efficiency, but could also include, for example, that some councils are more politically similar to each other, and may find it easier to transfer ideas.
The datasets and processing steps are available on GitHub.
Image: Max Böttinger
A couple of weeks ago, we announced that we’d added new functionality to WhatDoTheyKnow to help people challenge FOI refusals. In fact, this tool had been quietly rolled out at the end of May, giving us a little time to ensure everything was working before we shared it.
It tries to detect which exemptions were applied where information was withheld, and provides advice on next steps. This might involve making a revised request or asking for clarification, and the tool can also provide text fragments to use in a request for an internal review of the decision.
It is early days, but just in the last month users have sent 158 requests for internal review after seeing this advice. Most of these are still waiting for a reply, but around a third (16) of complete internal reviews have led to an improvement in the amount of information released. This is above the monthly average for the last year, but we need to wait for more data to determine if this will represent a sustained increase in the success of appeals.
As time goes on, we will be able to examine whether specific bits of advice or challenges to particular kinds of refusals are more likely to be successful than others. The goal is to give requesters the knowledge required to successfully challenge incorrect use of exemptions, and increase the percentage of internal reviews made through WhatDoTheyKnow that successfully lead to more information being released, either through increasing the quality and substance of appeals, or reducing appeals where exemptions have been applied correctly.
Process Requests Total number of times refusal advice acted on 415 Total number of times internal review submitted after acting on refusal advice 158 Internal review requested and awaiting response 108 Internal review completed, no improvement 34 Internal review completed, improvement over last status 16
Image: Waldemar Brandt
To understand the effectiveness of Freedom of Information laws requires good quality information about the volume of requests and how they are being processed. Picking up on previous work investigating FOI statistics for central and local government, we have built a new minisite that covers Cabinet Office statistics on Freedom of Information in UK central government and OSIC statistics on Freedom of Information in Scotland. This site is up-to-date with 2020 statistics, and can be viewed at: https://research.mysociety.org/sites/foi-monitor
We can do this because there are already sets of statistics gathered and publicly released for several groups of public authorities in the UK. But in most jurisdictions there is not good information about the overall working of the system, with data often limited to issues that are brought to the attention of the local equivalent of an Information Commissioner. In our recent report we made the case for more information gathering in the UK.
In the UK, there are two good partial sources of statistics about Freedom of Information requests. The Office of the Scottish Information Commissioner (OSIC) has a quarterly process where Scottish authorities deposit statistics about the SAR, FOI and EIR requests they have received, how they have been processed, and how exemptions and exceptions were applied. For the rest of the UK, there is far less data on the operation of the Freedom of Information Act. The Cabinet Office publishes a similar quarterly series covering FOI statistics covering requests made to a selection of central government ministries, departments and agencies.
In principle, FOI in the UK applies to far more public authorities than FOI in Scotland, but in terms of available information, the OSIC collects more information from more agencies. The Cabinet Office collects 76 sets of statistics from 40 agencies, while the OSIC collects 110 from 507. This means that the available picture of information on FOI is far more complete in Scotland. In both cases information is published and accessible as spreadsheets, making it possible to analyse differences between authorities and change over time.
However, this information is not always easy to understand or use. To generalise our previous analysis of Cabinet Office statistics, we have built a new minisite that also applies to information about Scottish authorities and in principle could be generalised for other jurisdictions.
This allows examination of trends in individual statistics (overall request volumes, how those requests were processed, or how individual exemptions were used) across sectors or individual authorities. These statistics are up to date with 2020 information.
We hope this will be a useful resource for all those interested in UK FOI statistics, and as a potential model for how FOI statistics collected can be made easier to access and understand.
The creation of this mini-site was partially supported by a Adesssium grant.
Header image: Photo by Tobias Messer on Unsplash
Exploring the history of expansions of FOI to private operators for our report on Reforming FOI led to trying to understand the history of how Network Rail became subject to FOI and EIR. This blog post explores how in very niche circumstances, the highest information court is the Office for National Statistics.
Network Rail is a body that owns and manages the infrastructure of most of the railway network in Great Britain (England, Scotland and Wales). As such, it holds information of potential public interest, but its status under Freedom of Information (FOI) and the Environmental Information Regulations (EIR) has been contested. A 2006 ICO decision concluded that Network Rail was not a public authority for the purposes of FOI (FER0071801 /FER0087031), but was for the purposes of EIR. This decision was overturned by the Information Tribunal (EA/2006/0061 EA2006/0062) in 2007, who held it was not a public authority for the purposes of EIR either. These decisions are now mostly irrelevant as in 2015 Network Rail became unambiguously a public body subject to FOI and EIR, and was added under s.5 to the FOIA schedule. The complication is that this change was not a political decision, but was effectively decided by the Office of National Statistics.
When implementing the 2010 European System of Accounts (ESA10) in 2014, the Office of National Statistics (ONS) retroactively reclassified Network Rail as a “central government controlled, nonmarket body classified as part of the Central Government sector”, and as having been so since 2004. This did not result from any new understanding of facts, but ESA10 included several new tests of government ‘control’ of an organisation: the ‘degree of financing’ and the degree of ‘risk exposure’. Previous tests (appointment of officers, provisions of enabling instruments, contractual agreements) had not concluded that Network Rail was government controlled but these new tests changed that picture.
The ‘degree of financing’ test required the government to ‘fully or close to fully’ fund the body for it to count as government controlled, this was not the case with Network Rail. The second test of risk exposure is sensitive to the question of who holds debt for the organisation. In this case, ONS argued that the debt was guaranteed by the Department for Transport, and there was an effective statutory obligation for the government to step in if Network Rail was to collapse. This by ESA10 criteria made Network Rail government controlled and part of the public sector. The reclassification was announced in 2014, and the new framework agreement between Network Rail and the Department of Transport agreed that Network Rail should be subject to Freedom of Information (1.15). While the body was added under section 5 of the FOI Act (rare), this order is in other respects similar to the frequent amendments made under section 4 as it reflected a change in ‘the public sector’ rather than including non-public sector bodies fulfilling a public function under the Act.
Given accounting change means that retrospectively Network Rail should be seen as part of the government accounts in 2007, does this have any impact on the underlying logic of the Information Tribunal decision that it was not subject to EIR at this time? In this case, the Information Tribunal did consider the degree of government control through the same test of board appointment and public funding that the ONS similarly considered Network Rail to ‘pass’. It did not consider the question of where the debt is guaranteed, but this would have been a novel approach. This wider idea of government control might be useful in future questions of examining government ‘control’ of an organisation for EIR purposes. However, subsequent decisions by the Upper Tribunal have suggested a strict definition of control as meaning “no genuine autonomy”, creating a high bar for a control based argument. The debt test of control is an inversion of the typical control argument. The organisation may (like Network Rail) have fairly clear operational autonomy but the government does not have autonomy because it holds all of the risk and none of the decision-making power. This is a situation that the public accounts (and good governance rules) should seek to correct but may not meet the strict test of control.
This more general idea of independence from government was a supporting part of the Information Tribunal’s argument and not the key argument. The main difference between the Information Tribunal and ONS arguments is this question of what would happen if Network Rail did not exist. The Information Tribunal argue that the services provided were not necessarily a public function, as “[i]f [National Rail Limited] did not perform these functions, they would be performed by some other similar body, not by central government”. In contrast, the ONS’ position was that the Department of Transport was on the hook both on the debt guarantee and through the Railways Act 2005 maintaining an older responsibility to “protect the interests of users of railway services”. This is interpreted by ONS as implying the government would need to keep the railways operating as “that government could allow Network Rail Ltd to fail while stepping in to support and protect the wider rail industry seems questionable, given that no one else is bearing any significant financial risks in relation to Network Rail”. The evaluation of who holds the financial risk questions the Information Tribunal’s premise that another similar body rather than central government would perform the functions of Network Rail. As Network Rail’s functions were only possible as a result of substantial arm’s length support from the central government, the same would be true for any replacement organisation and so the central government is not a disinterested party. This undermines the argument about lack of control, but it was just one supporting argument. It is unclear if the approach would have led to a different decision, but it seems unlikely.
In the niche situation that government control is a result of risk exposure, the Network Rail case shows the Information Tribunal and Upper Tribunal are not the only possible avenue. A large and significant organisation that was judged to not be subject to FOI and EIR is now retrospectively understood to have been part of the government accounts throughout the whole period. Accountancy arguments of control may or may not convince the courts, but they only need to convince the accountants.
- 1: Document 2.1 in this FOI
Header image: Photo by Felix on Unsplash
Following our report into Reforming Freedom of Information in the UK, there were several pieces of research that didn’t fit inside the final report. This exploration of differences in outcome and response times is one of them.
Freedom of information law in both Scotland and wider UK jurisdictions allows requesters who have had information withheld to request an internal review, where the authority is asked to reassess their original decision. This is a process that frequently changes the outcome. Based on data in Scotland, 40% of internal reviews result in some form of new information being released, and based on the UK central government roughly 25% of internal reviews lead to more information being released.
After internal reviews, requesters can appeal exemptions (or complain about delays in response) to the regulator. In Scotland this regulator is the Office of the Scottish Information Commission (OSIC), while in the rest of the UK this regulator is the Information Commissioner’s Office (ICO). In response to appeals, the ICO can issue a decision notice on whether an authority has properly complied with the law, and if not, how to rectify the situation.
In general, the OSIC is more likely to rule in favour of information disclosure than the ICO. Building on previous analysis by OpenDemocracy, in the 2015-2019 time range 53% of ICO decision notices fully upheld the original decision of the authority, whereas for the equivalent time range only 34% of OSIC rulings were fully upheld. The average time from complaint to decision notice was 120 days for an ICO complaint and 140 for an OSIC complaint. For OSIC decision notices, decisions that found in favour of the authority on average took longer (97 as opposed to 126 days), while for the ICO the reverse was true but with a smaller difference — decisions in favour of the authority took less time (120 as opposed to 125 days). In both cases, decisions that led to partial results took longer.
Diagram 1 – Decision notice outcomes for each regulator
Diagram 2 – Average days taken to reach different conclusions by different regulators.
OSIC ICO OSIC (time taken in days) ICO
(time taken in days)
For applicant 39% 28% 97.27 125.73 For authority 34% 53% 144.40 119.86 Partially upheld 27% 19% 198.60 141.28
Table 1 – Decision notice outcomes for each regulator
There are several different explanations for these differences. It may be that the stronger prejudice threshold in FOISA leads to more decisions in favour of the applicant, or it could reflect a different attitude of the regulator in how they approach decisions. Regardless of the mechanism, a typical complaint to the OSIC is more likely to result in a change to the decision, than the typical complaint to the ICO.
In both contexts, ministers have the ability to override the decisions of the Information Commissioner. This power is more restricted in Scotland, relating only to information of “exceptional sensitivity” that has been requested from the Scottish administration. This power has been used multiple times by ministers in rUK but has not been used in Scotland.
Following our report into Reforming Freedom of Information in the UK, there were several pieces of research that didn’t fit inside the final report. This exploration of differences in outcome and response times is one of them.
The right to access environmental information in the UK is the result of the implementation of an international convention (Aarhus Convention 1998) to provide greater access to environmental information, public participation in environmental decision making and a legal framework that allows challenge to decisions that run against the above. There are separate regulations creating the right for EIRs in Scotland (Environmental Information (Scotland) Regulations 2004) and rUK (Environmental Information Regulations 2004). Unlike FOI there are no substantial differences in the legal basis of EIR requests in Scotland. In both cases, it is a local implementation of directive 2003/4/EC of the European Parliament and of the Council and exceptions and time scales are the same in both jurisdictions.
The key differences are:
- EIR applies to a narrower range of information than FOI.
- EIR applies to more types of organisations than FOI.
- EIR requests have a higher bar to rejection.
EIR information is a subset of FOI information but a different set of exceptions apply and all exceptions include a public interest test. Authorities have to pass a harder test to withhold information in general. For emissions data, there are even fewer exemptions that can be used to withhold information. Data on usage in Scotland suggests that this leads to a higher rate of information being initially granted, and that appeals against use of exceptions are more successful. 79% of EIR requests yielded at least some of the requested information, compared to 75% of FOISA requests. There is also a slightly higher success rate for internal reviews for EIR requests. In 57% of EIR reviews the original review was upheld, compared to 60.4% in FOISA that were wholly/partially upheld. Direct comparison is difficult as by definition they are requests for different kinds of data, but EIR requests do seem to be more likely to return the information requested.
The regulations governing environmental information requests (EIRs) entered into force at the same time as the respective Freedom of Information Acts, but there are important differences between the two frameworks. The regulations apply only to Environmental Information, and so cover less information than FOI law does, but also bring new organisations into scope that are not covered by FOI laws. For instance, the ICO found in early 2020 that Heathrow airport was subject to EIR as it had functions entrusted by law. Bodies that carry out aspects of public administration are subject to EIR, but may not be to FOI. On the other hand, bodies that are only partially covered by the Freedom of Information Act may be exempt from EIR requests (for instance, the BBC).
Differences in Scotland
As in rUK, the definition of a Scottish public authority is wider under EIR than FOISA. This includes publicly-owned companies and those more generally under the “control” of a public authority. There are two points of distinction between the UK and Scottish legislation around what is covered. The first is that private bodies added under s.5 of the FOI Act are also explicitly added into coverage of EIR (the equivalent section in the UK says that inclusion under s.5 does not mean a body is subject to FOI). The second is a smaller point of distinction that the UK EIR reflects the language in the European directive that “any other body or other person, that carries out functions of public administration” are subject to EIR, but this is not present in the Scottish EIR. The result is that UK case law elsewhere that is focused on the definition of public administration is less applicable in Scotland, whereas as the OSIC put it (page 10), “[t]he decisions issued in Scotland have therefore focussed on the definition of “control” rather than of “public administrative functions”. That said, issues of control are not ignored in rUK case law with an upper tribunal judgement arguing this could only be engaged when “an entity had no genuine autonomy”. This distinction is of interest, because there are clear examples of when a body has been seen subject to EIR in Scotland but has been held not to be in rUK.
Through a combination of political decisions and OSIC decisions there are differences in equivalent bodies being subject to EIR in Scotland and rUK. Registered social landlords in Scotland were as of 2019 subject to EIR as a result of being added explicitly to FOISA, but the OSIC has held since 2014 that they were covered by EIR as they were under the control of the Scottish Housing Regulator. In the UK, the Upper Tribunal has ruled that similar registered social landlords in rUK are not “public authorities”. There are three possible explanations for this difference:
- A different legal context – the presence of a defined test around public administration has led to a more binary view of the importance of control in rUK law, compared to a more nuanced approach taken by the OSIC in the absence of a public administration test.
- A different regulatory context – the relationship between specific regulator and social landlord may be sufficiently different to pass “control” tests in Scotland that are not passed by the UK housing regulator.
- A different oversight context – OSIC may have been mistaken in their control test, but was not legally challenged.
This difference is mostly immaterial as Scottish FOI’s larger expansion to private organisations has made the difference on EIR irrelevant, but this reflects that differences in the practical availability of information rights can flow from for practicalities in regulation and oversight, as well as the law.
Fees are an example of regulatory difference where information is more accessible under UK EIR. Both EIR and FOI/FOISA regimes allow for a cost to be attached to the provision of information but FOI has a minimum ‘appropriate limit’ before this is engaged, and so most responses are uncharged. EIR has no minimum limit, and so all requesters may be charged the (small) costs of providing the information if the authority has made their charging regime public. As EIRs in Scotland and rUK are regulated by different bodies, practical differences in regulation even where the underlying law does not differ.
In a 2019 case, an ICO decision found that a ‘reasonable’ charge under EIR should effectively track the idea of an ‘appropriate limit’ in the Freedom of Information Act. The official guidance has not yet been updated to reflect this decision (through correspondence, this is still planned). The existing 2016 guidance did already argue that public authorities should ‘avoid routinely charging for all EIR requests’, but the equivalent OSIC guidance does not. The practical result of this is the ICO may uphold a complaint that Croydon Council should not charge a £50 fee for access to environmental information at all, whereas OSIC would agree that Glasgow City Council could charge £50 if this price was listed publicly. In practice, fees are rarely practically charged for EIR in Scotland but when the regulations are identical the dynamics of different regulators following different processes lead to differences in the practical implementation of EIR.
For more information, OSIC have published details on the distinction between EIRs and FOISA in Scotland. Paul Gibbons’s series in the Freedom of Information journal is a useful guide to the practical differences between the two legal regimes in rUK.
: As FOIA and FOISA have specific exemptions for environmental data, in general requests that can be considered under EIR should be, rather than under FOI rules
At the launch event for our new research report into reforming FOI in the UK, we said we would follow up questions after the event. This blog post answers questions that go beyond what we covered in the presentation. The report can be read online or downloaded as a pdf.
Is there any reason why the present Westminster govt would want to make any of the proposed changes? Is there any leverage?
One of the problems with improving transparency is that groups with the most interest in not expanding information rights tend to be those with the power to stop such improvements from happening. Given this, it is an interesting question why FOI laws ever happen in the first place. Dan Berliner tried to answer this question and found that change was most likely in “competitive political environments”, where frequent turnover in office means the benefits of transparency while out of office outweighed the costs while in office. In this framework, transparency comes from commitments made from parties while they were outside government (as was the case in the UK), or by governments who want an insurance policy if they should lose power. This is logical, but does not offer good immediate prospects or next steps.
Our current thinking is to approach this dynamic from a ‘parliament/executive’ angle and to argue that an effective FOI system should be seen as a component of broader anti-corruption work.
We position Parliament as the natural environment of an Information Commissioner because the role of the two institutions in facilitating public scrutiny of the executive can pull together.
While the original white paper cited above did not see a lot of distinction in 1997 between government and parliamentary power (for good reason), more recent experience shows that in the right circumstances Parliament can be a real independent force. Almost half of the period from when the Act came into force in 2005 until 2020 saw periods of minority or coalition control. Our argument that moving oversight of the ICO from the government to Parliament is in this context. Alongside this there’s more to be made of how parliamentarians themselves make use of FOI as a tool, and articulating how MPs/Lords can collectively work to protect and improve the usefulness of that tool.
We also think there is work to be done in making FOI less abstract and connecting a strong FOI system to accepted problems that it can be part of the solution to. For instance, Tortoise’s Chris Cook makes the argument that lobbying reform by tracking lobbyists raises a lot of complicated issues, and it is easier and more effective to have a stronger Freedom of Information process (“rather than trying to make a list of an ever-larger range of suspicious people in the private sector, just make sure that interactions with public authorities are accessible”). Given this, we’re keeping an eye out for opportunities to feed into lobbying and other future anti-corruption inquiries.
To answer another question asked, there is not an APPG on FOI, but possibly on this logic we don’t want one. What we do want is for members of APPGs in related areas (Anti-Corruption and Responsible Tax, Democracy and the Constitution, Legal and Constitutional Affairs, etc) to see FOI as something that is relevant to issues they care about. Improving FOI is a long game, and for the moment our approach is trying to expand the number of groups that see it as useful for them to protect and improve.
In separating out the two functions of the ICO is there a danger that the Data Protection side will ignore FOI considerations whereas at present the Information Commissioner needs to keep both in mind?
This is one of the reasonable questions to ask about our proposal. Indeed, to have everyone working from the same definition was part of the original justification for joining the roles.
I don’t think we can have an easy “that’s not something to worry about” answer, but the argument we make in the report is that the international standardisation of data protection (and that potential future commissioners are likely to be more familiar with this international system than domestic FOI laws) reduces the extent to which the ‘balance’ of data protection versus disclosure is one resolved by having the two roles in one person. As what data protection means is increasingly defined in standard terms, there is less room for differences in interpretation and so including FOI responsibilities has less of an influence on data protection considerations. This is an area where there’s a lot of space in the middle, but our position is that the original arguments about conflicting definitions of personal information are now resolved through a more widespread standard.
Would it be a good idea to give a better Appeals process from where the ICO refuses things, rather than having to go through an MP to get to the Parliamentary Ombudsman or face a costly Court case?
Don’t have developed thoughts on this one, but we are currently doing a research project investigating how appeals processes work for FOI across Europe, and it is making us appreciate a bit more the benefits of free appeals through the Commissioner in the UK compared to countries where the first remedy is through the courts. Not to say it can’t be improved, but having a free external appeal through the Commissioner (and that decision is then appealable, if not for free) is not bad as things go.
Are you in favour of changing the FOI act to allow for FOI lawsuits, like with the US version? What about removing the exemptions for the intelligence agencies? Again, under the US law there are national security exemptions but the agencies themselves are not automatically exempt.
For context, in the US requesters can sue for access to information when the response is not received within the statutory deadline.
For us, the first step is understanding more through statistics about how often the statutory deadline is not met, and in improving the logic of the appeal process through treating lack of response as a refusal. Ideally the FOI system should work without the frequent use of the legal system, and giving up on that is putting FOI beyond the reach of the normal citizen. In the US use of lawsuits seems common enough that they’re a solution for requesters who can pay, but not a good fix to the underlying issue of slow response times.
You also have to be careful how you construct legal processes and incentives. In the Netherlands, the law was constructed for a time so late fines were payable to the requester, creating a perverse incentive to overload authorities with requests.
Not really an answer to the question, but a good time to point out that DARPA (the US organisation that ARIA is based on and that is planned not to be subject to FOI) is subject to FOI.
The reports recommendations would be great, but I was wondering what we as individual requesters can do day to day?
Don’t have a full answer to this and we’re going to do more thinking about how the big picture and day to day come together. In general we think making requests in the open through WhatDoTheyKnow is a good, slow and steady way of demonstrating the value of the legislation by making the results open and accessible, and sharing knowledge about how the FOI process works. (If anyone has the interest, some ways of helping WhatDoTheyKnow more specifically are on the website).
What risk do the speakers think the possible changes to judicial reviews may make? That is a concern in itself, but would it also limit those FOI requests where the government claims exemption on various grounds?
The short answer is ‘probably not a big risk’, but getting to this answer required some thinking (and honestly some more research). The aspect of Freedom of Information that would be most relevant to the areas currently under discussion is the use of the ministerial veto. A Judicial review of the use of the ministerial veto certificate – R (Evans) v Attorney General – implied qualifications on the circumstances the veto could be used, and led to the release of the Prince Charles Letters.
For a brief background on current discussions, the 2019 Conservative Party manifesto said that “[w]e will ensure that judicial review is available to protect the rights of the individuals against an overbearing state, while ensuring that it is not abused to conduct politics by another means or to create needless delays.” The government then commissioned an Independent Review of Administrative Law which reported in early 2021. Generally this concluded that judicial review was functioning well, while recommending some procedural changes. The government is currently consulting on some details of that, but also on areas where the commission did not make strong recommendations.
From an FOI perspective, it is not wrong to pay attention to this review as the Evans case was highlighted. The review specified the Evans case as an example of where “at the least, an argument might have been made that the exercise of that power was non-justiciable or that the power could only be reviewed on the narrowest of grounds”. The review recommended that courts be given the option of making a “suspended quashing order” – which would come into effect if conditions are not met. The review suggest this approach “might have been useful in the Evans case in allaying governmental concerns created by some of the judgments in that case that no form of legislative words would ever have been accepted as authorising, in a sufficiently clear manner, an Attorney General to ‘overrule a decision of the judiciary because he does not agree with that decision’”. In other words, if the court held that the certificate was not legal, but it was technically possible to produce one in some conditions, time could be given to attempt this. This could make judicial review less useful to pursue, as questioning an invalid decision is less likely to yield a complete reversal.
The review generally favoured giving courts more procedural devices for gentler reconciliations of government and judicial power. The subsequent government consultation is also interested in action on the other end: what can Parliament do to increase the autonomy of the government? One area of interest in the consultation is ouster clauses, where a clause in legislation is used to prevent judicial review of particular acts of the executive. For the purposes of FOI this is potentially a concern as a method of removing judicial review from the use of vetoes, but before this becomes a serious concern it is worth noting (as the Review did) that a previous and more straightforward opportunity to clarify this law was not taken.
The earlier 2016 Independent Review of Freedom of Information had recommended amendments to reassert and clarify the ministerial veto’s scope (“where the executive takes a different view of the public interest in disclosure”) and this was not taken up by the government. This was possibly on the grounds that while solidifying the general right, it also marginally reduces the potential scope of the exemption and most of the problems of potential judicial review were resolvable without doing this. The main objection of the court was that the certificate had insufficient substance to override a court decision and so most of the legal obstacles can be avoided by issuing a certificate before the appeal stage. The government committed to doing this in future. To get back to the original point, if the government is interested in creating tools that reduce the scope of judicial review, these are potentially applicable to FOI but would require legislation to make use of, and if you’re doing that it is probably more robust just to expand the ministerial veto directly.
Another proposed change that could potentially impact FOI is removing the ability to appeal a refusal of a permission to appeal a first tier tribunal decision where the issue is on a matter of law in both cases. Our understanding is that this has not been used in information rights cases (almost all ‘Cart’ appeals are around immigration cases, and those that do not seem to relate to the Ministry of Justice in other respects). A few information rights decisions refer to the potential ability to appeal through this route but indicate a high bar. If anyone knows differently, let us know, but this does not seem likely to lead to a practical change in Information Rights appeals.
The right to access official information is fundamental in a healthy and vibrant democracy. Freedom of Information (FOI) legislation is a vital tool in research, journalism, and in supporting citizens and groups to hold their public institutions to account. In the UK, the Freedom of Information Act has now been in operation for over 15 years. Campaigns against adding new restrictions to Freedom of Information are generally successful and reflect the fact that FOI has become part of the constitutional settlement — but at the same time positive changes are resisted.
The Freedom of Information Act is static while the ways in which public services are delivered are changing. The regulator’s FOI work is underfunded and as such there is more focus on the data protection duties within the regulator’s portfolio. The picture of change that comes out of central government statistics is not encouraging, and there is not the data available to understand if this is a broader trend. Freedom of Information is unlikely to be abolished, but there is a danger of it sliding into obsolescence. Over time new classes of public body may never be covered by the Act, more public services are likely to be delivered by private sector organisations, and the legal rights that exist are less able to be enforced by an under-resourced regulator.
Devolution has led to a diversity of approaches where different parts of the Union can learn from useful decisions made in others. In our new paper, we take advantage of the existing (and potential for future) devolution of Freedom of Information legislation to suggest changes that learn from good examples in different systems. This has led to four sets of recommendations, based on transferring practice from one UK-based system to another:
Improving statistical knowledge of how FOI works in the UK – The Office of the Scottish Information Commissioner has built a comprehensive and invaluable picture of the functioning of FOI in Scotland by collecting statistics on how requests were received and processed by authorities. In the UK, this coverage is limited to central government and a rarely followed requirement that larger authorities publish their own statistics. The majority of FOI requests made to public authorities in the UK are not covered by public statistics, making the regulator (and the interested public) blind to trends over time, and less able to understand whether FOI is functioning well or not. We recommend the Information Commissioner’s Office (ICO) act as the host of a central repository.
Separating the Information and Data Protection components of the Information Commissioner – The UK’s Information Commissioner has two major roles: data protection and access to information. The first of these roles has always been larger, but its scope and importance has only increased over time. Separating the access to information function and transferring oversight and funding from a government department to Parliament would help solidify the role’s independence and set it up to deal with both current and future challenges.
Improving the operation of FOI and EIR across the UK – Taking examples of different approaches in the UK and Scotland, we recommend both regimes should adopt best practice from the other. This includes differences in philosophy around the strength of exemptions and extension to private operators, but also different practical approaches such as clearer rules on time scales, administrative silence, and harmonising rules on fees for FOI and EIR.
Exploring new paths for Welsh Freedom of Information – Currently the Welsh Parliament/Senedd has the ability to diverge in a similar respect to Scotland and set up a different system that applies to Welsh public authorities. We explore the implications of this and recommend a mini-divergence, where the Senedd legislates to give the Welsh Government the ability to add private organisations executing a Welsh public function to coverage of the Act.
You can find more information in the full report.
Research Mailing List
Sign up to our mailing list to hear about future research.
Header image: Photo by Toa Heftiba on Unsplash
Devolution in the UK means that some core datasets are produced separately by the statistics authorities in different nations. The statistics are generated according to local definitions that are better tailored to the needs of that area (for instance, Scottish data is far more concerned about geographically isolated communities than other nations). A problem for organisations that run UK-wide services (like mySociety) is that there often is not a single benchmark for the data they require. The data for different nations is often measuring similar concepts, but branches off in incompatible ways.
Our previous analysis of deprivation data has used English data. This represents a majority of our users, but far from all of them, and means that other users are either being excluded from analysis, or being separately analysed in much smaller pools of data (meaning that it is harder to draw firm conclusions). To try and solve this problem, we have started to construct datasets that let us compare geographic data from across the UK. The datasets described below are available on GitHub, with methodology and analysis of the resulting dataset.
The first dataset is a simple measure of whether an area is rural or urban. This is required because Scotland and Northern Ireland have different thresholds for whether an area is urban. These have been adjusted to use the England and Wales definition, with the creation of a third ‘More Rural’ category to roughly match the Scottish definition. For more precise analysis, this dataset also includes a ‘density decile’ to every area of the UK, depending where it falls on a scale from ‘most densely populated’ to ‘least densely populated’.
The second dataset is more complicated and tries to create a single UK-wide measure of multiple deprivation. All four nations produce different indexes of multiple deprivation. These use different indicators, and different weights on different kinds of deprivation to better highlight the different needs of the different areas. These indexes are incompatible, as they are not based on the same underlying indicator and all the scores have been through multiple transformations before ending up in the final index. Exactly where the nearest equivalent deprived area is in England to an area in Wales cannot be exactly determined.
That said, you can make useful comparisons without needing to be exact. A 2016 paper by Abel and colleagues provided a possible approach to this, but is now based on out of date measures of deprivation. Their key insight was that measures of income and employment deprivation are broadly compatible between nations. These only accounted for 50% of the overall construction of the index, but as other measures of deprivation are highly correlated with these, the common elements explain the large majority of the variation in multiple deprivation. They use this to effectively map scores from one index into another (their methodology and the refinements made are explained in the readme). The more recent index for Northern Ireland uses an incompatible measure of income. This means that a UK-wide model can only be standardised using the employment score (which is still fairly predictive of overall multiple deprivation scores). A separate GB index that only includes England, Scotland and Wales can use both the original measures. Both are included in the dataset.
There are significant qualifications on the use of this, but where the usage is broad and inexact anyway, analysis is possible that compares locations from across the UK. Postcode or point information can be translated into a single UK-wide measure of deprivation. Non-English data is not trapped in pools too small to be useful, and can be used to simply add data to larger sets of analysis. We have already started using this approach on our demographics explorer minisite for additional measures of national deprivation.
For more information see the technical details for: