To understand the effectiveness of Freedom of Information laws requires good quality information about the volume of requests and how they are being processed. Picking up on previous work investigating FOI statistics for central and local government, we have built a new minisite that covers Cabinet Office statistics on Freedom of Information in UK central government and OSIC statistics on Freedom of Information in Scotland. This site is up-to-date with 2020 statistics, and can be viewed at: https://research.mysociety.org/sites/foi-monitor
We can do this because there are already sets of statistics gathered and publicly released for several groups of public authorities in the UK. But in most jurisdictions there is not good information about the overall working of the system, with data often limited to issues that are brought to the attention of the local equivalent of an Information Commissioner. In our recent report we made the case for more information gathering in the UK.
In the UK, there are two good partial sources of statistics about Freedom of Information requests. The Office of the Scottish Information Commissioner (OSIC) has a quarterly process where Scottish authorities deposit statistics about the SAR, FOI and EIR requests they have received, how they have been processed, and how exemptions and exceptions were applied. For the rest of the UK, there is far less data on the operation of the Freedom of Information Act. The Cabinet Office publishes a similar quarterly series covering FOI statistics covering requests made to a selection of central government ministries, departments and agencies.
In principle, FOI in the UK applies to far more public authorities than FOI in Scotland, but in terms of available information, the OSIC collects more information from more agencies. The Cabinet Office collects 76 sets of statistics from 40 agencies, while the OSIC collects 110 from 507. This means that the available picture of information on FOI is far more complete in Scotland. In both cases information is published and accessible as spreadsheets, making it possible to analyse differences between authorities and change over time.
However, this information is not always easy to understand or use. To generalise our previous analysis of Cabinet Office statistics, we have built a new minisite that also applies to information about Scottish authorities and in principle could be generalised for other jurisdictions.
This allows examination of trends in individual statistics (overall request volumes, how those requests were processed, or how individual exemptions were used) across sectors or individual authorities. These statistics are up to date with 2020 information.
We hope this will be a useful resource for all those interested in UK FOI statistics, and as a potential model for how FOI statistics collected can be made easier to access and understand.
The creation of this mini-site was partially supported by a Adesssium grant.
Header image: Photo by Tobias Messer on Unsplash
Exploring the history of expansions of FOI to private operators for our report on Reforming FOI led to trying to understand the history of how Network Rail became subject to FOI and EIR. This blog post explores how in very niche circumstances, the highest information court is the Office for National Statistics.
Network Rail is a body that owns and manages the infrastructure of most of the railway network in Great Britain (England, Scotland and Wales). As such, it holds information of potential public interest, but its status under Freedom of Information (FOI) and the Environmental Information Regulations (EIR) has been contested. A 2006 ICO decision concluded that Network Rail was not a public authority for the purposes of FOI (FER0071801 /FER0087031), but was for the purposes of EIR. This decision was overturned by the Information Tribunal (EA/2006/0061 EA2006/0062) in 2007, who held it was not a public authority for the purposes of EIR either. These decisions are now mostly irrelevant as in 2015 Network Rail became unambiguously a public body subject to FOI and EIR, and was added under s.5 to the FOIA schedule. The complication is that this change was not a political decision, but was effectively decided by the Office of National Statistics.
When implementing the 2010 European System of Accounts (ESA10) in 2014, the Office of National Statistics (ONS) retroactively reclassified Network Rail as a “central government controlled, nonmarket body classified as part of the Central Government sector”, and as having been so since 2004. This did not result from any new understanding of facts, but ESA10 included several new tests of government ‘control’ of an organisation: the ‘degree of financing’ and the degree of ‘risk exposure’. Previous tests (appointment of officers, provisions of enabling instruments, contractual agreements) had not concluded that Network Rail was government controlled but these new tests changed that picture.
The ‘degree of financing’ test required the government to ‘fully or close to fully’ fund the body for it to count as government controlled, this was not the case with Network Rail. The second test of risk exposure is sensitive to the question of who holds debt for the organisation. In this case, ONS argued that the debt was guaranteed by the Department for Transport, and there was an effective statutory obligation for the government to step in if Network Rail was to collapse. This by ESA10 criteria made Network Rail government controlled and part of the public sector. The reclassification was announced in 2014, and the new framework agreement between Network Rail and the Department of Transport agreed that Network Rail should be subject to Freedom of Information (1.15). While the body was added under section 5 of the FOI Act (rare), this order is in other respects similar to the frequent amendments made under section 4 as it reflected a change in ‘the public sector’ rather than including non-public sector bodies fulfilling a public function under the Act.
Given accounting change means that retrospectively Network Rail should be seen as part of the government accounts in 2007, does this have any impact on the underlying logic of the Information Tribunal decision that it was not subject to EIR at this time? In this case, the Information Tribunal did consider the degree of government control through the same test of board appointment and public funding that the ONS similarly considered Network Rail to ‘pass’. It did not consider the question of where the debt is guaranteed, but this would have been a novel approach. This wider idea of government control might be useful in future questions of examining government ‘control’ of an organisation for EIR purposes. However, subsequent decisions by the Upper Tribunal have suggested a strict definition of control as meaning “no genuine autonomy”, creating a high bar for a control based argument. The debt test of control is an inversion of the typical control argument. The organisation may (like Network Rail) have fairly clear operational autonomy but the government does not have autonomy because it holds all of the risk and none of the decision-making power. This is a situation that the public accounts (and good governance rules) should seek to correct but may not meet the strict test of control.
This more general idea of independence from government was a supporting part of the Information Tribunal’s argument and not the key argument. The main difference between the Information Tribunal and ONS arguments is this question of what would happen if Network Rail did not exist. The Information Tribunal argue that the services provided were not necessarily a public function, as “[i]f [National Rail Limited] did not perform these functions, they would be performed by some other similar body, not by central government”. In contrast, the ONS’ position was that the Department of Transport was on the hook both on the debt guarantee and through the Railways Act 2005 maintaining an older responsibility to “protect the interests of users of railway services”. This is interpreted by ONS as implying the government would need to keep the railways operating as “that government could allow Network Rail Ltd to fail while stepping in to support and protect the wider rail industry seems questionable, given that no one else is bearing any significant financial risks in relation to Network Rail”. The evaluation of who holds the financial risk questions the Information Tribunal’s premise that another similar body rather than central government would perform the functions of Network Rail. As Network Rail’s functions were only possible as a result of substantial arm’s length support from the central government, the same would be true for any replacement organisation and so the central government is not a disinterested party. This undermines the argument about lack of control, but it was just one supporting argument. It is unclear if the approach would have led to a different decision, but it seems unlikely.
In the niche situation that government control is a result of risk exposure, the Network Rail case shows the Information Tribunal and Upper Tribunal are not the only possible avenue. A large and significant organisation that was judged to not be subject to FOI and EIR is now retrospectively understood to have been part of the government accounts throughout the whole period. Accountancy arguments of control may or may not convince the courts, but they only need to convince the accountants.
- 1: Document 2.1 in this FOI
Header image: Photo by Felix on Unsplash
Following our report into Reforming Freedom of Information in the UK, there were several pieces of research that didn’t fit inside the final report. This exploration of differences in outcome and response times is one of them.
Freedom of information law in both Scotland and wider UK jurisdictions allows requesters who have had information withheld to request an internal review, where the authority is asked to reassess their original decision. This is a process that frequently changes the outcome. Based on data in Scotland, 40% of internal reviews result in some form of new information being released, and based on the UK central government roughly 25% of internal reviews lead to more information being released.
After internal reviews, requesters can appeal exemptions (or complain about delays in response) to the regulator. In Scotland this regulator is the Office of the Scottish Information Commission (OSIC), while in the rest of the UK this regulator is the Information Commissioner’s Office (ICO). In response to appeals, the ICO can issue a decision notice on whether an authority has properly complied with the law, and if not, how to rectify the situation.
In general, the OSIC is more likely to rule in favour of information disclosure than the ICO. Building on previous analysis by OpenDemocracy, in the 2015-2019 time range 53% of ICO decision notices fully upheld the original decision of the authority, whereas for the equivalent time range only 34% of OSIC rulings were fully upheld. The average time from complaint to decision notice was 120 days for an ICO complaint and 140 for an OSIC complaint. For OSIC decision notices, decisions that found in favour of the authority on average took longer (97 as opposed to 126 days), while for the ICO the reverse was true but with a smaller difference — decisions in favour of the authority took less time (120 as opposed to 125 days). In both cases, decisions that led to partial results took longer.
Diagram 1 – Decision notice outcomes for each regulator
Diagram 2 – Average days taken to reach different conclusions by different regulators.
OSIC ICO OSIC (time taken in days) ICO
(time taken in days)
For applicant 39% 28% 97.27 125.73 For authority 34% 53% 144.40 119.86 Partially upheld 27% 19% 198.60 141.28
Table 1 – Decision notice outcomes for each regulator
There are several different explanations for these differences. It may be that the stronger prejudice threshold in FOISA leads to more decisions in favour of the applicant, or it could reflect a different attitude of the regulator in how they approach decisions. Regardless of the mechanism, a typical complaint to the OSIC is more likely to result in a change to the decision, than the typical complaint to the ICO.
In both contexts, ministers have the ability to override the decisions of the Information Commissioner. This power is more restricted in Scotland, relating only to information of “exceptional sensitivity” that has been requested from the Scottish administration. This power has been used multiple times by ministers in rUK but has not been used in Scotland.
Following our report into Reforming Freedom of Information in the UK, there were several pieces of research that didn’t fit inside the final report. This exploration of differences in outcome and response times is one of them.
The right to access environmental information in the UK is the result of the implementation of an international convention (Aarhus Convention 1998) to provide greater access to environmental information, public participation in environmental decision making and a legal framework that allows challenge to decisions that run against the above. There are separate regulations creating the right for EIRs in Scotland (Environmental Information (Scotland) Regulations 2004) and rUK (Environmental Information Regulations 2004). Unlike FOI there are no substantial differences in the legal basis of EIR requests in Scotland. In both cases, it is a local implementation of directive 2003/4/EC of the European Parliament and of the Council and exceptions and time scales are the same in both jurisdictions.
The key differences are:
- EIR applies to a narrower range of information than FOI.
- EIR applies to more types of organisations than FOI.
- EIR requests have a higher bar to rejection.
EIR information is a subset of FOI information but a different set of exceptions apply and all exceptions include a public interest test. Authorities have to pass a harder test to withhold information in general. For emissions data, there are even fewer exemptions that can be used to withhold information. Data on usage in Scotland suggests that this leads to a higher rate of information being initially granted, and that appeals against use of exceptions are more successful. 79% of EIR requests yielded at least some of the requested information, compared to 75% of FOISA requests. There is also a slightly higher success rate for internal reviews for EIR requests. In 57% of EIR reviews the original review was upheld, compared to 60.4% in FOISA that were wholly/partially upheld. Direct comparison is difficult as by definition they are requests for different kinds of data, but EIR requests do seem to be more likely to return the information requested.
The regulations governing environmental information requests (EIRs) entered into force at the same time as the respective Freedom of Information Acts, but there are important differences between the two frameworks. The regulations apply only to Environmental Information, and so cover less information than FOI law does, but also bring new organisations into scope that are not covered by FOI laws. For instance, the ICO found in early 2020 that Heathrow airport was subject to EIR as it had functions entrusted by law. Bodies that carry out aspects of public administration are subject to EIR, but may not be to FOI. On the other hand, bodies that are only partially covered by the Freedom of Information Act may be exempt from EIR requests (for instance, the BBC).
Differences in Scotland
As in rUK, the definition of a Scottish public authority is wider under EIR than FOISA. This includes publicly-owned companies and those more generally under the “control” of a public authority. There are two points of distinction between the UK and Scottish legislation around what is covered. The first is that private bodies added under s.5 of the FOI Act are also explicitly added into coverage of EIR (the equivalent section in the UK says that inclusion under s.5 does not mean a body is subject to FOI). The second is a smaller point of distinction that the UK EIR reflects the language in the European directive that “any other body or other person, that carries out functions of public administration” are subject to EIR, but this is not present in the Scottish EIR. The result is that UK case law elsewhere that is focused on the definition of public administration is less applicable in Scotland, whereas as the OSIC put it (page 10), “[t]he decisions issued in Scotland have therefore focussed on the definition of “control” rather than of “public administrative functions”. That said, issues of control are not ignored in rUK case law with an upper tribunal judgement arguing this could only be engaged when “an entity had no genuine autonomy”. This distinction is of interest, because there are clear examples of when a body has been seen subject to EIR in Scotland but has been held not to be in rUK.
Through a combination of political decisions and OSIC decisions there are differences in equivalent bodies being subject to EIR in Scotland and rUK. Registered social landlords in Scotland were as of 2019 subject to EIR as a result of being added explicitly to FOISA, but the OSIC has held since 2014 that they were covered by EIR as they were under the control of the Scottish Housing Regulator. In the UK, the Upper Tribunal has ruled that similar registered social landlords in rUK are not “public authorities”. There are three possible explanations for this difference:
- A different legal context – the presence of a defined test around public administration has led to a more binary view of the importance of control in rUK law, compared to a more nuanced approach taken by the OSIC in the absence of a public administration test.
- A different regulatory context – the relationship between specific regulator and social landlord may be sufficiently different to pass “control” tests in Scotland that are not passed by the UK housing regulator.
- A different oversight context – OSIC may have been mistaken in their control test, but was not legally challenged.
This difference is mostly immaterial as Scottish FOI’s larger expansion to private organisations has made the difference on EIR irrelevant, but this reflects that differences in the practical availability of information rights can flow from for practicalities in regulation and oversight, as well as the law.
Fees are an example of regulatory difference where information is more accessible under UK EIR. Both EIR and FOI/FOISA regimes allow for a cost to be attached to the provision of information but FOI has a minimum ‘appropriate limit’ before this is engaged, and so most responses are uncharged. EIR has no minimum limit, and so all requesters may be charged the (small) costs of providing the information if the authority has made their charging regime public. As EIRs in Scotland and rUK are regulated by different bodies, practical differences in regulation even where the underlying law does not differ.
In a 2019 case, an ICO decision found that a ‘reasonable’ charge under EIR should effectively track the idea of an ‘appropriate limit’ in the Freedom of Information Act. The official guidance has not yet been updated to reflect this decision (through correspondence, this is still planned). The existing 2016 guidance did already argue that public authorities should ‘avoid routinely charging for all EIR requests’, but the equivalent OSIC guidance does not. The practical result of this is the ICO may uphold a complaint that Croydon Council should not charge a £50 fee for access to environmental information at all, whereas OSIC would agree that Glasgow City Council could charge £50 if this price was listed publicly. In practice, fees are rarely practically charged for EIR in Scotland but when the regulations are identical the dynamics of different regulators following different processes lead to differences in the practical implementation of EIR.
For more information, OSIC have published details on the distinction between EIRs and FOISA in Scotland. Paul Gibbons’s series in the Freedom of Information journal is a useful guide to the practical differences between the two legal regimes in rUK.
: As FOIA and FOISA have specific exemptions for environmental data, in general requests that can be considered under EIR should be, rather than under FOI rules
At the launch event for our new research report into reforming FOI in the UK, we said we would follow up questions after the event. This blog post answers questions that go beyond what we covered in the presentation. The report can be read online or downloaded as a pdf.
Is there any reason why the present Westminster govt would want to make any of the proposed changes? Is there any leverage?
One of the problems with improving transparency is that groups with the most interest in not expanding information rights tend to be those with the power to stop such improvements from happening. Given this, it is an interesting question why FOI laws ever happen in the first place. Dan Berliner tried to answer this question and found that change was most likely in “competitive political environments”, where frequent turnover in office means the benefits of transparency while out of office outweighed the costs while in office. In this framework, transparency comes from commitments made from parties while they were outside government (as was the case in the UK), or by governments who want an insurance policy if they should lose power. This is logical, but does not offer good immediate prospects or next steps.
Our current thinking is to approach this dynamic from a ‘parliament/executive’ angle and to argue that an effective FOI system should be seen as a component of broader anti-corruption work.
We position Parliament as the natural environment of an Information Commissioner because the role of the two institutions in facilitating public scrutiny of the executive can pull together.
While the original white paper cited above did not see a lot of distinction in 1997 between government and parliamentary power (for good reason), more recent experience shows that in the right circumstances Parliament can be a real independent force. Almost half of the period from when the Act came into force in 2005 until 2020 saw periods of minority or coalition control. Our argument that moving oversight of the ICO from the government to Parliament is in this context. Alongside this there’s more to be made of how parliamentarians themselves make use of FOI as a tool, and articulating how MPs/Lords can collectively work to protect and improve the usefulness of that tool.
We also think there is work to be done in making FOI less abstract and connecting a strong FOI system to accepted problems that it can be part of the solution to. For instance, Tortoise’s Chris Cook makes the argument that lobbying reform by tracking lobbyists raises a lot of complicated issues, and it is easier and more effective to have a stronger Freedom of Information process (“rather than trying to make a list of an ever-larger range of suspicious people in the private sector, just make sure that interactions with public authorities are accessible”). Given this, we’re keeping an eye out for opportunities to feed into lobbying and other future anti-corruption inquiries.
To answer another question asked, there is not an APPG on FOI, but possibly on this logic we don’t want one. What we do want is for members of APPGs in related areas (Anti-Corruption and Responsible Tax, Democracy and the Constitution, Legal and Constitutional Affairs, etc) to see FOI as something that is relevant to issues they care about. Improving FOI is a long game, and for the moment our approach is trying to expand the number of groups that see it as useful for them to protect and improve.
In separating out the two functions of the ICO is there a danger that the Data Protection side will ignore FOI considerations whereas at present the Information Commissioner needs to keep both in mind?
This is one of the reasonable questions to ask about our proposal. Indeed, to have everyone working from the same definition was part of the original justification for joining the roles.
I don’t think we can have an easy “that’s not something to worry about” answer, but the argument we make in the report is that the international standardisation of data protection (and that potential future commissioners are likely to be more familiar with this international system than domestic FOI laws) reduces the extent to which the ‘balance’ of data protection versus disclosure is one resolved by having the two roles in one person. As what data protection means is increasingly defined in standard terms, there is less room for differences in interpretation and so including FOI responsibilities has less of an influence on data protection considerations. This is an area where there’s a lot of space in the middle, but our position is that the original arguments about conflicting definitions of personal information are now resolved through a more widespread standard.
Would it be a good idea to give a better Appeals process from where the ICO refuses things, rather than having to go through an MP to get to the Parliamentary Ombudsman or face a costly Court case?
Don’t have developed thoughts on this one, but we are currently doing a research project investigating how appeals processes work for FOI across Europe, and it is making us appreciate a bit more the benefits of free appeals through the Commissioner in the UK compared to countries where the first remedy is through the courts. Not to say it can’t be improved, but having a free external appeal through the Commissioner (and that decision is then appealable, if not for free) is not bad as things go.
Are you in favour of changing the FOI act to allow for FOI lawsuits, like with the US version? What about removing the exemptions for the intelligence agencies? Again, under the US law there are national security exemptions but the agencies themselves are not automatically exempt.
For context, in the US requesters can sue for access to information when the response is not received within the statutory deadline.
For us, the first step is understanding more through statistics about how often the statutory deadline is not met, and in improving the logic of the appeal process through treating lack of response as a refusal. Ideally the FOI system should work without the frequent use of the legal system, and giving up on that is putting FOI beyond the reach of the normal citizen. In the US use of lawsuits seems common enough that they’re a solution for requesters who can pay, but not a good fix to the underlying issue of slow response times.
You also have to be careful how you construct legal processes and incentives. In the Netherlands, the law was constructed for a time so late fines were payable to the requester, creating a perverse incentive to overload authorities with requests.
Not really an answer to the question, but a good time to point out that DARPA (the US organisation that ARIA is based on and that is planned not to be subject to FOI) is subject to FOI.
The reports recommendations would be great, but I was wondering what we as individual requesters can do day to day?
Don’t have a full answer to this and we’re going to do more thinking about how the big picture and day to day come together. In general we think making requests in the open through WhatDoTheyKnow is a good, slow and steady way of demonstrating the value of the legislation by making the results open and accessible, and sharing knowledge about how the FOI process works. (If anyone has the interest, some ways of helping WhatDoTheyKnow more specifically are on the website).
What risk do the speakers think the possible changes to judicial reviews may make? That is a concern in itself, but would it also limit those FOI requests where the government claims exemption on various grounds?
The short answer is ‘probably not a big risk’, but getting to this answer required some thinking (and honestly some more research). The aspect of Freedom of Information that would be most relevant to the areas currently under discussion is the use of the ministerial veto. A Judicial review of the use of the ministerial veto certificate – R (Evans) v Attorney General – implied qualifications on the circumstances the veto could be used, and led to the release of the Prince Charles Letters.
For a brief background on current discussions, the 2019 Conservative Party manifesto said that “[w]e will ensure that judicial review is available to protect the rights of the individuals against an overbearing state, while ensuring that it is not abused to conduct politics by another means or to create needless delays.” The government then commissioned an Independent Review of Administrative Law which reported in early 2021. Generally this concluded that judicial review was functioning well, while recommending some procedural changes. The government is currently consulting on some details of that, but also on areas where the commission did not make strong recommendations.
From an FOI perspective, it is not wrong to pay attention to this review as the Evans case was highlighted. The review specified the Evans case as an example of where “at the least, an argument might have been made that the exercise of that power was non-justiciable or that the power could only be reviewed on the narrowest of grounds”. The review recommended that courts be given the option of making a “suspended quashing order” – which would come into effect if conditions are not met. The review suggest this approach “might have been useful in the Evans case in allaying governmental concerns created by some of the judgments in that case that no form of legislative words would ever have been accepted as authorising, in a sufficiently clear manner, an Attorney General to ‘overrule a decision of the judiciary because he does not agree with that decision’”. In other words, if the court held that the certificate was not legal, but it was technically possible to produce one in some conditions, time could be given to attempt this. This could make judicial review less useful to pursue, as questioning an invalid decision is less likely to yield a complete reversal.
The review generally favoured giving courts more procedural devices for gentler reconciliations of government and judicial power. The subsequent government consultation is also interested in action on the other end: what can Parliament do to increase the autonomy of the government? One area of interest in the consultation is ouster clauses, where a clause in legislation is used to prevent judicial review of particular acts of the executive. For the purposes of FOI this is potentially a concern as a method of removing judicial review from the use of vetoes, but before this becomes a serious concern it is worth noting (as the Review did) that a previous and more straightforward opportunity to clarify this law was not taken.
The earlier 2016 Independent Review of Freedom of Information had recommended amendments to reassert and clarify the ministerial veto’s scope (“where the executive takes a different view of the public interest in disclosure”) and this was not taken up by the government. This was possibly on the grounds that while solidifying the general right, it also marginally reduces the potential scope of the exemption and most of the problems of potential judicial review were resolvable without doing this. The main objection of the court was that the certificate had insufficient substance to override a court decision and so most of the legal obstacles can be avoided by issuing a certificate before the appeal stage. The government committed to doing this in future. To get back to the original point, if the government is interested in creating tools that reduce the scope of judicial review, these are potentially applicable to FOI but would require legislation to make use of, and if you’re doing that it is probably more robust just to expand the ministerial veto directly.
Another proposed change that could potentially impact FOI is removing the ability to appeal a refusal of a permission to appeal a first tier tribunal decision where the issue is on a matter of law in both cases. Our understanding is that this has not been used in information rights cases (almost all ‘Cart’ appeals are around immigration cases, and those that do not seem to relate to the Ministry of Justice in other respects). A few information rights decisions refer to the potential ability to appeal through this route but indicate a high bar. If anyone knows differently, let us know, but this does not seem likely to lead to a practical change in Information Rights appeals.
The right to access official information is fundamental in a healthy and vibrant democracy. Freedom of Information (FOI) legislation is a vital tool in research, journalism, and in supporting citizens and groups to hold their public institutions to account. In the UK, the Freedom of Information Act has now been in operation for over 15 years. Campaigns against adding new restrictions to Freedom of Information are generally successful and reflect the fact that FOI has become part of the constitutional settlement — but at the same time positive changes are resisted.
The Freedom of Information Act is static while the ways in which public services are delivered are changing. The regulator’s FOI work is underfunded and as such there is more focus on the data protection duties within the regulator’s portfolio. The picture of change that comes out of central government statistics is not encouraging, and there is not the data available to understand if this is a broader trend. Freedom of Information is unlikely to be abolished, but there is a danger of it sliding into obsolescence. Over time new classes of public body may never be covered by the Act, more public services are likely to be delivered by private sector organisations, and the legal rights that exist are less able to be enforced by an under-resourced regulator.
Devolution has led to a diversity of approaches where different parts of the Union can learn from useful decisions made in others. In our new paper, we take advantage of the existing (and potential for future) devolution of Freedom of Information legislation to suggest changes that learn from good examples in different systems. This has led to four sets of recommendations, based on transferring practice from one UK-based system to another:
Improving statistical knowledge of how FOI works in the UK – The Office of the Scottish Information Commissioner has built a comprehensive and invaluable picture of the functioning of FOI in Scotland by collecting statistics on how requests were received and processed by authorities. In the UK, this coverage is limited to central government and a rarely followed requirement that larger authorities publish their own statistics. The majority of FOI requests made to public authorities in the UK are not covered by public statistics, making the regulator (and the interested public) blind to trends over time, and less able to understand whether FOI is functioning well or not. We recommend the Information Commissioner’s Office (ICO) act as the host of a central repository.
Separating the Information and Data Protection components of the Information Commissioner – The UK’s Information Commissioner has two major roles: data protection and access to information. The first of these roles has always been larger, but its scope and importance has only increased over time. Separating the access to information function and transferring oversight and funding from a government department to Parliament would help solidify the role’s independence and set it up to deal with both current and future challenges.
Improving the operation of FOI and EIR across the UK – Taking examples of different approaches in the UK and Scotland, we recommend both regimes should adopt best practice from the other. This includes differences in philosophy around the strength of exemptions and extension to private operators, but also different practical approaches such as clearer rules on time scales, administrative silence, and harmonising rules on fees for FOI and EIR.
Exploring new paths for Welsh Freedom of Information – Currently the Welsh Parliament/Senedd has the ability to diverge in a similar respect to Scotland and set up a different system that applies to Welsh public authorities. We explore the implications of this and recommend a mini-divergence, where the Senedd legislates to give the Welsh Government the ability to add private organisations executing a Welsh public function to coverage of the Act.
You can find more information in the full report.
Research Mailing List
Sign up to our mailing list to hear about future research.
Header image: Photo by Toa Heftiba on Unsplash
Devolution in the UK means that some core datasets are produced separately by the statistics authorities in different nations. The statistics are generated according to local definitions that are better tailored to the needs of that area (for instance, Scottish data is far more concerned about geographically isolated communities than other nations). A problem for organisations that run UK-wide services (like mySociety) is that there often is not a single benchmark for the data they require. The data for different nations is often measuring similar concepts, but branches off in incompatible ways.
Our previous analysis of deprivation data has used English data. This represents a majority of our users, but far from all of them, and means that other users are either being excluded from analysis, or being separately analysed in much smaller pools of data (meaning that it is harder to draw firm conclusions). To try and solve this problem, we have started to construct datasets that let us compare geographic data from across the UK. The datasets described below are available on GitHub, with methodology and analysis of the resulting dataset.
The first dataset is a simple measure of whether an area is rural or urban. This is required because Scotland and Northern Ireland have different thresholds for whether an area is urban. These have been adjusted to use the England and Wales definition, with the creation of a third ‘More Rural’ category to roughly match the Scottish definition. For more precise analysis, this dataset also includes a ‘density decile’ to every area of the UK, depending where it falls on a scale from ‘most densely populated’ to ‘least densely populated’.
The second dataset is more complicated and tries to create a single UK-wide measure of multiple deprivation. All four nations produce different indexes of multiple deprivation. These use different indicators, and different weights on different kinds of deprivation to better highlight the different needs of the different areas. These indexes are incompatible, as they are not based on the same underlying indicator and all the scores have been through multiple transformations before ending up in the final index. Exactly where the nearest equivalent deprived area is in England to an area in Wales cannot be exactly determined.
That said, you can make useful comparisons without needing to be exact. A 2016 paper by Abel and colleagues provided a possible approach to this, but is now based on out of date measures of deprivation. Their key insight was that measures of income and employment deprivation are broadly compatible between nations. These only accounted for 50% of the overall construction of the index, but as other measures of deprivation are highly correlated with these, the common elements explain the large majority of the variation in multiple deprivation. They use this to effectively map scores from one index into another (their methodology and the refinements made are explained in the readme). The more recent index for Northern Ireland uses an incompatible measure of income. This means that a UK-wide model can only be standardised using the employment score (which is still fairly predictive of overall multiple deprivation scores). A separate GB index that only includes England, Scotland and Wales can use both the original measures. Both are included in the dataset.
There are significant qualifications on the use of this, but where the usage is broad and inexact anyway, analysis is possible that compares locations from across the UK. Postcode or point information can be translated into a single UK-wide measure of deprivation. Non-English data is not trapped in pools too small to be useful, and can be used to simply add data to larger sets of analysis. We have already started using this approach on our demographics explorer minisite for additional measures of national deprivation.
For more information see the technical details for:
Along with several other transparency organisations, we’ve cosigned a letter from Open Government Network, adding our voice to the message of concern at the UK government’s failure to meet its own targets as laid out in the National Action Plan for Open Government — and calling for it to get back on track before the next Action Plan is released in September.
The UK was one of the founding members of the Open Government Partnership, an international coalition launched in 2011 with a commitment for participating governments to work with civil society groups and the public towards ‘ambitious and radical’ improvements in transparency, accountability and democracy. Yet the organisation has now placed the UK under review for poor outcomes in open government.
The National Action Plans (NAPs) are the mechanism by which targets are set — supposedly in consultation with participating NGOs — on a cyclical basis; these are then assessed independently through mid- and end-term reports.
Clearly the aims and vision underpinning the OGP are very much in line with mySociety’s own missions and values, and we were commissioned last year to author the end-term design report to check how effective, and inclusive, the 2019-2021 NAP has been.
It was this report which brought to light just where, and to what degree, the government has fallen short of the required standards for public involvement, failing to liaise and take on board recommendations from civil society — and which has led to the OGP adding the UK to its watch list, putting us alongside eight other countries including Greece, Israel and Malawi.
Consultation and co-design of the UK Action Plan with civil society, a prerequisite of the mechanism, has been lacking: for example well-evidenced suggestions for improvements to Freedom of Information have been unacknowledged and unadopted. As the government heads towards the next Action Plan, due for September, there are no signs of improved engagement.
The letter asks the UK government to commit to four points to put it back on track as a leading partner in the network, including a review of previous unmet commitments to see why they were not met and whether they can be included in the new NAP. The letter also appeals for a timely publication of the next NAP, before which urgent meetings with civil society stakeholders need to be held, and the actions that arise from them implemented.
The current NAP expires in September 2021, and we, along with our civil society colleagues, implore the UK government to commit to speedy and meaningful engagement on developing high quality and effective open governance. This is especially vital for civil society and the public as a whole to be sufficiently informed to hold our government to account, now more than ever, as we recover from the COVID-19 pandemic, and development as an isolated trading entity outside of the EU.
Image: Timo Wielink
When TheyWorkForYou launched back in 2004, it was a world first. Never before had parliamentary data been used to power a digital tool designed specifically for citizens to better understand how their MPs were representing them in parliament.
Innovations like TheyWorkForYou and our open source code Pombola, which was designed to help people elsewhere run their own parliamentary monitoring sites, have helped make mySociety a bit of a global expert in the digital transformation of parliaments and parliamentary data, and over the years we have been working a lot with international parliaments to help them to realise their own digital potential, so that their people can better hold them to account.
One of our key partners in this work is Westminster Foundation for Democracy, alongside whom we have worked with parliaments from Morocco to Uzbekistan to Myanmar. While each parliament is fascinatingly unique, there are very common opportunities, risks and barriers that arise in digital transformation, and an exploration of these themes is the subject of a new report published today.
‘Connected Parliaments’, published by Westminster Foundation for Democracy during Participation and Openness Week 2021, is a jointly authored report by WFD and me, mySociety’s Head of Research. It considers how local and contextual factors affect the digitisation of parliamentary business, and the potential for digital tools to empower citizens to better hold their political institutions to account.
Image: Fabio Bracht
In 2019, mySociety was involved in several projects working with local councils around using participatory or deliberative democracy to address a local issue (Public Square and the Innovations in Democracy Programme). Something that kept coming up at the fringes of these projects were the political considerations that led councils to find the idea of alternate forms of democracy appealing in the first place.
Understanding more about this seemed important to the future spread of these ideas, and so as part of the Public Square project, we set out to find out how local councillors viewed ‘new’ forms of democracy, and how these views varied by the political situation of the councils and of the councillors themselves.
Using a survey of local councillors, we tried to learn about different awareness and attitudes towards deliberative or participative exercises. We found that partisan and structural factors shape the perceptions of local representatives of citizen participation, and a wide-range of views among local councillors. Some were supportive of more weight being put on citizen participation, while others argued that if decisions are made by elected councillors there is someone to hold accountable. Both awareness and support for participatory methods increased if there was local experience of an exercise. Even opposition councillors tended to be quite supportive (76%) of participatory processes when run by the current leadership of the council.
Where there is more disagreement was in how the outcome of processes should be handled. Very few councillors favour approaches where the result is authoritative or binding. Councillors in councils where there is no one party with an overall majority are more likely to give greater weight to participatory exercises (59%) than those where there is a single party majority (38%). Every policy area except Children’s Social Care had over 50% acceptance that a participatory exercise could be appropriate. Programmes related to environment and cultural programmes rated highly, while programmes concerning social care scored lower. For all categories except planning and public health, councillors rated these activities as more appropriate if their council had previously engaged in such an exercise.
Overall, this survey told us that councillors make personal evaluations of participatory exercises based on a mix of political and practical factors. While there is a tension between participatory and representative democratic structures, in practice this tension can lead to a variety of outcomes. The success or failure of future participation requires understanding about how this tension affects not just the form of deliberative exercises, but how results will be interpreted and implemented.
Image: Lucas Benjamin