For the the third in our series of discovery workshops, we invited people working with FOI in public authorities to discuss how a network might support them — and we had more than 50 attendees joining us from a range of organisations and specialities. 

Discussion was lively and informative, with many expressing a thirst for community and knowledge-sharing in their roles.

We began with a group brainstorming session to discuss the challenges and obstacles people were facing in their work with FOI. 

From this, we pulled out the four major groupings below, so that smaller breakout groups could discuss what attempts had been made previously to mitigate these challenges, how effective these had been, and what an FOI Network could do to help.

Volume/complexity of requests

The increasing volume of FOI requests being received (a challenge which overlaps with that of the lack of resources, below) came up as a common issue, especially in conjunction with the increasing use of AI to generate requests. 

Here, there are two concerns: that AI is leading to more complex (if not necessarily more effective) requests; and that there is potential for a deliberate, malicious use of AI-generated FOI requests that might overwhelm an authority without their necessarily being aware of it. 

AI-generated/assisted requests are hard to formally recognise (although many are developing a ‘gut feeling’ around them), but also not inherently illegitimate. The group discussed tactics such as asking for clarification or ID, to flush out potentially inauthentic requests if suspected. A participant from the ICO also shared their recent AI guidance.

What could an FOI Network contribute here? Convening people was seen as useful in helping to understand patterns and themes between authorities, and shared approaches. This might take the form of directly organised networking activity, or supporting and promoting the informal networks that already exist. 

Building centralised resources might also help in creating tools for assessing thresholds of vexatiousness, while also providing better assistance to requesters on what good and bad degrees of complexity looks like in an FOI request. 

Locating information

Finding the information that is being requested is at the very heart of what an FOI officer does  — and can present a sizable challenge, especially where data is not collected or stored consistently. 

Discussion touched on issues around record keeping, proactive publication, resource and support from colleagues in sourcing/collecting  information: while the officer is the entry point for requests for information, they most likely hold little of it themselves. 

The amount of resource, support and priority is given to record keeping and FOI across the organisation affects how effective an officer can be. 

Information can only be easily accessed if it is stored well: some participants talked about requests for data that is not currently centrally held, but which requesters argue should be, leading to antagonistic interactions, despite the Information Officer not being to blame. 

Participants talked both about resourcing conflicts where other priorities were legitimately higher (eg “The information holders are clinical staff (NHS) and trying to get them to answer FOI requests when they are busy with patients is not reasonable”); but also situations where requests not taken seriously by senior teams, or other departments were slow to engage with them. 

As such, a key challenge for information officers is navigating both the formal and relational structures of their organisation, and a key challenge for an FOI Network is finding ways to support this role in developing a culture of transparency and good record keeping practice.

Part of this fits with our theory that good FOI statistics are an important factor in empowering information officers — because this visibility would make FOI performance between organisations more salient, and so a greater concern for senior decision-makers. 

Lack of resources

This group discussed the lack of resources, staffing and slashed funding — including elsewhere in the organisation, where diminished budgets can remove the institutional knowledge and capacity to effectively find information.

This was another area where there was great enthusiasm for better connections between officers across organisations, especially for the small, isolated teams. This would allow all to benefit from the knowledge of a wider group. 

For a longer-term fix to the lack of resources, a united network could lobby to central government. This isn’t just about “more money”, but the effective production of centralised resources that would help everyone (eg software, tools and licences). 

It was noted that redaction was a problem that was significantly time consuming, and available redaction tools (like Adobe Pro) were expensive and had limited licences. 

Proactive publication was also identified as a resourcing issue: in repeated requests for hot topics; but also in that publishing information can lead to more requests asking for specifics. 

From our point of view running WhatDoTheyKnow, this should still mean a greater public benefit from the information provided (people who wouldn’t ask for it have access to it), but does caution against an easy “publish more, request numbers decrease” approach — which does not align with the experience of practitioners. 

Working with requesters

This group discussed complaints resulting from a requester not knowing how to navigate the FOI system or complaints processes, and thoughts on more effective communication. A lack of requester awareness that the FOI route is not the same as the complaint route was identified, as well as public confusion between FOI and data subject rights. 

There is recognition that the requester doesn’t always know what information is available, which can lead to complex initial requests. But there was also a reported increase in adversarial/angry complaints, with a perception from request-makers that information was being denied when in didn’t exist. Where people are making requests across multiple authorities, getting refusals from some but not from others, can lead to this impression, while actually just reflecting differences in what data is collected. 

Better information and signposting about how to make a good FOI request was considered helpful, but within limits. Improved web forms can be helpful, but are not the only route in. At the same time, from a volume and overload point of view, a concern that greater awareness of the act might lead to more requests. This makes it important to define what we’re after as a network that can reconcile both a civil society “it’s good if more people are aware of and use their rights” with the practicalities of make that right real, which includes understanding of capacity.

In general, a lot of the potential in this area is around helping those making, and those answering requests to understand each other, or at least understanding more about how things work behind the scenes. 

 —

Finally, there was a more general discussion about forms a network can take, including the difficulty of convening both requesters and practitioners. Creating spaces for authority-side practitioners to talk helps with the smooth functioning of the FOI Act; these spaces would be more hesitant if always shared with civil society groups (and vice versa). 

We want to find ways to bridge these groups, while recognising that both individually can be constructive.  We need a set of layered discussions about how to make FOI work in practice, that can manage both communities of practice, also bridging both sides — recognising where common frustrations and collective goals can be served through better communication and coordination. This is inherently going to be complex, but will be so worthwhile to explore. 

 

—

Illustration: Alghozy

 

We continue with exploratory work that will help shape what the FOI Network will become, and how it will best serve specific types of those who use Freedom Of Information. Last week, in our second workshop of three, we convened journalists and specialist users to help us understand what the barriers are to their use of FOI, and how the Network will be best positioned to help.

More than 20 people joined us, from a range of organisations and specialities. 

A group brainstorming session surfaced the challenges and obstacles that attendees face when using FOI in their work. From these, we pulled out four priority themes; then we further discussed how best to mitigate these challenges; and what an FOI network could do to help.

Themes and conclusions

Passive pushback

The phrase “passive pushback” describes a culture within authorities not of active hostility, but in which FOI requests are not processed promptly, and where transparency is neither celebrated nor prioritised. 

Participants talked about different kinds of delays and obstructions, and strategies they’d attempted to get around this, including formal approaches such as referring to relevant ICO decision notices, or working with lawyers so correspondence included their letterhead. 

Could an FOI Network support, incentivise or celebrate a culture of openness and compliance within authorities? The group discussed the importance of both carrots and sticks (or shame and envy), rewarding and highlighting good practice; and potentially putting out comparative FOI statistics to make performance more visible and easy to benchmark. 

Active pushback

Active pushback covers more explicitly obstructive behaviour, including the use of rules and processes to slow down the progress of a request (for example, the use of public interest exemptions which may later be reconsidered and overturned), and coordination between authorities (where the intention is rarely to ensure everyone is being correctly open, but more likely to be working against the spirit of the ‘applicant blind’ principle that is written into the FOI Act). 

Participants shared the methods they’d attempted in the face of such stonewalling, including complaints to the ICO, “meta requests” asking authorities about their coordination mechanisms, and naming and shaming authorities for obstruction. 

As a network, we can provide peer support and resources around such tactics, and useful responses. 

But there is also an important advocacy angle, where building evidence and lobbying to change ICO/OSIC enforcement strategies can be an important collective rather than individual counter to obstructive approaches. 

A State of FOI report could cover both the positive “culture of openness” stories, but also dig more into patterns in obstructive responses. 

Writing/managing good requests

One important issue that came up in conversation was that it is a skill to write and manage FOI requests well — one that takes time to learn. It requires both an understanding of when legal approaches are helpful, but also a sense of what is possible through Freedom of Information (where a focus on specific information that already exists is required). 

Building this expertise can be a problem if FOI skills are not already well embedded in your organisation, and you are starting effectively from scratch. 

Here, community and training would be helpful. As a newcomer takes time to build up accumulated wisdom, peer support and mentor programmes could be helpful in walking through concrete examples of where FOI can and can’t be helpful. 

Going beyond that, we discussed the concept of training products and services that could help support increasing specialist use. These might include a custom newsletter featuring FOI tricks and tips, not aimed at first time users, but for journalists and specialists. The network already includes several organisations and individuals who could provide such training and expertise — and a willing audience. Here, the value of the network would be in joining people up, rather than trying to do everything itself.

The role of technology was also discussed. It can help with the challenge of managing requests  — one peron mentioned (unprompted!) WhatDoTheyKnow Pro as helpful for tracking and extracting data. 

Some are using AI to help them refine their requests, and we discussed the potential ways we could make AI assistance lean more towards “sharper, easier to process” requests, and away from the problems, already observed, of for example where AI hallucinates ICO notices and inserts them into correspondence. Here a network could help disseminate a ‘skill’ to bring more specialist knowledge towards shaping the AI’s actions; and a network that was also inclusive of FOI practitioners could help refine that from both sides. 

Understanding/using appeal processes

The final challenges we discussed were around understanding and using the appeal processes that are built in to the FOI system, noting that there’s a strong difference between Scottish FOI and the rest of the UK: legal appeals to OSIC are rare (and only possible on points of law). 

We again talked about the value of peer support; as well as building collections in response to common pushbacks. It was acknowledged, though, that taking things to tribunal is hard, and that one can be outgunned by the representation of the public authorities. 

Here it’s important that network activity is not just support for individuals, but lobbying and campaigning for improvements to the overall appeal system. 

This might be through pressing for improved data and information such as ICO timescales, and the tribunal returning to publishing current and upcoming cases; but also through lobbying for greater funding for regulators to be able to run an effective and timely system of appeal. 

Takeaways

Thanks to all who took part in the conversation, which has helped shape our understanding of what is useful activity to support journalists and specialist users.

Practical peer support for those in smaller organisations might be especially important — but dealing properly with the issues raised also involves wider campaigning activity around making sure the overall system is functional to be able to engage with. 

 —

In our third workshop, we move to a focus on those working in public authorities, where we want to explore more of these questions around supporting a culture of openness, and also our common interest in improving the quality and clarity of requests. If you are an Information Officer or practitioner, please do join.

 —

Image: GuerillaBuzz

The FOI Network is an informal coalition of civil society organisations, journalists and academics with an interest in Freedom of Information, convened by mySociety and State of the Future. Last week, we held our first meet-up, in the shape of an online workshop, to discuss and prioritise the ways in which we might strengthen and defend the right to information in the UK. With potential threats rumbling on the horizon, it was a timely conversation.

Attendees came from a range of organisations and specialities. We had a group brainstorming session to identify firstly, opportunities to strengthen or expand FOI; and secondly, key threats and challenges.

From these, we pulled out four themes, which we discussed from the angle of which activities would bring the most impact for the effort expended.

Thematic groupings

The four topics discussed were:

• The expansion of FOI to currently uncovered bodies/sectors
• AI as an opportunity and challenge, and proactive / better publication
• Practical difficulties/support for FOI within public bodies
• The government’s opposition/lukewarm attitude towards existing/greater transparency

FOI expansion

As  FOI’s reach is expanded, so is its utility to new groups, who would benefit from the Act in different ways. This is an approach that can benefit communities who have historically had low levels of power.

Expanding the FOI Act to new authorities would make possible new avenues of research and enquiry, providing access to useful data where it is not currently available. This includes important areas such as housing; or private contractors to government, where the case for increased transparency is easy to make.

We’re fortunate that we can look to Scotland, where Registered Social Landlords are subject to FOI requests, as an example: there is already a good evidence base for successful expansion. Equally, good arguments for expansion could be made by showing the types of essential questions that cannot currently be answered under the regime as it stands.

AI as opportunity and challenge, and proactive / better publication

In this area, the group decided that there are no highly effective actions that would also be easy to implement. Instead, we would be looking at a range of sensible small interventions around better guidance, training research, and more intensive technical work around proactive disclosure and unlocking the benefits of public data.

There is a wider problem around AI potentially overwhelming appeal mechanisms (for more on this, see the two Information Commissioners’ talks at FOI Fest). There is more to explore here, around triage methods, AI and increased volumes of both requests and appeals.

Practical difficulties/support for FOI within public bodies 

An effective FOI system requires information officers to be well-resourced and supported within their organisation.

Here, potential actions ranged from campaigns for better stats around FOI (making FOI more visible to decision-makers, as in Scotland); sharing and promoting the success stories of FOI to show the value of the work; better networking/surveys of the profession; campaigning for statutory FOI officers; and technical support on document management/search technologies. 

There was also some discussion around organisations where responding to requests pulls officers away from other work, affecting the prevailing attitudes towards FOI. The concept of statutory officers would have some bearing on this.

We will develop this segment in a further workshop, to which practitioners themselves will be invited. 

Government opposed to/has a lukewarm attitude towards transparency

A key concern is how we improve FOI, when some of the mood music coming out of government is in favour of greater restrictions. But at the same time, “government” is a wide term: while there will be some institutional reluctance to transparency, there will also be some pockets where it aligns with other stated objectives. 

We need a clearer map and understanding of these factions. We may need to be both defensive, pushing back against threats to transparency, while also building diverse institutional support. One benefit of an FOI network is that different parts of the coalition can do both at once.

From this follows a need for positive, public advocacy for the benefits of transparency, as well as a clear narrative of how it fits into wider government agendas around the redress of historic injustice, anti-corruption, value for money and so forth.

So that’s the summary of our discussion. We’ll keep you posted with progress reports from the FOI Network.

The next meet-up is about how the Network can support journalists and specialist users of FOI: if that’s of interest, sign up here.

 

—

Image: Mark Fletcher-Brown

We’ve just started a new project exploring how improved data and training could help youth services in England – building their financial sustainability and representing their communities at a national policy level.

Working with the Social Investment Business (SIB), which coordinates the government’s Youth Investment Fund, we’ll be talking to youth organisations across the North of England between now and July, to understand what challenges they face with coordinating action, demonstrating support, and evidencing impact for fundraising.

Through a short series of interviews, workshops, and free online training sessions, we’ll find out together where there might be opportunities to build these organisations’ capacity for data analysis and engagement with public authorities.

We’ve already seen how climate and nature organisations have used the Local Intelligence Hub to identify local partners, compare approaches nationally, and prepare for conversations with their councillors and MPs. Through this project, we’ll get a chance to explore whether that same model could help organisations in the youth sector to build stronger partnerships and deliver their services more effectively.

Nick Temple, SIB CEO, said: “One of the key challenges we hear from youth organisations is how hard it can be to evidence their impact while juggling frontline delivery. This project with mySociety gives us a valuable opportunity to explore practical ways that data, insight and shared learning can support organisations to grow stronger, collaborate more effectively, and make their voices heard—building on the legacy of the Youth Investment Fund for the long term.”

We’ll be inviting current and former YIF grantees to get involved, via SIB’s network, shortly.

But if you’re a youth organisation in the wider sector and would like to find out more about the project, or how your organisation could make the most of mySociety’s other tools like TheyWorkForYou and WhatDoTheyKnow, you can still be involved – just get in touch!

In a recent blog post, we set out why proposals to reduce the Freedom of Information cost limit would reduce government transparency, without meaningfully reducing the burden on public authorities, since then the opposition to the move has grown. 

What’s the problem?

On 18 March 2026, The Financial Times reported that the government is considering lowering the cost limit for FOI (and therefore increasing the amount of FOI requests that can be rejected). The justification given for this is rising request volumes,  financial pressure on departmental budgets, and a mooted national security question (paywalled link).

What people are saying

Across our FOI network and beyond, these arguments have been widely challenged. In his initial blog post, journalist George Greenwood argues that the risks being cited are already well understood and already addressed within existing FOI exemptions. He goes further in the Times (paywalled), describing the proposals as a “democratic retreat” that would make it harder to uncover government wrongdoing and major public scandals. 

Looking at the cost argument, Jenna Corderoy’s piece for Democracy for Sale highlights how departments have spent significant sums fighting FOI requests in tribunal, often unsuccessfully. This shows that some of the highest costs of the FOI system are not just from processing requests, but are the result of  departmental decisions to resist disclosure in the first place.

Claire Miller’s #FOIFriday roundup questions whether FOI admin costs are significant in the context of overall public spending, and highlights evidence that lowering cost limits is unlikely to reduce overall workload. Instead, Claire points to the role of internal reviews, complaints, and inefficient handling as key drivers of cost, and emphasises the benefits of proactive publication and better systems.

Editorial and institutional voices have also weighed in. A Sunday Times editorial (paywalled) frames the proposals as part of a broader pattern of excessive government secrecy, warning that lowering the cost limit would create a “serious democratic deficit”. The Society of Editors has similarly warned that restricting FOI would damage press freedom and make it harder for journalists to hold power to account.

Sector-specific responses highlight the wider impact. The Committee for Academic Freedom have written that lowering the cost limit would disproportionately affect complex, investigative requests, and in a university setting transparency is already hard-won, so scrutiny efforts should be supported, not diminished. The Press Gazette has argued that reducing the cost limit would put public interest information “beyond scrutiny”.

The story has reached elected representatives too: last week local councillors in North Yorkshire raised concerns about how a possible cost limit reduction would negatively impact local government transparency.

What are the real problems, and where can solutions be found?

The current debate reflects real pressures within the system, but the response from across the FOI community and beyond is clear that restricting access to information is not the answer. 

If the aim is to reduce cost and pressure, there are better ways to do it. Efforts to reduce access to information risk introducing larger costs elsewhere, including from inefficiency, poor decision-making, and reduced public trust. Better proactive disclosure from authorities will mean fewer requests need to be made, and fewer fights at tribunal level would save the government money. 

Transparency is not a “nice to have” that can be scaled back when budgets are tight.  It is an essential component of public services that work in the public interest. Governments that think they cannot afford transparency will be surprised at the corruption and inefficiency they will need to afford in its absence.

We’ll be continuing to write and campaign on this issue, make sure you’re signed up to our mailing list if you want to be notified of any developments. 

Photo by Markus Winkler on Unsplash

 

Last Thursday, a diverse set of people with a common interest came together at Birkbeck College in London for FOI Fest — a one day conference examining the successes, challenges and future of the Freedom of Information Act. We heard from a wide range of perspectives: those who use FOI, those who process requests, those who research the workings and effects of the Act, and those who oversee its application.

We couldn’t have wished for a better day: there was so much to discuss, with relevant and timely presentations, and real engagement in the room. As this was a kick-off event for the incipient FOI Network, the enthusiasm was gratifying; and there seems to be consensus that this should be considered the inaugural FOI Fest, rather than a standalone event.

We heard from both FOI regulators, listened to expert discussion, learned from an excellent set of skills sessions where experts shared insights on how to use FOI well; and enjoyed a set of short lightning talks where users explained how they had brought change through FOI. We’ve added links to each video presentation so you can watch any that are of interest.

Gavin Freeguard, longtime mySociety associate and FOI expert, was our host and co-organiser, kicking off proceedings by pointing out that the introduction of the FOI Act represented a “fundamental and vital change in the relationship between the public and government”. Our rights under FOI have survived two decades, and in the process an ecosystem of activists, researchers, campaigners, journalists and commissioners has arisen.

Information Commissioner John Edwards couldn’t be with us in person, but did send a video address, talking frankly about how the body is equally as stretched as they know many authorities are too — partly thanks to the increased use of AI. This was a theme that would recur through the day, from different authorities, although, as John was keen to point out, more FOI requests means more people accessing their right to information, and that, in itself, is a real positive.

Warren Seddon, Director of FOI and Transparency at the Information Commissioner’s Office, provided our first keynote, picking up on the theme of the increase in requests needing their oversight (they’re on track to receive more than 10,000 FOI complaints this year alone). AI-generated requests tend to be longer, contain inaccuracies and are less easy to understand. However, there was good news too: the ICO are exploring setting up an external monitoring system for public authorities outside of central government; Warren told us to watch this space.

Next, investigative journalist Jenna Corderoy, CEO of the Campaign for FOI Maurice Frankel, and Transparency International’s Rose Whiffen joined Warren for a panel discussion to interrogate the question “What has 21 years of FOI changed?“. 

Maurice began with the observation that the ICO’s powers are not strongly enough applied, resulting in many authorities experiencing no penalties for bad faith or late responses, albeit he’s detected some improvement since John Edwards took post. 

Jenna pointed out that times have changed since the introduction of FOI, but the Act hasn’t kept up, as can be seen for example by the government’s use of WhatsApp for messages that can be deleted and therefore cannot be publicly released. There’s an attitude issue, too: many authorities still see FOI as burdensome or even malign, when really we should all be supportive of this essential mechanism for transparency, allowing the vital uncovering of corruption and wrongdoing. 

Rose asserted that FOI has shifted public expectations, and built momentum for better government transparency, “rebalancing the information asymmetry which is where corruption thrives” and “shifting the risk calculation around corruption”.

Warren added that the things politicians were afraid of when the Act was first introduced have, for the most part, not been realised; and that it’s a great thing that any one of us can find out information from our local school, hospital or council.

In the next session, we enjoyed three ‘lightning talks’ – five minute presentations. Eleanor Shaikh, whose work you can read about in more detail in our blog post, used her slot to explain how the Act had given her the tools to uncover fundamental aspects of the UK Post Office Horizon scandal, many of which made front page headlines. She stressed the importance of taking your time to avoid being labelled ‘vexatious’, spacing requests out.

Isaac Beevor, from our Scorecards partners Climate Emergency UK, explained their use of FOI requests to obtain standardised datasets around councils’ road expansion plans, EPC ratings in social housing, energy procurement and more, as detailed in this blog post. He included a shoutout for WhatDoTheyKnow Pro and its Projects functionality, which meant they could share the work across a team of volunteers.

In the Q&A, an audience member expressed how very inspiring these two examples were, showing how one person can use FOI to bring real change.

Finally, Andreas Pavlou, Lead at the Open Government Partnership Independent Reporting Mechanism, brought the good news of positive advances in Access to Information in countries such as the Netherlands and Brazil, banging the drum for one of our favourite watchwords, collaboration.

Before breaking for lunch, we enjoyed three skills sessions.

George Greenwood, Investigations Reporter at the Times, shared ten tips from his long experience of using FOI, which you can see for yourself on our Bluesky thread.

Lucas Amin of Democracy For Sale shared his experience of using EIR (Environmental Information Regulation) to uncover big stories: he’s also shared these at one of our ATI Network online events, so you can rewatch that here if you’re interested. 

And Ben Worthy, senior lecturer in Politics at Birkbeck College, explained his research project which used FOI to dig more deeply into how FOI works, experimenting to see whether formal requests were more likely to get a response (they are).

After lunch, we complemented the morning’s keynote with another, this time from David Hamilton, the Scottish Information Commissioner, who echoed his ICO counterpart’s observations about the rise in complaints — partly because of AI and partly because public recognition of FOI is “higher than it’s ever been” (partly due to the recent high profile inquiry into the First Minister). “If you see the rights, you’re happy to use the rights”, he said, noting that the Scottish Information Commissioner is on the front page of a newspaper on average every three days at the moment, and that transparency was going to be a massive issue in Scotland’s forthcoming May election.

David also ran through the FOI Reform Bill currently going through Scottish Parliament, noting the differences between this and the FOI Act (which covers England, Wales and Northern Ireland, plus UK-wide public authorities that are based in Scotland).

For our second panel of the day, asking “What next for FOI?“, David was joined by Director of the Campaign for Freedom of Information in Scotland Carole Ewart, Ben Worthy, and mySociety’s Alex Parsons and Louise Crow.

Louise began by reminding us that not every user of WhatDoTheyKnow is making a Freedom of Information request: in fact, each response is viewed on average 160 times, massively multiplying the benefit of the information contained within it. The publication of responses also means that authorities don’t have to deal with the same requests multiple times, as the information is free for all to see. She then highlighted the importance of FOI adapting to the current rise of AI systems, and the necessity of record-keeping so that transparency about how they work is not allowed to become opaque.

Ben shared his research on the use of AI in requests, observing that in the last year the number of authorities indicating that they are receiving requests composed by AI has risen from 30 to 70%. 

Carole was frank about the blocks to reform, saying that the attitudes towards FOI needed to change so that it was no longer seen as adversarial or aggressive — but she welcomed the changes on the horizon for Scotland.

Then, in our second skills session, Maurice Frankel of the Campaign for FOI shared tips for success with FOI requests, not least around keeping them within cost limits; and Martin Rosenbaum, author of Freedom of Information: A Practical Guidebook and for many years BBC News’ FOI expert, explained the importance both of persistence and of wording your requests carefully. Finally, mySociety’s own Julia Cushion and Gareth Rees explained what we’ve been doing on WhatDoTheyKnow and Alaveteli recently, from our beginners’ guide to FOI, to this page showcasing what FOI has done around the world. 

There was just time for three more lightning talks: reshowing this video from Gabriel Geiger, Investigative Reporter at Lighthouse Reports, on their award-winning Suspicion Machines investigation that showed how machine learning algorithms were disproportionately targeting individuals based on ethnicity and gender; followed by talks from Alex Homer, Senior Journalist at the BBC Shared Data Unit who shared a live investigation around the National Police Chiefs’ Council; and Carole Ewart, Director, Campaign for Freedom of Information in Scotland spoke about their work aiding NGOs in the use of FOI.

Finally Louise Crow wrapped up the day, and before we knew it, it was time to repair to the pub to carry on conversations in a more informal setting. An all-round success – and, we hope, the start of a very useful new network that we’ll be progressing over the next few weeks. Please fill in this form if you have an interest in being part of it.

Thanks to all our speakers, and to everyone who came and helped make the day a success. We’re also grateful to JRRT for supporting this exploration of the network, and eCase for additional sponsorship. Special thanks to Ben Worthy and Birkbeck for providing the excellent venue for the conference, and to Gavin Freeguard for compering the day, and all his work bringing the network together.

Submitting evidence to Parliamentary inquiries is one of the ways in which we can have an effect on the way things work in this country.

As you may recall, we recently contributed to the Scottish Parliament’s Standards, Procedures and Public Appointments Committee on Freedom of Information reform, and now we’ve submitted written evidence to the UK Procedure Committee’s inquiry on Written Questions.

This time, we’re making the case for more alignment between FOI requests and Parliament’s Written Questions system. Written Questions are a mechanism by which MPs and Lords can hold Ministers to account, in much the same way that the public can request information from government authorities through FOI.

We have recommended that a rejected parliamentary question should be retrospectively converted to an FOI request to allow making an appeal; and that given the influence of FOI in Parliament (both by parliamentarians, and in how they use FOI requests made by others), government statistics on FOI effectiveness should be scrutinised alongside Written Question statistics.

Finally, reflecting one of the findings in our WhoFundsThem work, we think that when a Member submits a Written Question and declares an interest, they should be required to say what that interest is.

For much more detail on each of these points, you can see our submission on the Parliament website.

 

WhatDoTheyKnow Pro, our Freedom of Information service for users such as journalists, researchers and campaigners, now comes with Projects bundled in at no extra cost. That means that, as well as sending batch requests more easily, you can also bring in colleagues or volunteers to help you refine and analyse the data you receive in response.

How WhatDoTheyKnow Pro can help you

WhatDoTheyKnow Pro is a useful tool for those sending the same FOI request to multiple authorities: what we call ‘batch requesting’. Often, when embarking on an investigation, or gathering data for research or to inform a campaign, it’s helpful to gather data from many sources to create a full picture.

For example, in this recent blog post, Zarino described how he used Pro to ask every local authority in the UK how many safeguarding referrals they had received from schools that they manage. Climate Emergency UK have also used Pro to good effect, gathering data about councils’ climate action that wasn’t otherwise publicly available.

WhatDoTheyKnow Pro helps with two of the more difficult elements of bulk requesting:

• finding/compiling a list of all the relevant authorities’ email addresses; and
• keeping track of which authorities have responded, and which need following up.

What Projects adds

Now, the inclusion of Projects eases another big challenge of bulk requesting: sorting through the masses of responses to pull out the information you need.

Even when you frame your request to ask for data in a certain format, as permitted by the FOI Act, experience suggests that you’ll rarely receive responses that fit neatly into a spreadsheet for your instant analysis.

As Zarino noted in his follow-up post on requesting safeguarding data, much depends on how the authority are storing the information at their end: “We think, in reality, very few of the authorities held this data in a format structured enough to count as a ‘dataset’, but a few did send over their data in spreadsheet format, which was nice to see! Others, however, sent us tables in Word documents, in PDFs, SharePoint links, even ASCII-art tables in raw email text.”

These days, AI might be helpful with some data-refining tasks; but as we discovered with our WhoFundsThem project recently, sometimes humans are the best bet for combing through responses and pulling out the parts you need, in the format you need. Climate Emergency also took the time to train large cohorts of volunteers to ensure the assessments they were pulling out of their FOI responses for the Council Climate Action Scorecards were fair and accurate. Both projects made good use of WhatDoTheyKnow Pro and Projects.

What can Projects do for you? 

If you have one or more associate working with you, or if you can assemble a team of willing volunteers, you can share the work of going through the FOI responses as they come in. Projects makes collaboration easy. 

You can use Projects to give your team an online interface where you describe the aims of your investigation, and set out the questions you need answers to. Your helpers will then go through each response in turn and identify the parts you need, putting them into your standardised format. At the end, all their inputs are pulled into a nice, tidy spreadsheet that allows you to do the analysis you need.

Contributors don’t need a Pro membership themselves, so there’s no extra cost to you, and the only extra effort required is in setting out what data it is that you need to pull out from the responses — something it’s useful to have straight anyway!

No team to help you? Projects can also be used solo, and still helps you keep track of the information you’re pulling out — helpful if there are lots of data points.

Subscribe to WhatDoTheyKnow Pro (with Projects included), or see the Help page for more detail.

 

 —

Image: Kylie Haulk

At the beginning of the year, we set ourselves an ambitious goal: to help a group of small organisations working with marginalised communities to run Freedom of Information–based campaigns using WhatDoTheyKnow Pro’s batch-request and project features. We recruited groups working in areas as varied as domestic abuse, arts funding, youth health, SEND provision, parental leave, fuel poverty, and migrant justice. 

As the year draws to a close, we’re reflecting on the project and the lessons we’ve learned from it. It’s been a total privilege working closely with these organisations, because it gave us a front-row view of the real challenges of frontline campaigning and community support. 

What became clear early on was that the hardest part of a batch-request project isn’t actually pressing “send”.  Campaigners know their issues intimately, but FOI requires a specific kind of precision: pinning down exactly what data will answer their question, what format it should be in, and which public bodies actually hold it. Moving from “we want to understand this issue” to “we need these five questions answered from these 150 authorities” is a surprisingly big leap. 

Luckily, WhatDoTheyKnow’s knowledgeable  volunteers were able to help our groups go from vague policy areas to precise questions, and to understand what information was already out there. One of our groups didn’t end up submitting a big batch request, as in the course of their preparatory research they found an already-published dataset from an industry body they didn’t know existed. This is still a win — proactive publication by authorities makes everyone’s life easier.

In the cases where we had good questions and had identified the right authorities, we then still had to tackle the practical reality: for small teams already stretched thin, a large FOI project which asks a lot of questions requires capacity to deal with the answers. These can come in a diversity of forms: follow-ups, clarifications, refusals, delays, internal reviews. Our Projects tool helps to make dealing with the range of responses easier, but the scale of the challenge can still require serious commitment of time and resources. Zarino shared his experience of this on our blog back in October.

Just this week we had a moment that illustrates this: one of the groups we were supporting sent a batch FOI request to 133 universities on 5 July. As I write this in December, they are still receiving responses. The most recent one, a refusal, arrived five months after the original request! 

We’ve got two strands of thought here. On one hand, it’s good to be realistic. Although these moments are frustrating, they also teach us to be prepared for slow, unpredictable timelines, and that persistence is part of the craft. On the other hand, we feel strongly that citizens shouldn’t have to be quite so persistent, that pace shouldn’t be quite so slow, nor unpredictable. That’s why we’re advocating for upstream policy improvements, such as in our recent evidence to the Scottish Parliament, and in our upcoming FOI Fest conference.

Although it’s not always been straightforward, this year reinforced why FOI is worth the effort. A particularly strong example came from SCALP and Netpol’s From Scotland to Gaza report, which, with our help, used batch FOI requests to uncover policing practices around protests. Their methodical approach combined data from public bodies with testimonies to make a compelling case that has shaped media coverage and public debate. It’s a reminder that FOI doesn’t just extract information, it empowers communities to speak with confidence.

All of this left us with a clearer sense of what we can do in future to help make big FOI projects work. A few lessons stood out:

• Start smaller: a 10-authority pilot builds confidence and tests the strength of the question.
• Co-design the requests: working together on wording and structure reduces uncertainty: the organisations have expertise of their area, while our volunteers have second-to-none understanding of how to write a clear request.
• Prepare organisations for the long tail: follow-ups, delays, and refusals are, unfortunately, to be expected, not signs of failure of the project. 
• Volunteers can help with the volume of work: Climate Emergency UK have set the standard for how to train, empower and mobilise the cohorts they need to churn through large quantities of data.
• See FOI as a strategic, not administrative tool: it’s most useful when tied directly to campaign goals.

We fundamentally believe that every organisation can benefit from FOI; they just need the right scaffolding and resources. If you know what you’re in for, the whole process becomes far less intimidating.

What next? We’re refining our approach, watching what happens with our initial batch of projects, and constantly updating our guides and help pages to support our users in their big and small FOI projects. Every request is a small act of collective muscle-building. We’re excited to keep learning and keep improving the support that makes those acts possible.

Photo by Danist Soh on Unsplash

Around the world today, organisations and communities are recognising the 26th International Day for the Elimination of Violence Against Women. This is a moment to reflect on one of the most prevalent and pervasive human rights violations in the world – but it’s also a call to action.

As one small action in that continued effort, we’ve been working with the End Violence Against Women Coalition (EVAW) this year, to explore how something like our Local Intelligence Hub could help their members organise for change.

Little did we expect that, in building a prototype Data Hub for them to explore their needs, we’d discover a gaping hole in data collection about the safety of girls at schools in 16% of local authorities.

But first, what were we aiming to achieve?

In a post last month, I shared some of the goals of this work – such as using data to galvanise support from MPs, to monitor patterns that official bodies might miss, and to help EVAW’s members make the case for increased local funding to address violence against women and girls (VAWG).

I also shared how we were using the batch request tools in WhatDoTheyKnow Pro (our advanced Freedom Of Information service) to generate new public data on VAWG prevalence in schools.

And, of course, all of this work builds on top of the Local Intelligence Hub we designed and built with The Climate Coalition and Green Alliance – which has already proved its worth as a tool for community organising and public affairs, including through events like this Summer’s #ActNowChangeForever Mass Lobby, and The Climate Coalition’s Great Big Green Week.

Now it’s time for an update – how did we get on?

A replicable pipeline of brand new VAWG data

When we built the Local Intelligence Hub with The Climate Coalition (TCC), much of the data we included was already publicly available: MP information from Parliament, demographic data from the ONS, public opinion data shared by polling companies. Combined with TCC member organisations’ own data on their local support and activities, the Hub was able to present a nuanced picture of climate and nature are being protected across the whole country.

We knew we faced a different challenge with the VAWG data hub. As I explained last month, public data in this space is often incomplete, or missing entirely. We wanted to use this as an opportunity to test how WhatDoTheyKnow and the Local Intelligence Hub could work together to generate and then publish brand new datasets on VAWG prevalence or activity, made public through FOI requests to local authorities and policing bodies.

We chose school safeguarding referral figures as a suitably challenging example that was also indicative of levels of risk to children. When school staff fear a child may be in danger in any way, they are meant to refer it to the safeguarding team at their local authority. The UK government collects some information about these referrals as part of its Children In Need census, but the definition of a “child in need” is somewhat open to interpretation, and we and EVAW both suspected that, as a result, the official data was only telling part of the story. The census also only covers local authorities in England, leaving Scotland and Wales to collect their own, incompatible data (the CRCS census in Wales, and Children’s Services Plans in Scotland).

With the help of the WhatDoTheyKnow volunteers, we drafted an FOI request to be sent to every UK local authority with a responsibility for education, asking for three things:

1. The total number of safeguarding referrals made to them, by schools in their area – this is data that technically should be collected by the CIN census for English authorities, but we suspect is not
2. Any sort of categorical breakdown they held about those referrals, such as a breakdown of the genders of the children involved – this doesn’t currently appear in any public dataset that we know of
3. The total number of schoolchildren in their area

You can browse the requests and responses on WhatDoTheyKnow. Here are some key things we learned through the process:

No matter how much you research your request, something will slip through

Our background research and even our first pilot requests failed to reveal that the total number of schoolchildren is something that’s already published for England, Scotland, and Wales. Thankfully, many authorities simply pointed us to this data (with a “Section 21” refusal – “information already accessible”), but others continued to provide the data for each year we requested. Had we known in advance that the data was already available to us, we could have left it out of our requests to English, Scottish and Welsh authorities. We can only hope, since this is such basic information, the authorities who did go on to provide the data to us didn’t spend too long gathering it.

You will receive information in every format imaginable, and your data extraction process needs to handle that

We asked for responses to be provided in a “re-usable, machine-readable format” if the authority deemed the information to meet the FOI Act definition of a ‘dataset’. We think, in reality, very few of the authorities held this data in a format structured enough to count as a ‘dataset’, but a few did send over their data in spreadsheet format, which was nice to see! Others, however, sent us tables in Word documents, in PDFs, SharePoint links, even ASCII-art tables in raw email text.

We also knew authorities might hold the information by calendar, academic, or financial reporting year, so we gave them the freedom to provide it to us in whichever scheme they had. Unsurprisingly, we received responses across all three (57% calendar year, 35% financial year, 8% academic year).

Happily, the crowdsourcing interface in WhatDoTheyKnow Projects enabled us to make relatively quick work of extracting the data we needed, but we were ultimately only able to extract a fraction of the information some authorities provided and we found that some of the interpretation of the responses (ie: “is this a financial year, or an academic year?”) heavily relied on human intuition, which means we’ll need to think carefully about the way we structure future requests, if we want to process the data through any sort of automated pipeline.

Complex requests are a risk

The more information you request, the more useful it might be to you, but the more you risk the public authority refusing to answer it on “Section 12” cost grounds. WhatDoTheyKnow’s advice is to keep your request as short and focused as possible. But we knew that historical data, across a few metrics and a few years, would be most useful to the VAWG Data Hub’s users, so we asked for as much as we felt we could justify – and it mostly paid off.

70% of responses to our batch request contained both key pieces of data we wanted (the total referrals for multiple years, and the gender breakdown). Another 7% contained just the yearly totals, without any gender breakdown.

7% are still awaiting a response, even now, over a month after the statutory deadline. And 6% of authorities said they didn’t hold the information at all (because they, surprisingly, don’t record the referrals they receive). Which leaves 10% who refused our request on cost grounds. If our request had been simpler, this number of refusals would likely have been smaller.

However, this result is in itself interesting: at least 16% of local authorities responsible for handling safeguarding referrals either don’t record them, or record them in such a way that it would take more than 18 hours of officer time to report how many they received in a given year, or how many relate to girls.

If the government is serious about halving violence against women and girls within a decade, this is precisely the sort of data local authorities will need at their fingertips, in order to monitor progress and allocate resources. The fact that it’s effectively inaccessible to 16% of them right now is a worry.

Combining data for new patterns and new questions

Remember how I mentioned we were adapting the Local Intelligence Hub for EVAW’s needs?

With our FOI data extracted through WhatDoTheyKnow, we were able to very quickly load it into a prototype VAWG Data Hub. Alongside it, we loaded in a whole new area type to filter by—“Policing areas” or Police & Crime Commissioners—as well as some examples of crime prevalence data (the number of reported VAWG-related incidents, by policing area) and public policy guidelines data (the Council Of Europe’s recommended minimums for VAWG service provision).

Thanks to my colleague Alex’s improvements to our TheyWorkForYou Votes infrastructure, we were also able to make quick work of importing VAWG-related MP data into this new hub – including VAWG-related parliamentary groups that MPs might be sitting on, or relevant votes and motions they’d supported.

Plus, of course, there was all the usual MP and area data that campaigners and public affairs teams have already found so useful on the Local Intelligence Hub – things like election results, public attitudes polling, and income and poverty indicators.

Data in action

With the data in place, it was possible for us to give EVAW’s member organisations a demonstration of how they could use a data hub like this as part of their campaigning, fundraising, and policy influencing work. For example, to find the council areas with the most school safeguarding referrals for girls and also the highest overall deprivation:

Or to find the MPs with the strongest support for VAWG prevention, but in constituencies with high VAWG prevalence:

All of the data we demonstrated this Autumn is still a work in progress, but it was reassuring to see almost 70% of members on a recent demo call saying that a VAWG data hub like this would “definitely” be useful to them in their day-to-day work.

We look forward to honing the VAWG Data Hub further with EVAW and their members, to make sure we’re asking the right questions, and presenting an accurate picture of the VAWG landscape.

Header image: Khyati Trehan, for Google Deepmind – Pexels Free License.