Scrutinising governments’ use of AI is turning out to be one of the pressing demands of our time, so we’re very glad to have received funding from the Joseph Rowntree Charitable Trust that allows us to conduct meaningful and sustained work in this area. The project has already kicked off with fact-finding, consultation and exploration, and will culminate in the creation of a service for civil society, journalists, authorities — and individuals affected by AI-based decision-making.

Artificial intelligence systems are increasingly used in democratic decision-making from welfare and healthcare to policing, with significant potential for both positive and negative impacts on individuals, society and the environment. 

The effective use of data-driven AI systems in government and the public sector could transform the delivery of public services and support evidence-based policymaking, bringing benefits to citizens, saving costs and driving policy innovation across many domains. 

But a data-driven algorithmic approach also has known risks, including that of reproducing or even amplifying unfair or discriminatory outcomes that are represented in the source data. We want to know how transparent evaluations of AI deployment in government can support both effective use and democratic accountability. 

This is not an abstract question, nor one for a distant future. A recent LGA survey revealed that a significant percentage of local authority respondents are using AI in health and social care — 44% of them in services for adults, and 31% in services for children. Large Language Model (LLM)-based tools are being piloted across diverse applications in government, from social housing and planning to the DWP.

As a new generation of LLM AI systems come into use in which the rules of operation are not explicit, explanations of behaviour are challenging, and training data is not available, the need to both assess and demonstrate the fairness, performance and value of systems through adequate and transparent evaluation of their effectiveness and impact becomes much more important. 

Across the two year span of this project, we’ll work with others to tackle the practical questions of how we collectively evaluate government AI systems and their impact in a way that supports fairness and benefits society as a whole. Based on this, we’ll go on to develop an accessible digital service, fostering transparency and accountability where AI systems are used. 

That last bit is crucial: this is not just a research project. The initial discovery will ensure that the service is designed to be as effective as possible, with primary users being civil society organisations and journalists, alongside public authorities learning from their peers — but it will also be designed with public participation in mind, providing structured pathways for affected individuals. 

We’ve started already. Last week, we shared the outcomes of a roundtable, the first activity in this project, helping us to understand what useful infrastructure already exists for scrutiny, where the gaps are, and where there are shared needs across civil society. 

There’s plenty more to come: watch this space, and do sign up for updates (check “artificial intelligence” as an interest) to receive our newsletter direct to your inbox.

 —

Image: Jamillah Knowles & Digit via Better Images of AI (CC-by 4.0)