In our latest online webinar, we convened three experts to speak around the topic of how Freedom of Information works in practice – in other words, how does the law work when it comes into contact with the real world?

Speakers were:

• Toby Mendel, Centre for Law and Democracy
• Giovanni Esposito, Université Libre de Bruxelles
• Mária Žuffová, European University Institute

Sign up for updates and we’ll let you know when the next events come up. Don’t forget to check the box ‘Conferences and events’ if you want to know about every event we put on, and/or any of the other topics you have an interest in.

Music: Grand_Project from Pixabay
Image: Nick Fewings

---

Transcript

Myf Nixon 0:00
Hello again. Today we’re sharing our latest online webinar, which was called “Putting transparency to the test: evaluating FOI in practice”.

Myf Nixon 0:10
So the idea here was to look at how Freedom of Information is supposed to work on paper, and how it actually works when it comes into contact with the real world.

Myf Nixon 0:21
You are about to hear from three really expert speakers. First of all, there was Toby Mendel from the Centre for Law and Democracy, and they’re an interesting organisation who have run the Right to Information Rating since 2011.

Myf Nixon 0:39
This is a kind of ranking of all the countries in the world. You can see at a glance which countries are performing well across all sorts of different indicators around transparency and Freedom of Information, and which ones are not performing quite so well.

Myf Nixon 0:53
And Toby will explain how a ranking can have really interesting effects, which is something that was particularly of interest to us at mySociety as well, when we think about things like our Council Climate Action Scorecards, which have had great knock-on effects amongst councils hoping to compete to get to the top,

Myf Nixon 1:11
and also looking back in time at our Write To Them ranking in which we used to rank MPs by their responsiveness.

Myf Nixon 1:21
After Toby, we hear from Giovanni Esposito from the University of Brussels, and he will be describing a set of field experiments that he conducted in collaboration with Transparencia.be, and they are an Alaveteli Freedom of Information site operating very actively within Belgium.

Myf Nixon 1:42
Giovanni’s experiments were looking at what factors might make a difference to responsiveness when you put in a request for information. So they asked for the same document from several different municipalities. And you can find out what the results were by listening to this podcast.

Myf Nixon 2:00
And then finally, Maria Žuffová of the European University Institute shared some research which was again of great interest to us at mySociety, but we think will also be of great interest to anyone with a curiosity about humankind.

Myf Nixon 2:16
And this research goes into what the UK public actually want to know. And she based it all on WhatDoTheyKnow requests going back for years. So some real analysis going on there, and some great insights.

Myf Nixon 2:30
Over to the three speakers, and I hope you enjoy listening.

Toby Mendel 2:34
We launched the RTI rating in 2011 so just a year and a half after ClD was founded. It had, to me, been apparent for quite a long time that we were seeing this massive growth in the number of RTI laws that were in place and being adopted each year.

Toby Mendel 2:56
A bunch of new laws coming onstream. And as civil society advocates ourselves and others were spending quite a lot of time pushing for stronger laws and advocating around that and things and the the idea that a sophisticated assessment methodology for seeing how strong laws were would be an enormously beneficial tool in the sense of, first of all, it would be a scientific and standardised and objective, if I can put it that way, way of assessing a law.

Toby Mendel 3:31
There are nothing like ratings to inspire countries to move up and push up. Over the years of the RTI rating, I have used it endlessly in that way, and sort of say to country in this region, your neighbour over there, is like, six points higher than you. What are you going to do about that?

Toby Mendel 3:48
And it just, it’s amazing how easy it is, in a way, to get the policymakers and decision makers to bite on that, whereas if you just say, Oh, the law is weak, you should improve it, that’s a lot less convincing for them.

Toby Mendel 4:01
So the need for this and the benefits of this were very, very apparent to me. I pitched the idea to Helen Derbyshire at AccessInfo Europe, who, of course, tragically died a couple of years ago, and she immediately saw the logic of it, and we worked together on it.

Toby Mendel 4:19
We basically put together a draft methodology, so essentially a set of questions or standards against which we would evaluate a piece of legislation. And the idea was that those standards would probe into everything that a good law should have.

Toby Mendel 4:40
So hopefully we more or less achieve that. We put in place an advisory team with leading RTI experts from around the world, some of them advocates, some of them academics, some of them practitioners, and even on the government side and things. So we had a fairly balanced all round advisory team.

Toby Mendel 5:01
We piloted the methodology on a few laws to see how things were shaken out. The challenge, of course, was then, at that time, there were, if I remember correctly, around 85 laws globally, and which is a lot less than 140 of today, but it’s still quite a few laws, and so we then had to go through and assess all of those laws.

Toby Mendel 5:25
And we also have another part of the methodology whereby we have a local experts review it. Because there may well be, for example, court decisions that we’re unaware of that, even though the language as we read it suggests an interpretation going in that direction.

Toby Mendel 5:41
They may put a more progressive interpretation on it, or they may apply a constitutional interpretation. So they may change the meaning of the text through constitutional interpretation, maybe amendments or regulations that we didn’t pick up on, or that we didn’t manage to get in one of the languages that we can work in, because we we have a few languages available to us, but obviously not all of the languages in the world.

Toby Mendel 6:04
So we have a local reviewer to make sure that we’re picking up on all of those nuances.

Toby Mendel 6:12
We didn’t manage to find local reviewers for every single country, but most of the countries have been reviewed by local reviewers. So we go through that procedure before we finalise the weighting and put it out, I would say, in terms of changing over time,

Toby Mendel 6:25
We have indicators, and most of the indicators get a score of one or two points, and a few of them have slightly more points depending on the design of the indicator.

Toby Mendel 6:35
We have not changed the base indicators since it was first developed. And that would be, I mean, we, we sort of would aspire to that, but that would be a huge project. We would have to get quite a lot of funding.

Toby Mendel 6:47
We would have to go through the whole procedure of renewing the indicators, but then we would also need to go back and we rate every country right. So that’s not a small thing to do, but we have adjusted the scoring interpretation or protocols over time.

Toby Mendel 7:05
So we sort of realised that this indicated we were a bit loose on some part of it, and tightened it up. We’ve gotten more scientific or more concrete, or even in some cases, we’ve changed a little bit the way we score.

Toby Mendel 7:15
So we’ve had some flexibility in that respect, yeah, and in terms of strengths, I mean, really, the biggest strength is this advocacy potential.

Toby Mendel 7:26
I mean, I’ve lost count of the number of countries where I’ve used that feature. And gone to policymakers or civil society. And also, I mean, it allows civil society groups, you know, we do a rating of draft laws, especially, and then the local civil society groups, which are advocating for can say, OK, here are exactly where the weaknesses are.

Toby Mendel 7:46
These are important to us. We’re going to focus our firepower on that and try to get changes for that.

Toby Mendel 7:52
So, I mean, it’s both in the geeing up by comparative reference, but also in terms of pinpointing exactly the weaknesses and things, I think it’s been been very, very strong; very, hugely beneficial in terms of weaknesses.

Toby Mendel 8:07
I mean, one of the challenges is that we’re putting a template on the whole world in terms of what should be in a law. And, I mean, that doesn’t always fit.

Toby Mendel 8:17
And we only credit issues, which are legally or at least binding policy driven. So if you don’t have a binding policy on something, at the minimum, you’re not going to get any points.

Toby Mendel 8:30
So for example, one of them is our public bodies required to train their officials. So in a lot of Western countries, there’s no rule about that.

Toby Mendel 8:40
But of course, officials are actually trained very well, much better than in some countries where there is a rule on that, but the rule isn’t followed.

Toby Mendel 8:49
But we’re still not giving them the points because… you know, there are some some frictions, I would say, when you try to apply a template to the whole world, but by and large, I would say it’s been a very successful piece of work.

Toby Mendel 8:59
When the rating first came out, and Western countries were way down,, and there were a bunch of Western people, including commissioners, government officials and whatever, who were either incredulous or unhappy or whatever it was with that.

Toby Mendel 9:14
When we come back at them and say, Look, basically, your law is weak, we’ll do a nuts and bolts on it. We’ll go, bing, bing, bing, bing, and show you how your law is weakened.

Toby Mendel 9:23
That. The problem there is that your law is weak, and if you want to not be at the bottom, fix it, you know.

Toby Mendel 9:28
And when we when we come back like that, you know, defending our methodology because it is defensible. And there may be five points that they could have had that they should know, whatever, but ultimately, you know, countries that are doing poorly are weak laws, and they need to fix it. That’s the end of it.

Toby Mendel 9:46
Implementation is much, much more complicated business with, not to exaggerate, but a million different moving parts.

Toby Mendel 9:55
For example, the primary duty holders under these laws are individual, public authorities, each one of which has to separately, implement the law, and each one of them will be doing better or worse in any given country.

Toby Mendel 10:08
And so how do you assess all of that? So the RTI evaluation, as we call it for short, and that’s the website name as well. We don’t come up with a ranking. We don’t give a score. We give a red, yellow, green, which gives a general impression of you doing OK, or not too well or better.

Toby Mendel 10:25
Because it’s not that scientific. We rely on a selection of public authorities, because at the end of the day, you can’t assess all of them, because there are at least hundreds in every country, and often thousands or even more than that.

Toby Mendel 10:38
So you just can’t, like it’s not feasible to measure them all, not properly anyway. So we grab a selection of them, we let local reviewers, who the methodology has to be driven by local actors.

Toby Mendel 10:51
So unlike the RTI rating, which we essentially do in house, so we have to work with local actors, we let them do that.

Toby Mendel 10:57
They do test requests as part of the thing. So we have four assessment areas. One is central measures. So the oversight system.

Toby Mendel 11:04
One is institutional measures. So whether public authority, individual public authorities have adopted appointed information officers, train them, manage their records, do that kind of stuff, proactive disclosure and reactive disclosure.

Toby Mendel 11:17
So the two main vehicles for actually disclosing it for proactive disclosure, we look at, is it proactively disclosed and reactive disclosure?

Toby Mendel 11:25
Among other things, we do test requests. So obviously, and we let the local actors select the test request. So there’s a huge variance.

Toby Mendel 11:33
You know, they select some public authorities. They may select better performers. Then another country may, by chance, select poor performers.

Toby Mendel 11:42
We can’t really compare that, and they may put harder questions or weaker questions or whatever. But what I will say is that it is a much more in-depth and complicated methodology.

Toby Mendel 11:51
It takes us a long time to apply it and things, but it does. It’s a deep dive. It gives you a pretty complex… like every question that should be asked could be asked.

Toby Mendel 12:02
And we, by the way, we had a whole system for developing it and piloting it and improving it as well, which I won’t get into too much detail about, but it really does give you a pretty good picture of what’s working and what’s not working in your system, because it really looks at everything.

Toby Mendel 12:20
We’ve applied it in about eight to ten countries. So it’s a lot more difficult. Requires quite a lot of resources to apply and things like that.

Toby Mendel 12:30
And I can say that firstly, we’ve been quite pleased with the way that public authorities have and governments have engaged with this.

Toby Mendel 12:39
You know that, by and large, you know, we’re not, we’re not trying to hit them over the head with a hammer. I mean, we’re trying to help them improve their system. And oversight bodies in particular, have, by and large, engaged quite well with this. So that’s nice.

Toby Mendel 12:53
The issues that we’re exposing, while there’s been a little bit of debate about them, by and large, people accept that those are issues, and the things that do need to be improved.

Toby Mendel 13:03
Of course, to the next stage of it, whether you actually improve them, and that sometimes requires a bit more resources and things, but I would say that overall, we’ve had quite a good engagement with it.

Toby Mendel 13:13
And it’s led to in our in our projects on that side, we usually try to have additional resources for remedial actions following the the test.

Toby Mendel 13:25
So you assess implementation, you find out where things are weak, and then you do something to address those at least. It’s been quite successful. And the biggest challenge for us with that is getting the resources to apply anymore in more places. It’s just, it is pretty expensive.

Toby Mendel 13:44
Yeah, I can’t stress enough the importance of an empowered and independent oversight system, Information Commissioner It really is the make or break.

Toby Mendel 13:59
There’s almost a kind of a bright blue line, you know, between those countries which don’t have that and those countries which do.

Toby Mendel 14:04
It doesn’t mean that every country which has one of those is doing great. That would be a little too optimistic, but their chances of doing better are very strong. So that, that, for me, is, is a huge issue.

Toby Mendel 14:18
Of course it’s expensive for countries to stand up those institutions, I would argue that they’re going to save the money and addressing corruption and inefficiency.

Toby Mendel 14:27
And you know, maladministration is… not everything is about corruption. Sometimes just things aren’t working properly because people aren’t doing their job properly, or whatever.

Toby Mendel 14:38
Not everything is a grand, grand corruption issue, but, but this helps expose all of that, and being open engages citizens in your development process.

Toby Mendel 14:45
I think that just leads to better development outcomes. So I think the money is easily recouped, but it’s still a cut off the budget, if you will.

Toby Mendel 14:53
So countries are sometimes a little resistant. We just did a major exercise where we put the same two requests for information to as many of the 140 countries with laws in the world as we could.

Toby Mendel 15:07
And I can say that 38% of the responses that we got were mute. No refusals, nothing. Nobody said anything. Nobody came back at all.

Toby Mendel 15:16
I mean, we had a timeline of 30 days, which is not the timeline in every piece of legislation, although very few laws go beyond that, like this.

Toby Mendel 15:24
A handful of them allow longer responses to that. We got a couple of formal extensions, but only a couple, you know.

Toby Mendel 15:31
So, I mean, a mute refusal is a killer, right? So that’s the biggest problem. We did not get a lot of people claiming exemptions or saying, No, this information is secret.

Toby Mendel 15:41
We did not get a lot of charges. We did not get a lot of, as I said, exemptions. And we got a lot of positive responses.

Toby Mendel 15:47
We got some information in 51% of the cases. So 51 plus 38 putting it close to 90. So it’s only a 10% that fell into other categories, but 38% refusal.

Toby Mendel 15:58
So I mean that probably those public bodies didn’t appoint proper Information Officer, didn’t have an Information Officer, trained person, wasn’t doing their job, basic, basic things.

Toby Mendel 16:09
RTI evaluation, we are very, very open to civil society groups leading in terms of the implementation of that. And so if there are groups – we don’t have resources to support that around the world – but if there are groups that are interested in doing it and have the resources themselves, we will provide them with the training and their you know, the tools and the information that they need.

Toby Mendel 16:33
We really welcome that. So please anyone out there who’s listening to this, and we also very much encourage oversight bodies to lead on that process again, if they can, you know, stand up a few resources for it.

Toby Mendel 16:46
We estimate it takes about 30 person days if you hire a consultant to manage the whole process. Soit’s a piece of work, but it’s not a an enormous piece of work. And we will support throughout the thing. We’re very interested in that.

Julia Cushion 17:01
And now over to you, Giovanni.

Giovanni Esposito 17:03
It’s really a pleasure to be with you here today and share my work. OK, so the focus of the presentation I’m giving you today is related to this project we have here at ULB University in Brussels, which is called TransAct: transparency in action, and it addresses the issue, the critical issue that Toby Mendel mentioned before, which is how to understand, study, explore the actual implementation of FOI laws.

Giovanni Esposito 17:42
Okay, so this is the scope of this project. Let me start with few background information about this project. As I said, this is part of the work I’m conducting here at ULB as a director of this small but very interesting research center, which is called Cepap, so that’s Vincent Mabillard – so it’s a research unit on public policy and administration at ULB.

Giovanni Esposito 18:12
And one of the research axes we have is government transparency and Freedom of Information law, an axis that I’ve set up with with my colleague and friend, Vincent Mabillard, a professor here at ULB.

Giovanni Esposito 18:29
And what we what do we mean by government transparency? In in our work, we mean we refer to this as the practice of making government’s actions, decisions and governance processes accessible to the public.

Giovanni Esposito 18:46
And why this matters for a research unit in Public Policy and Administration, because when these policies are effective, that information asymmetry between citizens government officials is reduced, or anyways, mitigated.

Giovanni Esposito 19:01
And this has a number of positive effects in terms of governmental accountability. So it improves if you want the functioning of democratic systems in this sense.

Giovanni Esposito 19:11
And it also has positive effects in terms of efficiency and effectiveness of the public administration positions. It leads to a reduction of corruption phenomena, in this sense.

Giovanni Esposito 19:26
So in this sense, it can improve to the efficiency of public administration operations. So within the framework of this research axis that focuses on government transparency, here at ULB, we have set up this project, which is called TransAct.

Giovanni Esposito 19:49
A huge support in the setting up of this project was provided by our fellow colleagues in the University of Naples who particularly provided us with expertise on field experiments.

Giovanni Esposito 20:04
And we’ll come back in a little bit on how field experiments are useful for measuring the what we call the implementation gap.

Giovanni Esposito 20:16
And as also was mentioned by Julia in the introduction, this project relies on citizen science. What it means is that in the research process there are certainly professional researchers, academics, and you have here the broad network that we have, and that has enabled us to run field experiments beyond the Belgian context.

Giovanni Esposito 20:37
OK, we’re now running similar experiments in Switzerland, Romania, we’re preparing one in South Africa.

Giovanni Esposito 20:44
But when we rely on citizen science methodologies, non professional researchers are involved in the design of the research process and in the collection of the data needed to accomplish this research, and so we work a lot with civil society organisations.

Giovanni Esposito 21:07
The project that I will present you today within TransAct is our field experiment in Belgium. And this was not possible to be accomplished without the support of Transparencia.be, which is the organisation managing a Alaveteli platform here in Belgium, and they were crucial to work and to design and implement the field experiment here in Belgium.

Giovanni Esposito 21:32
But why we are so interested in in FOI laws and that implementation? Well, this slide is particularly explicative of this interest. And as Toby Mendel was saying, also before, over the last years, there has been clearly an increasing adoption, formal adoption of FOI laws around the world, across both democratic and non democratic countries.

Giovanni Esposito 22:02
So regardless of the political system adopted within the country, these laws were increasingly adopted by legislators, and you see on the right hand that a wide majority of countries around the world has nowadays an FOI law, but the literature on these laws, is also clear about the fact that governments frequently comply with their own FOI legislation.

Giovanni Esposito 22:31
So in practice, when a citizen files an FOI request to a public authority, there might be no response, partial response, a refusal in disclosing information.

Giovanni Esposito 22:44
And so this leads to low de facto compliance rates with the existing legislation. And this is why, and this is what we exactly call the implementation gap.

Giovanni Esposito 22:55
So the implementation gap between the formal adoption of the law and the implementation in practice of the law, if you want, there is a discrepancy between the de jure dimension of the law, the law on paper, the one adopted by a country, and the de facto dimension of the law, how the law is practiced in the day to day activities of public administrations.

Giovanni Esposito 23:19
And so what we want to do with our project, and what we are doing actually, since 2022 is to measure this implementation gap, so to size it, but also to explain why this implementation gap exists.

Giovanni Esposito 23:36
As Toby Mendel was saying there are several indicators to measure FOI laws: he presented the RTI index. There is also the CPI index, which is the Corruption Perception Index with the transparency index.

Giovanni Esposito 24:00
But perhaps you might have noticed while I was going through the slides that the ranking of countries according to the index was changing, and so here there’s an issue with the measurement of transparency.

Giovanni Esposito 24:15
And this becomes clear for issues if we look at what are the top five countries when we use the RTI index that mainly measures the de jure dimension of the law. And when we look at countries and FOI systems through the lenses of an index such as the transparency index, which tries to combine both the de facto dimension the de jure dimension.

Giovanni Esposito 24:41
So this is why, at one point we were with my colleagues and folks, let’s say, from the network, we got interested in analysing and measuring the de facto dimension.

Giovanni Esposito 25:00
Intention of the law, and so we found that very nice way of doing this is the use of field experiments.

Giovanni Esposito 25:07
OK, so what is a field experiment? It’s pretty intuitive.

Giovanni Esposito 25:13
Experiments happen, traditional experiments happening in in the lab. Field experiments happen in the field, so directly in the in real world settings.

Giovanni Esposito 25:24
So in relation to FOI laws and the implementation of FOI laws, it consists the field experiment in submitting systematically request to public administrations following a protocol, experimental protocol.

Giovanni Esposito 25:37
And I’ll be back in a little bit about this, observing the responses of the public authorities and occasionally modifying the content of the request to book the impact of these modifications on the responsiveness of the public authorities.

Giovanni Esposito 25:53
And so far, there have been many experiments, several field experiments, conducting in different countries.

Giovanni Esposito 26:02
The one in Italy was led by our friends from the University of of Naples.

Giovanni Esposito 26:08
We actively contribute, working on these experiments, and the one in Belgium that I will present you now, as you see, Belgium was not among the countries covered but by these approaches to study the implementation gap.

Giovanni Esposito 26:29
This was interesting for us, because Belgium has had an FOI law since 1994. Transparency is part of the Belgian Constitution, Article 34 and regional laws have been adopted across the three regions to transpose the constitutional principles.

Giovanni Esposito 26:50
And so nowadays, there are minimal differences among the regional laws, but an individual based on these laws can submit written requests, including via mail, to public administrations to have access to to government held documents.

Giovanni Esposito 27:07
There is no need to demonstrate a specific interest, and public administrations have up to a maximum of 45 days to make the request document available to the requester unless, of course, the document is not covered by legal exemptions of of the law.

Giovanni Esposito 27:27
And so with Transparencia.be, we established contact with Transparencia.be who, beside managing the platform in Belgium, is also active in lobbying, at advocacy on these matters in Belgium, and we decided to set up a field experiment.

Giovanni Esposito 27:46
So we had two main questions. The first one was, do Belgian municipalities respond to FOI requests and provide the request information?

Giovanni Esposito 27:55
Second question here is, do they discriminate among requesters? We know from the literature that has used field experiments that different requesters’ profiles can trigger, let’s say, different responsive behaviours in the administration that they approach.

Giovanni Esposito 28:18
And so we decided to send requests in the Belgian experiment from three different profiles,: a common citizen with a generic standard Belgian name; a French speaking university professor known in the French speaking environment for his research on transparency – in our case, it was Vincent Mabillard; and requests sent from the transparencia.be platform, so requests sent by individuals who are a part, let’s say, of a civil society environment that has an interest in FOI laws.

Giovanni Esposito 28:58
We also knew from the literature that what can influence the likelihood of answering an FOI request is whether the requester mentioned or not, the law in force in their countries and granting them the right to access the document.

Giovanni Esposito 29:17
And so we decided that part of the request of the common citizen and the university professor were also sent either in parts by mentioning the law and in other parts, without mentioning the FOI law.

Giovanni Esposito 29:31
We sent this request to all 581 municipal administrations of Belgium, and we had two experimental objectives.

Giovanni Esposito 29:41
So the first one was to understand how municipalities react to different types of requested profiles.

Giovanni Esposito 29:46
The second one was to test whether explicitly mentioning the FOI law influences the likelihood of responsiveness, of a response by municipal administration.

Giovanni Esposito 30:00
With Transparencia, we decided to ask for this document: the agenda of the last municipal council meeting with its annexes, for two reasons.

Giovanni Esposito 30:12
One is methodological. This document is a standard document available in all Belgian municipalities, and so we could ask for the same document from all administrations.

Giovanni Esposito 30:23
The second one was related to the previous advocacy work of transparencia.be, which was particularly interesting and related to having public and municipal administrations in Belgium making this document available to citizens when this was requested by the citizens, and so they told us that a nice test would be around this this document.

Giovanni Esposito 30:51
OK, so here you see our experimental design. OK, so we based it on the story I told you before, and the design we did with Transparencia.

Giovanni Esposito 31:01
So in this sense, this was a citizen science experiment, because we designed the experiment with Transparencia.be.

Giovanni Esposito 31:09
We had five experimental groups: a control group; the common citizen baseline scenario, not mentioning FOI; then the other four groups add some variations to the to this baseline scenario.

Giovanni Esposito 31:22
And these are what we call the treatment groups. So a citizen mentioning FOI in the request. So the idea is to look at the effect of invoking rights when doing the request.

Giovanni Esposito 31:33
The third group: Professor, the effect of status.

Giovanni Esposito 31:37
The fourth scenario, fourth group fourth treatment group, Professor mentioning the FOI law, the combined effect of invoking rights and bringing in status.

Giovanni Esposito 31:47
And the fifth group, the advocacy organisation. So requests sent from transparencia.be. When they arrive to the public administration that is the signature of the platform, so they know that it’s coming from the environment of transparencia.be.

Giovanni Esposito 32:04
And then, as usually doing in experiments, we randomised the allocation of the requests, which stratified the randomisation, because we want to check if there are any effects related to the region in which Tthe municipality is placed and the sites of the city, and these are standard approaches in the literature that deals with countries such as Belgium, which are federal countries, and so have sub national levels of of governments.

Giovanni Esposito 32:36
So the emails were sent in English and Dutch, depending on the language on the municipality. And we did this with the help of students from my course Innovations in Policy Evaluation.

Giovanni Esposito 32:51
Part of the teaching I do to my students is field experiments with an application to FOI laws. And so we decided to do together this experiment as it was in 2023 as a part of the of the class.

Giovanni Esposito 33:07
And so they were sending the emails and they coded under our supervision. Our colleagues from Naples were also involved. This is a picture from the class.

Giovanni Esposito 33:19
We coded the answers using five modalities – five categories: no reply, denial, unsatisfactory reply, partial reply and complete reply. Okay, so five modalities to code the the answers.

Giovanni Esposito 33:35
And these are the results. So first of all, the 36% of municipalities, regardless of the treatment, so of the of the the experimental group they were allocated, did not respond, whereas about 30% provided at least a partial reply.

Giovanni Esposito 33:58
What is a partial reply? Perhaps they send the agenda, but not the annexes. This is an example, where we asked for all the annexes, then they sent only 30% of the annexes.

Giovanni Esposito 34:14
Secondly, while almost 50% of small municipalities did not provide a reply, over 50% of large municipalities provided at least a partial reply.

Giovanni Esposito 34:27
So what is our hypothesis? These are just descriptive statistics, so it’s hard to have, statistically, causality that can be established here. We’re working on that now.

Giovanni Esposito 34:39
But the hypothesis – I mean, one interpretation is that municipal size can can impact the response behaviour, with larger municipalities being more responsive and likely to provide at least partial disclosure compared to smaller municipalities.

Giovanni Esposito 34:55
So basically, large municipal administrations, because perhaps they have more resources, they perform better in in answering.

Giovanni Esposito 35:04
The second aspect, regional dimension. I said that there was the regional stratification in the randomisation. This is the results: municipalities providing at least partial replies, so partial or full.

Giovanni Esposito 35:22
In Brussels, are the 58% in Brussels region; Flanders region, 46%; Walloonia 21% -so one way of interpreting it is that perhaps regional context can impact response and disclosure rates.

Giovanni Esposito 35:37
Okay, so different governance systems in regional administration can have an impact.

Giovanni Esposito 35:42
The fourth aspect is the profile of the requesters, so we see that common citizens mentioning FOI laws in their request have the highest disclosure rates, whereas professors mentioning FOI ,and requests coming through Transparencia.be have the highest no reply rates.

Giovanni Esposito 36:06
Okay, so municipalities. One way of interpreting this is that municipalities have higher resistance to reply to FOI requests coming from individuals that they perceive as knowledgeable about FOI requests, so somehow they perceive this request riskier. They see more risk in these people.

Giovanni Esposito 36:25
Perhaps they could do some inquiry through the request. And the last aspect that we decided to check was this one.

Giovanni Esposito 36:38
So I told you that we had group of students sending the requests. And we noticed that some groups were very motivated. So they were very fast in following up with administrations asking for more information about the request.

Giovanni Esposito 36:53
Others were just following the baseline of the protocol, the experimental protocol, so we decided to measure this commitment, using the grading system of Belgian education universities.

Giovanni Esposito 37:09
And what we found was that higher levels of citizen commitment, so the grade correlates with municipalities’ higher response rates and higher disclosure rates, and we also see that higher commitment correlates with higher citizen bureaucrat interaction, so more interactions between the requester and the administration on the other side.

Giovanni Esposito 37:35
So this was interesting, because this suggests that citizens’ commitment in submitting FOI requests can influence municipalities’ response behaviour.

Giovanni Esposito 37:46
So I move towards the end, and what are the key takeaways of this experiment, and what are the hypotheses we’re working now with with our colleagues?

Giovanni Esposito 37:58
First, regional differences matter. So there are distinct governance models that can impact how the laws are implemented. In practice, the municipal size matters, and we see that larger municipalities perform better in implementing the law.

Giovanni Esposito 38:14
OK, so there is here, perhaps something to be understood in terms of equipment and resources in handling FOI requests.

Giovanni Esposito 38:23
The requesters’ identity plays a role as well in the implementation of the law. In practice, we see that actors seen as knowledgeable in FOI, because they are perhaps perceived more risky are those who are who face more difficulties in accessing the law.

Giovanni Esposito 38:47
This is clearly problematic, especially for journalists, for issues, especially if we consider the periods we are living now in democratic societies.

Giovanni Esposito 38:59
And fourth, the citizen’s engagement. This is very interesting, because on the one end, we usually look at the supply side of transparency policy. So basically, the information providers, the public administration. How can we improve the supply side? How we improve the way that information is provided.

Giovanni Esposito 39:20
But here we see that the demand side is also important. So basically, the information seekers and how the information seekers deploy strategies to secure the information.

Giovanni Esposito 39:34
Last slide about the next steps: as I said, now we’re trying to to have a better understanding of the hypothesis that I showed you while I was mentioning the results, and we’ve also launched more experiments.

Giovanni Esposito 39:47
As I said, one was just accomplished in Switzerland. One is going to start in Romania, and we’re planning one in South Africa. Besides that, we’ve decided to retrieve information also from Alaveteli platforms using Transparencia.be as a case study, and we want to run the same type of study.

Giovanni Esposito 40:08
Basically, we want to see how the request, topic, tone, the recipient, authority influence the likelihood of receiving a reply or a refusal or a silence on the side of the administration. And I stop here. Thank you for your attention.

Julia Cushion 40:28
Thank you so much, Giovanni, that was fantastic. I will now invite Maria to give her presentation.

Mária Žuffová 40:36
Thank you so much. Yes. So I’m looking into the demand side.

Mária Žuffová 40:39
So my main research question was: what is it that the public is actually interested to know from information requests?

Mária Žuffová 40:47
And I started this looking at the requests sent to the UK central government through the WhatDoTheyKnow platform.

Mária Žuffová 40:55
So just to give you a brief overview of the presentation: I’ll present the main research question. I discuss a bit the literature that I draw from, but very briefly, and then talk about the data I used, how I analysed them, and then I get to the findings.

Mária Žuffová 41:14
So the main research question that somehow drove this research was exactly this: what is it that the public want to know from the from the government, and in this case, from the from the UK Government?

Mária Žuffová 41:28
And I was primarily interested whether it is this accountability seeking, like holding the governments accountable, that drives citizens to submit Access to Information requests, or are the interests of requesters much broader?

Mária Žuffová 41:47
And how I approach Access to Information in this work – at the beginning I started with this very common definition of Access to Information as access to complete accurate public interest information.

Mária Žuffová 42:04
But I put the public interest into brackets, because my findings actually contest this this definition to some extent, and I’ll get back to this when I discuss findings.

Mária Žuffová 42:17
I was mostly interested in demand, because I feel the research on transparency – and I consider myself a researcher focusing mostly on the politics of transparency reforms -we focus on many different areas, but much of this research, including my own, was always based on this assumption that Access to Information policies serve a government oversight function, so they are mostly envisaged and adopted with this intention to hold government and other powerful actors accountable.

Mária Žuffová 42:55
And that has also implications for the use. So there is this underlying assumption that Access to Information, legislation and requests, they would be submitted mostly by investigative journalists or activists exactly to expose corruption or mismanagement of public resources.

Mária Žuffová 43:17
So there has been always this transparency /accountability relationship, but this has been very rarely tested, so I was curious to look into the information request for this precise reason, to see if this is actually the case.

Mária Žuffová 43:34
If citizens submit information requests with this in mind. And I think this is not only because of the lack of interest in the topic, but also because of the difficulty to study this, because there are very few public administration portals that make this data available.

Mária Žuffová 43:52
I was aware of the Mexican government portal that allowed to see all the interactions between requesters and public servant and to see how they responded and when they responded. So that was possible, and there has been fantastic research done by Dan Berliner and his co authors on Mexican information requests.

Mária Žuffová 44:16
There’s something by Dokeniya on Albanian requests, and also something on the US. But in general, this is very difficult to study because there’s a lack of comprehensive information.

Mária Žuffová 44:30
And when I looked at the information demand, I found this information gathering matrix, a conceptual framework developed by Greg Mitchener and Benjamin Worthy quite useful because they argue that actually the uses might be very heterogeneous, and requesters might submit requests in public or private interest and in terms of content, it might be political or completely non-political.

Mária Žuffová 45:02
And I’m mentioning this because it was very useful for me. Then when I looked into the requests to somehow categorise them, why I decided to focus on the UK, and WhatDoTheyKnow?

Mária Žuffová 45:16
So also for what Giovanni mentioned that in the UK. I mean, I think it’s one of the cases where there’s also quite big demand, but also quite good supply and access to information.

Mária Žuffová 45:32
Legislation in the UK has been widely used. And I think something that helped, to some extent, also to make it very well known as a legislation was also an MPs’ expenses scandal. And so there has been an increase in in requests to the central government, at least, based on the on the statistics that the Cabinet Office publishes on an annual basis, but at the same time, also, as Giovanni mentioned, that one thing is that the citizen engagement can drive responsiveness and better responses.

Mária Žuffová 46:13
It also works the other way around. The better responses then encourage citizens to engage more with the law so and in the UK, still, I mean, although the rate of information requests that are being responded on time is decreasing every year, that’s also available in the Cabinet Office annual statistics, but still, I would say that the rate of 76% being responded on time is quite, quite decent.

Mária Žuffová 46:40
So for me, it was mostly this use and also the supply of information that makes it interesting, because there’s a wealth of data, and it’s also interesting because information requests are real world interactions, so there’s very little bias that comes that comes in.

Mária Žuffová 46:59
And so what data did I use and analyse? So I have to first of all thank mySociety, because my research wouldn’t be possible without them and without the WhatDoTheyKnow platform.

Mária Žuffová 47:13
So the data I analysed is basically around 40,000 almost 40,000 information requests that were sent to UK central government bodies via WhatDoTheyKnow within a decade, from 2008 to 2017.

Mária Žuffová 47:29
And I’m looking in the full text of requests and the date it was submitted. Mostly these two variables and just maybe some descriptive data in case you’re interested.

Mária Žuffová 47:44
So the institution or the government bodies that received the highest number of requests was the Department of Work and Pensions and Ministry of Defense, Ministry of Justice, Department of Education, Department of Health and Social Care, and I haven’t included all of them, because some really receive – there are actually a few that receive a handful of requests per year.

Mária Žuffová 48:11
And I also must disclose that I haven’t included the Home Office for privacy reasons, because there were too many personal data in in the requests themselves, so the Home Office is completely excluded from the analysis.

Mária Žuffová 48:30
And how I analysed the data, so I use the text data approaches concretely topic modelling. So I estimated the topic model on this data, which means that it basically estimates the probability of finding certain topics within a corpus of textual data, in this case, information requests.

Mária Žuffová 48:53
And I also had to get rid of the words that are very common or occur frequently in communication between public servant and requester. So things like greetings or titles or then the institutions, like when they address their their officer at the Ministry x and y, and these are the results.

Mária Žuffová 49:18
So when it comes to the topics that were mostly predominant in the sample of information requests from 2008 to 2017, the most prevalent, most predominant was welfare reform, then the second one was labeled as related to transparency, and requesters mostly asked about correspondence between government members and lobbyists or other important figures of political and economic life.

Mária Žuffová 49:57
And then the third one. Official statistics for the most predominant again linked to transparency, and requesters mostly asked about meetings, minutes, expenses, then then it was followed by asking for for legal documents, for rules, guidelines related to public administration, and then education, and then you can see the list of those remaining.

Mária Žuffová 50:29
What was, for me, interesting to see that, yes, there were topics that clearly deal with accountability seeking.

Mária Žuffová 50:40
So the topic where requesters asked about correspondence, or topic 20, when they asked for meetings, minutes or expenses, there were there were some others as well.

Mária Žuffová 50:52
So another one that was labeled as transparency, asked for for documents, for contracts. But in general, when I had a look at this information gathering metrics that I introduced at the beginning and tried to categorise all these topics within these quadrants of this matrix, I was able to came to the conclusion that, basically, yes, accountability seeking requests are very important.

Mária Žuffová 51:22
They are they are still predominant, but in reality, they account only for a third of all requests. So the rest were these requests that could have been labeled as informing to empower or securing some personal benefits or ensuring fairness.

Mária Žuffová 51:38
And I was curious also about this most predominant topic that I labeled as welfare reform, because all of these requests, to some extent, related to welfare reform.

Mária Žuffová 51:53
And what you can see in this figure is how this topic, how the how the proportion of requests related to welfare reform, how they evolved over time during the studied period, so from 2008 to 2018 and as you can see, there has been a slight increase, starting in 2010/11.

Mária Žuffová 52:16
And then there has been a high increase in information request asking about welfare reform around 2013/14, and the interest in the topic was, let’s say, up until 2016 and if you look at the …

Mária Žuffová 52:32
I mean, how the method works is that the first part is basically just a statistical method trying to somehow group these words based on the group these topics based on the semantic proximity of the words.

Mária Žuffová 52:47
But then, of course, I had to read the sample for each topic in full so I’m able to make sense of these topics.

Mária Žuffová 52:56
And it was possible to clearly see that, yes, there is a link, because in 2012 there was this Welfare Reform Act in the UK adopted.

Mária Žuffová 53:06
And then in 2015 there was Welfare Reform and Work Act adopted. And when I read through the through some of the information requests, mostly it was not even information request per se as they aredefined in legal terms, but mostly complaints of these policy beneficiaries who said that they are very confused about the changes that have been introduced, that they don’t fully understand them and they really did not receive well these policy changes, so mostly they raised criticism and also sought clarification.

Mária Žuffová 53:47
There were many information requests asking about what is the method for calculating the pension age, or many just basically concluded that severely disabled people with lifelong conditions were suddenly assessed as fit to work.

Mária Žuffová 54:05
So oftentimes information requests were used to raise these concerns about about this new policy in place and how it was ill conceived.

Mária Žuffová 54:17
So they argued that that the reforms left vulnerable disadvantaged groups behind and stigmatised them even further.

Mária Žuffová 54:25
So for me, this was one of the most interesting findings, that information requests were used in a completely different way that they were initially intended when these reforms were adopted.

Mária Žuffová 54:40
So beyond this accountability seeking, and here I also included the another topic that was labeled as disability and fit for work assessment over time, where the results were basically very similar to what I just described about the welfare reforms.

Mária Žuffová 54:59
So again, mostly, rather than information requests, in a strict sense, they were used as a platform to raise concerns and to conclude, because this session is called, “is Access to Information working well in practice?”, And I was thinking about this question in relation to my research, and I think it very much depends on where you stand and what what you expect from transparency.

Mária Žuffová 55:33
So my findings definitely concluded that the uses are manifold. They go beyond accountability seeking and the use of information requests, but also the public demand. D

Mária Žuffová 55:50
emand for information is very heterogeneous and but I still think that despite they are used for a goal that was not initially intended in this reform.

Mária Žuffová 56:04
I think it still works quite well. And I think we might want to look beyond this accountability framework and think about other benefits of information requests, because for me, it was really kind of a litmus test of public support for new laws and policies in case of this welfare reform, because during the same period, there were other important legislation adopted in the UK that were very much discussed in the media, but they it wasn’t reflected in information requests, so people didn’t ask about these.

Mária Žuffová 56:45
And the reason was that these were accepted very well, whereas welfare reform people had many reservations, so I think it’s also a useful source of information for policy makers on how people actually react to to the new policies.

Mária Žuffová 57:04
So I would argue that maybe information requests are not always submitted in line with what they were seen for, but still, this is very useful, so they might point to other things that don’t work well in practice.

Mária Žuffová 57:22
So for instance, if someone has to use an information request to ask about his immigration status or to ask about how his pension age should be calculated, it clearly shows that there’s something not working well with these with these policies that should or with the agencies that should be dealing with these issues.

Mária Žuffová 57:42
So in my view, I would conclude that, yes, maybe the uses are different than we expected, but I think it’s still extremely useful.

Julia Cushion 57:52
Thank you. Thank you so much, Mária, that was fantastic. So interesting, and I feel like we covered a really great range of topics across the three presentations. Thank you to our amazing speakers, Mária a nd Giovanni and Toby.

Myf Nixon 58:09
We’ve put on a lot of online webinars and events recently, and it’s all with the aim of sharing knowledge amongst our global networks of civic tech organisations, and of course, anybody else who has an interest in mySociety’s topics of democracy, transparency, climate and community.

Myf Nixon 58:26
If you’d like to be kept informed about upcoming webinars, you can sign up for our newsletter. I’ll put the link in the show notes and make sure that you check the box which is marked conferences and events if you want to hear about every event that we put on right across all of our activities.

Myf Nixon 58:43
Or if you have a particular interest, say in transparency or climate or whatever it might be, just make sure you check the right boxes, and events will be part and parcel of the communications that we send out in that category.

Myf Nixon 58:58
All right, that’s it. Thanks very much for listening. Bye.