At the Impacts of Civic Technologies conference TICTeC last year, we were treated to a presentation on a resource for the Civic Tech research world. OGRX, the Open Government Research Exchange is a repository of digital, eGov and Civic Tech peer-reviewed and stand-out articles, out of GovLab NYU — you can see the in-depth presentation here.
As one of the featured Editors of OGRX, mySociety’s Head of Research Dr Rebecca Rumbul was recently invited to make her top five picks from the collection. As she explains, her selection runs from the “paper that should be read by all newcomers to the Civic Tech world” to the “important piece of literature that I take inspiration from when designing my own research projects”.
Rebecca has this to say about the value of sharing other people’s research in the sector, and the benefits of OGRX:
“The mySociety research team spends lots of time asking interesting questions about Civic Tech, and dreaming up ways to answer those questions, but one of the main things we do that is not so obvious is read about other people’s research. A LOT!
“Before we start any new research project, we carefully review what others have done before us, thinking about what worked for them, what kind of experimental design they used, what other writers inspired their research, and what insights they were able to draw from their work. Learning about others’ research is one of the main parts of the job of researching the impacts of Civic Tech.
“This is one of the reasons that we began the annual TICTeC conference: to give practitioners and researchers interested in the impacts of Civic Technology a genuine opportunity to learn, share and challenge each other in a safe space.
“Alas, TICTeC only happens once a year, and outside of that, it can be tough to know where to go to find interesting, current and relevant research, especially if you don’t have university access to online journal articles.
“That’s where OGRX really fills the gap. If you are thinking of submitting something for TICTeC and want some inspiration, or if you are just interested in accessing good quality and relevant content, why don’t you have a look around? You can also submit your own research!”
I’m just a few weeks into my position of Research Associate at mySociety and one of the things I’m really enjoying is the really, really interesting datasets I get to play with.
Take FixMyStreet, the site that allows you to report street issues anywhere in the UK. Councils themselves will only hold data for the issues reported within their own boundaries, but FixMyStreet covers all local authorities, so we’ve ended up with probably the most comprehensive database in the country. We have 20,000 reports about dog poop alone.
Now if you’re me, what to do with all that data? Obviously, you’d want to do something with the dog poop data. But you’d try something a bit more worthy first: that way people won’t ask too many questions about your fascination there. Misdirection.
How does it compare?
So, starting with worthy uses for that massive pile of data, I’ve tried to see how the number of reports in an area compares against other statistics we know about the UK. Grouping reports into ONS-defined areas of around 1,500 people, we can match the number of reports within an area each year against other datasets.
To start with I’m just looking at English data (Scotland, Wales and Northern Ireland have slightly different sets of official statistics that can’t be combined) for the years 2011-2015. I used population density information, how many companies registered in the area, if there’s a railway station, OFCOM stats on broadband and mobile-internet speeds, and components from the indices of multiple deprivation (various measures of how ‘deprived’ an area is, such as poor health, poor education prospects, poor air quality, etc) to try and build a model that predicts how many reports an area will get.
The good news: statistically we can definitely say that some of those things have an effect! Some measures of deprivation make reports go up, others make it go down. Broadband and mobile access makes them go up! Population density and health deprivation makes them go down.
The bad news: my model only explains 10% of the actual reports we received, and most of this isn’t explained by the social factors above but aspects of the platform itself. Just telling the model that the platform has got more successful over time, which councils use FixMyStreet for Councils for their official reporting platform (and so gather more reports) and where our most active users are (who submit a disproportionate amount of the total reports) accounts for 7-8% of what the model explains.
What that means is that most reasons people are and aren’t making reports is unexplained by those factors. So for the moment this model is useful for building a theory, but is far from a comprehensive account of why people report problems.
Here’s my rough model for understanding what drives areas to submit a significantly higher number of reports to FixMyStreet:
- An area must have a problem
Measures of deprivation like the ‘wider barriers to housing deprivation’ metric (this includes indicators on overcrowding and homelessness) as well as crime are associated with an increase in the number of reports. The more problems there are, the more likely a report is — so deprivation indicators we’d imagine would go alongside other problems are a good proxy for this.
- A citizen must be willing or able to report the problem
Areas with worse levels of health deprivation and adult skills deprivation are correlated with lower levels of reports. These indicators might suggest citizens less able to engage with official structures, hence fewer reports in these areas.
People also need to be aware of a problem. The number of companies in an area, or the presence of a railway station both increase the number of reports. I use these as a proxy for foot-traffic – where more people might encounter a problem and report it.
Population density is correlated with decreased reports which might suggest a “someone else’s problem” effect – a slightly decreased willingness to report in built-up areas where you think someone else might well make a report.
- A citizen must be able to use the website
As an online platform, FixMyStreet requires people to have access to the website before they can make a report. The less friction in this experience makes it more likely a report will be made.
This is consistent with the fact that an increased number of slow and fast home broadband connections (and fast more than slow ones) increases reports. This is also consistent with the fact that increased 3G signal in premises is correlated with increased requests.
Reporting problems on mobile will sometimes be easier than turning on the computer, and we’d expect areas where people more habitually use mobiles for internet access to have a higher number of reports than broadband access alone would suggest. If it’s slightly easier, we’d expect slightly more – which is what this weak correlation suggests.
Not all variables my model includes are significant or fit neatly into this model. These are likely working as proxy indicators for currently unaccounted for, but related factors.
I struggle, for instance, to come up with a good theory why measures of education deprivation for young people are associated with an increase in reports. I looked to see if there was a connection between an area having a school and having more reports on the basis of foot-traffic and parents feeling protective over an area – but I didn’t find an effect for schools like I did for registered companies.
So at the moment, these results are a mix of “a-hah, that makes sense” and “hmm, that doesn’t”. But given that we started with a dataset of people reporting dog poop, that’s not a terrible ratio at this point. Expanding the analysis into Scotland and Wales, analysing larger areas, or focusing on specific categories of reports might produce models that explain a bit more about what’s going on when people report what’s going wrong.
I’ll let you know how that goes.
Researching in an unstable environment
It’s been nearly two years since the InfoLib Liberia project with iLab Liberia started. In that time the project has faced many hurdles, some predicted, and some completely unforeseen.
The iLab team have seen their country devastated by Ebola, only 11 years after the end of their second civil war, bringing tragedy and instability along with it. As you can probably imagine, the impact of curfews, fear and death in communities has made it difficult for people to continue with their daily lives. The social impact of such a disease is wide-reaching. Distrust, marginalisation and exclusion can be directed at those who show symptoms, or even who suffered and survived.
These are challenges that our local partners have had to contend with every day, both when holding training sessions and more crucially when researching the impact of the project on people’s lives.
However, by far the largest hurdle for this particular project has been a mixture of low internet penetration and lack of government will to release information. The team on the ground have been working tirelessly to create an ecosystem of requesting and training Public Information Officers (PIOs) to reply – even providing them with tablets to scan documents without needing electricity, let alone a computer. But if those officers have no access to the information that has been requested, their jobs become virtually impossible.
The project is now drawing to a close and we’re undertaking our final research survey. It seemed like a good time to take a look at what we’ve learnt about the impact of our joint Freedom of Information project in Liberia.
When designing the project we decided that impact could best be measured in terms of whether or not the project increased confidence in government transparency.
We carried out surveys in January 2016 and April 2016, to provide a baseline picture and then an assessment of impact at midline. The final survey is being conducted in August 2016 just as the project ends.
The first survey – the baseline – was carried out mainly in the rural areas. iLab Liberia teamed up with LFIC to survey 152 participants who had been involved in the FOI workshops that LFIC had held in the counties.
We had to attempt the second survey twice, as it turned out to be more challenging than we’d expected. We needed the participants from the first questionnaire to answer the same questions we’d asked them initially, in order to measure change — but it proved hard to locate all of them.
There were many factors which caused this, but the main one was economic drivers, forcing people to move to where the opportunities are. It’s a problem many researchers must run into working in the field.
Carter, the project lead at iLab Liberia told us:
“There are several reasons why this happens […]. People migrate a lot between markets, farms. Several persons who participated in the baseline could not be reached as they [had] travelled to other cities/counties. [Or] the job that allowed them to reside in that city/county is no longer available so they might have left seeking after another job.”
Our second attempt was more successful. We managed to contact a large percentage of the original participants in the survey: 112 of the 152.
We’ve found out some interesting things from doing this research. We saw that 74% of people who use the internet daily say it’s their main source of information, though it is still only a small percentage of the population who have access to the internet.
So the next biggest source of information? Radio! 85% of people with with no access to the internet give radio as their main source of information. Thinking of the migration of workers between cities and counties – you suddenly appreciate why Radio is such an important medium for getting hold of information. Thankfully, as you’ll remember from our original blog post, we’re covering both of these media in the InfoLib project.
In the months since we began studying the impact of this project we also learned that fear of making a request has dropped by 5% in the individuals surveyed . The amount of people who reported that they didn’t know how to ask for information dropped from 24% to 21%. This is pretty great news to us as it shows that our training and our encouragement is working – albeit slowly.
Finally we saw the percentage of people who believe government would be more transparent if citizens could see the information they hold rise by 3% to 93% of the surveyed respondents. Even if this figure hadn’t risen, this demonstrates a clear existing demand from the citizens of Liberia for the Government to release more information about its activities which is great news overall!
No project is without its challenges, and as you’ve seen above one of the big ones is ensuring that the same people respond from survey to survey. Not being able to pin down precisely the same set of people means that we can’t say with 100% certainty that we have a true measure in the difference in attitude.
As a result of the economic and social drivers mentioned above, the workforce in Liberia is very transient. This makes disseminating information through radio and internet mediums even more important. This research has shown that these are the primary sources of news and official information for the majority of Liberians, and continuing to improve knowledge about, and access to, information via these sources will empower the population further.
Finally, it can be challenging to demonstrate impact in projects like these, simply because research is not the main focus for our local partners. We partner with local groups because they are passionate, capable, and able to engage and mobilise citizens around a certain issue. We cannot expect small grassroots groups to have the resources or experience to conduct academic surveying, sampling or interviewing that could detect and definitively isolate the short term impact of a small project. This piece of research has provided some encouraging interim results, but most of all, it has provided valuable lessons to us at mySociety in trying to conduct this kind of impact research remotely and in partnership.
While we wait for the outcome of the final survey we can feel cautiously hopeful that this project has caused a small change in the way access to information operates in Liberia. infoLib will continue to run after the project officially ends, and mySociety will continue to support the work that iLab does in this area . However it may take longer than we had expected or hoped, to see the governmental shift towards releasing information.
mySociety’s flagship Freedom of Information (FOI) request portal WhatDoTheyKnow.com, operating in the UK since 2008, has amassed a whopping 330,000 FOI requests (and counting) from citizens over its eight year life-span.
That equates to approximately 15-20% of all FOI requests made in the UK. It also represents the largest database of FOI requests in the country, having provided a platform for requests and responses to over 17,000 UK public authorities to be published publicly online.
Those are some impressive numbers: however, until now we haven’t known much more about what requests are being made, whether there are trends, or indeed, whether the responses that people are receiving are satisfactory.
We thought it was about time that we took a look under the bonnet of WhatDoTheyKnow to find out the answers to some of these bigger questions.
We decided first to look at what themes and policy areas people were most interested in when making an FOI request. We chose this area because we suspected that many people would be asking for similar things from similar authorities. If this is the case, then this would be a clear evidence-based argument for authorities to increase pro-active publication of certain information.
The task itself was not an easy prospect. WhatDoTheyKnow does not have a tagging or categorising system, so there are over 330,000 requests that we had no quick or easy way of comparing. The volume of data was also so high that we couldn’t reasonably extract every request and analyse which policy area(s) it was relevant to.
To solve this issue, we decided to focus on the 20 authorities receiving the highest volume of FOI requests between 2008-2016. This way we could rely on a large sample of requests for both both local authorities and government departments. The list of authorities is below.
Department for Work and Pensions
Department for Education
Wirral Borough Council
UK Borders Agency
Birmingham City Council
Brighton & Hove City Council
Liverpool City Council
Transport for London
Westminster City Council
Ministry of Defence
Metropolitan Police Service
Bristol City Council
Ministry of Justice
Lambeth Borough Council
Camden Borough Council
Department of Health
Kent County Council
Taking all the requests made to these public bodies gave us a total of 49,500.
With the generous support of Thomson-Reuters, we were able to use OpenCalais, their automated tagging system, to assign one or more thematic tags to each FOI request made. Over 100,000 hyper-granular tags were automatically applied in this way.
Once that was complete, we went through each tag and the requests it was associated with. We grouped tags into policy areas and checked for any that had been incorrectly assigned. We then split the authorities into two groups: Local Authorities and Departmental Bodies, to compare the most requested information.
Among Local Authorities, the top requested information concerned:
- Housing Specifically, information on social housing stock/occupancy/waiting lists, facilities for homeless and at-risk individuals, and planning permission
- Social Care Information concerning care providers and their funding/operations, care in the community arrangements, social worker qualifications and staffing levels, and information concerning the operation and monitoring of social work departments
- Accounts and Budgets Citizens commonly request accounting and budgetary information at a far more granular level than authorities are currently publishing.
- Authority management Citizens also wish to know with greater detail how authorities are operating internally, requesting management and meeting information, emails about decision-making, and structural information concerning development, contracting and relationships with private companies
- Business rates Concerning long-term empty properties, the impact of rates on town centres, charitable or other discounts, and regeneration.
These are the top five of thousands of tags, but common themes were clear when comparing these authorities.
Generally, requesters have shown they want information in a more detailed form than authorities are currently providing, in particular in the expenditure of public funds and those services catering for complex cross-cutting social issues. Given the ongoing housing crisis in the UK, coupled with the ageing population, it is likely that information concerning these policy areas will be in increasing demand.
Conversely, among Departmental Bodies, the top requested information showed few common themes. This is primarily due to the differences in policy areas between the departments. There were, however, significant spikes in certain policy areas within departments, particularly around immigration, and this will be the focus of future investigation.
In conclusion, we understand that very few FOI requests are completely identical in subject matter, but broad trends are clearly evident.
If Local Authorities proactively publish more granular information about the policy areas we now know citizens are actively interested in, they may see a dip in formal FOI requests.
Publishing information and data in a machine-readable format may even enable other civic technologists like ourselves to build tools to assist councils in their delivery of vital services. Whilst this will not eradicate FOI requests completely, it would hopefully begin a shift in behaviour.
In short: wouldn’t it be great if, instead of authorities seeing FOI as an administrative burden, they began to see pro-active publication as an opportunity to harness the skills, initiative and flexibility of citizens.
When you send a Freedom of Information request through a site like WhatDoTheyKnow, do authorities respond in the same way as to a request sent via email? Our latest research would suggest that there is a small but crucial difference.
Just one channel
We provide Alaveteli, the software that underpins Freedom of Information sites all over the world — but of course, those sites are not the only means by which citizens can make FOI requests.
A Right to Know means that citizens can request information via whatever means are allowed in their country’s law: traditionally, that’s by post, but many authorities will accept requests via phone and email, and there are even examples of responses being obtained via Twitter.
So Alaveteli sites make a complicated and potentially intimidating process easier, and they also have the benefit that they publish requests and responses online for everyone to access, but they represent just one channel via which information can be accessed.
Something that we’ve often wondered is whether there is any difference in the way authorities respond via an Alaveteli site, or via the email system.
So mySociety’s research team got together with Informace Pro Všechny, the Czech Republic’s Alaveteli site, to conduct an experiment.
The question under scrutiny: Are Freedom of Information requests sent via email treated the same as requests sent via an Alaveteli platform that allows citizens to make requests via an online portal, and publishes all responses publicly on its website?
We wanted to know:
- Would responses be the same?
- Would it take the same amount of time to get a response?
- Would you overall get a better or worse service via Informace Pro Všechny than via a personal email address?
These questions are especially pertinent to us because we want to make sure that our technology is working for people, rather than against them. At the very least, we want to ensure that using an Alaveteli platform such as Informace Pro Všechny will provide the same level of service that citizens can expect from using private email addresses. If using a site such as this does not result in the same level of service, then this would be an issue we as civic technologists should know about and try to address.
Our experiment was simple. We sampled 100 public authorities (town halls and ministries), and sent them two separate requests via a private email address, and two separate requests via Informace Pro Všechny.
The information requested was deliberately simple and uncontroversial, and clearly subject to Freedom of Information law, to avoid any deliberation by public authorities about whether to release it.
The good news is that using both channels of communication — individual email or Informace Pro Všechny — results in the same quality of response. Neither method of communication was found to be inferior to the other with regard to how substantive the response was.
The even better news is that use of Informace Pro Všechny resulted in faster responses to requests. Whereas private email requests were provided on average within 9.2 days, responses to requests sent via Informace Pro Všechny took only 7.2 days – two days quicker.
This is a positive outcome that was by no means certain, and at this point we are unable to fully explain it. It is possible that public authorities were quicker to respond to Informace Pro Všechny requests because these were known to be published online, and therefore, a slower response would be more noticeable.
Or the quicker response rate via the site could be attributed to the fact that its users are known to be politically active, politically interested or involved in journalism: a quicker response might reduce negative coverage or feedback. Or it could be that other external factors we were unaware of influenced the result.
More research would be required to determine the causes of these differences, however, at this point, we are simply delighted to say that Informace Pro Všechny is currently the quickest tool to use to request information from government in the Czech Republic.
The chairs have been stacked, the banners rolled away, and 142 delegates have returned to their 29 home countries. TICTeC, the Impacts of Civic Technologies conference, is over for another year.
The 1.5 day event saw a concentration of wisdom and expertise from across the civic tech sector, and we’re keen to ensure that we share as many insights as possible.
To that end, we’ll be publishing materials such as photos, videos and slides, as soon as we can. We hope that, if you weren’t able to attend, they’ll give you a taste of the TICTeC experience — and, if you were there, they’ll serve to keep it fresh.
Some materials will take a little time: videos, for example, are currently in post-production, and should be ready within a few weeks. We’ll be announcing on the mySociety Twitter feed, Facebook page and this blog when they’re online, or check the TICTeC resources page.
Meanwhile, here’s what’s available right now:
- Slides from all the speakers Click on each speaker’s name to access them.
- Photos: all under Creative Commons, so feel free to download and share them if you wish.
- A Storify to help you relive the experience through hashtagged tweets and photos.
- The TICTeC Google Group: everyone who attended the conference is a member, so this is the place to continue discussions or begin new ones.
Thanks so much to everyone who participated, making TICTeC a real success. We hope to see you all again.
How do researchers accumulate knowledge about civic technology? So far, many of our most active discussions still happen at conferences and in the informal writing of blog posts. Individual organizations (like mySociety) are investing more heavily in the production of academic-quality research, so we can all see more clearly what happens when we try to use technology to bring people and governments together. But is this work yet diffusing into academic use?
I wrote about this question while researching the context of mySociety’s current work on US city-led civic tech. Although the term “e-government” seems to have been largely bypassed in current civic technology practice, it’s useful to see that “e-government” is still a very important concept. For those of us trying to see how governments approach the use of technology in connecting to citizens, research on e-government provides a useful perspective. At the same time, however, the perspectives of e-government and civic technology also diverge — and that too is important to note.
Why Civic Technologists Should Still Care About E-Gov
As a recovering academic, I spend more time than I should going down rabbit holes: reading articles which cite interesting studies, which I then go find and read, and then they cite interesting papers, and pretty soon way too much time has gone by. I’m currently researching the role of civic technology projects enacted by U.S cities for mySociety, so I’ve been actively looking for those civic tech rabbit holes.
There’s just one hitch: civic technology barely exists in the academic context. E-government, on the other hand, continues to have an active research program.
Searching “e-government” in Google Scholar returns 169,000 cited works—10,600 published since 2015 alone. Searching “civic technology,” meanwhile, nets a total of 185. (And one of the top results references technology in the Honda Civic.)
Read more at Civicist.
mySociety’s Head of Research, Rebecca Rumbul, gives an overview of our research work in the latest publication from the World Bank’s Open Knowledge Repository. Also featured is an experiment in citizen engagement from Mzalendo in Kenya, that was first shared at TICTeC.
Evaluating Digital Citizen Engagement: A Practical Guide is a handy collection of examples and lessons from practitioners in Brazil, Uganda, Cameroon and Kenya, on how to measure the impact of civic technologies.
Rebecca explains the methods we’re currently using to answer questions like, “are institutions equally responsive to citizens?” and, crucially, “are our tools genuinely making a difference?”.
Meanwhile, Lily L. Tsai and Leah Rosenzweig, who contributed last year to our Impacts of Civic Technologies conference TICTeC, give an overview of how they used Facebook ads to draw conclusions about what makes people take concrete political actions online.
You can download the guide for free here — and don’t forget, if you’d like to hear more about the ways in which civic tech’s impact is being tested by projects around the world, there are still a few tickets available for TICTeC 2016.
TICTeC is our annual conference on the impacts of civic technologies.
This year, we’ll be in Barcelona, Spain, with a diverse programme of speakers from all over the world.
Guy Grossman, assistant professor at the University of Pennsylvania’s political science department, will deliver one of two keynotes. See details of our other keynote speaker, Helen Milner OBE, here.
Guy has had a long history in civic tech research, with a special focus on Uganda and Sub-Saharan Africa, and we are delighted that he will be sharing his insights to our audience of practitioners and researchers.
Hi Guy! Tell us what you’ll be covering in your TICTeC keynote.
The state of academic knowledge about the promises and pitfalls of ICT applications designed to increase voice and accountability in low-income countries. Specifically:
- Under what conditions are citizens more likely to communicate with government officials using mobile-platforms? If we “build them – will they come”?
- Do mobile platforms exacerbate inequalities in political access, privileging groups (men, urban, better-off) that are heavier users of ICTs?
- What can be done to help increase the participation rates of marginalised populations (such as women and the poor)?
Low-income countries have leapfrogged past the landline era and directly into the mobile era. Citizens can now connect with each other and with their public officials in unprecedented ways.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited.
The massive penetration of mobile technology, even to remote areas of the least developed countries, has great potential to reshape both the social and political landscapes.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited. My goal at TICTeC is to help take stock of what we know and don’t know in this emerging field.
What are you hoping to get out of TICTeC?
I’m hoping to interact with practitioners, policy makers, technologists and fellow academics to figure out where the overlap is between practical needs and academic research.
I’m also hoping to get exposed to promising and innovative applications that I am unaware of.
Your civic tech research focuses especially on Uganda and Sub-Saharan Africa. What led you down this path?
I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries.
I care deeply about poverty and social and gender inequality, so working and studying in Africa has been a natural choice.
How did I get involved with civic tech? In the past decade I have witnessed an increasing number of mobile-based applications designed to enhance such things as agriculture productivity and financial transactions or match between buyers and sellers.
On the other hand, I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries. I came to believe that this was a great void that needed to be filled.
In fact, unfortunately, we are still quite far from saturation in the development of governance-related ICT applications.
What do you think are the big unanswered questions when it comes to civic tech?
Does participation in mobile communication spill into more traditional forms of political participation?
Here’s a very partial list of some of the answered questions when it comes to civic tech in the global south:
Can simple innovations in mobile technology be used to facilitate new meaningful forms of political participation?
If so, what types of mobile-based political communication are most likely to be adopted by voters?
What is the potential of ICTs to flatten political access to marginalised populations?
What are some of the ways to increase the usage of mobile-based political communication in the face of clear collective action problems?
How can ICT be used to overcome political economy constraints to growth (e.g. by resolving monitoring challenges in government bureaucracies, reducing the cost for citizens of tracking bureaucratic performance, or making it easier to detect corrupt practices in public-sector salary distribution)?
And, does participation in mobile communication spill into more traditional forms of political participation?
If you could make one recommendation to those developing new civic tech, and wanting to see real impact from it, what would it be?
Adopt a user perspective — make sure that the suggested application/platform addresses a real need that could not be addressed using a low-tech solution.
You won’t want to miss what Guy has to say at TICTeC, so make sure you book your tickets now.
The latest paper from our Head of Research, Dr Rebecca Rumbul, is available for download.
Drawing on interviews with 40 individuals in government and civil society, this research strives to answer the question: how do government and civil society initiatives and innovations in New Zealand and Australia attempt to reduce digital exclusion amongst digitally under-represented user-groups?
Rebecca pulls out the best practices from these two countries, and looks at how they could be replicated here. The result is four strong recommendations for the UK’s policymakers.