Researching in an unstable environment
It’s been nearly two years since the InfoLib Liberia project with iLab Liberia started. In that time the project has faced many hurdles, some predicted, and some completely unforeseen.
The iLab team have seen their country devastated by Ebola, only 11 years after the end of their second civil war, bringing tragedy and instability along with it. As you can probably imagine, the impact of curfews, fear and death in communities has made it difficult for people to continue with their daily lives. The social impact of such a disease is wide-reaching. Distrust, marginalisation and exclusion can be directed at those who show symptoms, or even who suffered and survived.
These are challenges that our local partners have had to contend with every day, both when holding training sessions and more crucially when researching the impact of the project on people’s lives.
However, by far the largest hurdle for this particular project has been a mixture of low internet penetration and lack of government will to release information. The team on the ground have been working tirelessly to create an ecosystem of requesting and training Public Information Officers (PIOs) to reply – even providing them with tablets to scan documents without needing electricity, let alone a computer. But if those officers have no access to the information that has been requested, their jobs become virtually impossible.
The project is now drawing to a close and we’re undertaking our final research survey. It seemed like a good time to take a look at what we’ve learnt about the impact of our joint Freedom of Information project in Liberia.
When designing the project we decided that impact could best be measured in terms of whether or not the project increased confidence in government transparency.
We carried out surveys in January 2016 and April 2016, to provide a baseline picture and then an assessment of impact at midline. The final survey is being conducted in August 2016 just as the project ends.
The first survey – the baseline – was carried out mainly in the rural areas. iLab Liberia teamed up with LFIC to survey 152 participants who had been involved in the FOI workshops that LFIC had held in the counties.
We had to attempt the second survey twice, as it turned out to be more challenging than we’d expected. We needed the participants from the first questionnaire to answer the same questions we’d asked them initially, in order to measure change — but it proved hard to locate all of them.
There were many factors which caused this, but the main one was economic drivers, forcing people to move to where the opportunities are. It’s a problem many researchers must run into working in the field.
Carter, the project lead at iLab Liberia told us:
“There are several reasons why this happens […]. People migrate a lot between markets, farms. Several persons who participated in the baseline could not be reached as they [had] travelled to other cities/counties. [Or] the job that allowed them to reside in that city/county is no longer available so they might have left seeking after another job.”
Our second attempt was more successful. We managed to contact a large percentage of the original participants in the survey: 112 of the 152.
We’ve found out some interesting things from doing this research. We saw that 74% of people who use the internet daily say it’s their main source of information, though it is still only a small percentage of the population who have access to the internet.
So the next biggest source of information? Radio! 85% of people with with no access to the internet give radio as their main source of information. Thinking of the migration of workers between cities and counties – you suddenly appreciate why Radio is such an important medium for getting hold of information. Thankfully, as you’ll remember from our original blog post, we’re covering both of these media in the InfoLib project.
In the months since we began studying the impact of this project we also learned that fear of making a request has dropped by 5% in the individuals surveyed . The amount of people who reported that they didn’t know how to ask for information dropped from 24% to 21%. This is pretty great news to us as it shows that our training and our encouragement is working – albeit slowly.
Finally we saw the percentage of people who believe government would be more transparent if citizens could see the information they hold rise by 3% to 93% of the surveyed respondents. Even if this figure hadn’t risen, this demonstrates a clear existing demand from the citizens of Liberia for the Government to release more information about its activities which is great news overall!
No project is without its challenges, and as you’ve seen above one of the big ones is ensuring that the same people respond from survey to survey. Not being able to pin down precisely the same set of people means that we can’t say with 100% certainty that we have a true measure in the difference in attitude.
As a result of the economic and social drivers mentioned above, the workforce in Liberia is very transient. This makes disseminating information through radio and internet mediums even more important. This research has shown that these are the primary sources of news and official information for the majority of Liberians, and continuing to improve knowledge about, and access to, information via these sources will empower the population further.
Finally, it can be challenging to demonstrate impact in projects like these, simply because research is not the main focus for our local partners. We partner with local groups because they are passionate, capable, and able to engage and mobilise citizens around a certain issue. We cannot expect small grassroots groups to have the resources or experience to conduct academic surveying, sampling or interviewing that could detect and definitively isolate the short term impact of a small project. This piece of research has provided some encouraging interim results, but most of all, it has provided valuable lessons to us at mySociety in trying to conduct this kind of impact research remotely and in partnership.
While we wait for the outcome of the final survey we can feel cautiously hopeful that this project has caused a small change in the way access to information operates in Liberia. infoLib will continue to run after the project officially ends, and mySociety will continue to support the work that iLab does in this area . However it may take longer than we had expected or hoped, to see the governmental shift towards releasing information.
On January 8th Liberia launched their new Freedom of Information platform, infoLib, based on our Alaveteli software — not just by pressing a button to put the site live, but with a public event that reached many sectors of society.
The launch was a great success: it was attended by representatives from groups including university students, government ministries and NGOs, each of which will be able to use the site for their own needs.
The Liberian Government and many of the country’s NGOs are excited about infoLib’s ability to monitor when requests come in and to ensure that they are replied to on time.
Attendees expressed happiness with the platform and excitement about what it means for Liberia. Many have said they’re more optimistic that requests will be answered, now that there is a clear, transparent way to scrutinise the government.
The event featured a Q&A session about compliance and functionality: the many questions from the audience were answered by the newly trained Public Information Officers as well as the team from iLab.
So what’s next?
Focus is on driving usage; iLab will be accompanying the Liberia Freedom of Information Coalition on their nationwide tour talking about FOI.
In our last post, we talked about how the site is attempting to reach the country’s offline population as well as those who have internet access. On tour, the team will take requests from users, either on paper or directly onto the site if there’s an internet connection.
Growing usage of the site will be a slow process. While there’s enthusiasm for the project, it’s all very new and people want to see proof that it works — so we have a lot of hard work ahead of us in the coming months.
In addition to this, iLab are going to be running FOI surgeries on community radio stations in the counties and Monrovia. People will have the opportunity to phone in and make an FOI request, and the answers to previous FOI requests will be shared.
Finally we’ll be working on training up the last PIOs and building their skills to give them the best chance to answer requests promptly, online, and with the relevant information.
Everything’s going to plan so far, and there are many aspects of this launch that people launching Alaveteli sites in the future can learn from. Thanks for sharing your progress, iLab, and best of luck as you go into the next phase of your journey.
Next time you sit down at your computer to find out some information, remember that things aren’t quite so simple everywhere.
A new Freedom of Information website launches in Liberia today, hoping for success despite the fact that many in the country have little or no access to the internet. If the idea of running an email-based requesting system under such circumstances sounds slightly ambitious, read on to see just how iLab Liberia will make it work, in collaboration with the Liberia Freedom of Information Coalition, and funded by the Making All Voices Count project.
The Liberia Freedom of Information Request Platform – InfoLib – is the latest site to use our Alaveteli software. Like all Alaveteli sites, it will send requests for information to public authorities by email, while publishing both the requests and the responses online. In time, responses build into a public archive of information.
Online services, offline
So how do you run a site like this in a country with low internet penetration? With a little ingenuity and a knowledge of which effective networks already exist, it seems.
The project will make use of an existing network of regional offices and training centres, set up by the Carter Centre and LFIC. In these hubs, staff have been trained up to submit and receive requests on behalf of citizens, and citizens have attended workshops on how FOI can benefit them. There’s no need for users to have access to a computer, or an understanding of how to use a website — there will be staff who can do it on their behalf.
And they’ve also spent time training the Public Information Officers, or PIOs, on the use of technology to make responding to requests easier. iLab are also providing a similar service within Monrovia, Liberia’s capital, where they already run successful computer and ICT training programmes for interested citizens.
We’ve seen this offline-to-online approach with other projects. At the AlaveteliCon conference we heard from people running Alaveteli sites in Rwanda and Uganda, also areas with low internet access, and we’ve experimented in the past with a similar system to allow people to make FixMyStreet reports via SMS texts to a central office.
In Liberia, almost everyone has access to a radio. Community radio stations are a part of daily life, and the main source of news for many.
iLab Liberia will be putting out regular radio segments, explaining what FOI is and how you can use your rights under Liberian law to access information. They’ll also highlight the most interesting information that’s been released through the site. This approach should see FOI become an increasingly familiar topic, a right that everyone understands and knows that they have access to.
We wish InfoLib the best of luck — and we’ll be keeping a close eye on how these initiatives work out.
About 6 weeks ago we arrived back from Monrovia, having just undertaken our first design exercise out there. Paul wrote about our experience in this blog as a broad overview. After further long distance design calls we wanted to delve a little more deeply into the process we’re following and what we’re learning about it.
To begin with, I should mention is that this is the first project where mySociety International will be leading on the implementation of a project using Design Thinking (the South African was a trial using a cut-down version of the approach and furthermore the implementation is being carried out solely by a local team).
Another important point for us is that the Design Thinking approach encompasses far more than just thinking about software development. The aim of the process is to develop an understanding what is required to ensure that the users’ need is addressed. Some of the solution might be technical, but much is likely to be about the processes and people that are required to ensure that needs are met.
For example, in the case of the Liberian FOI project where the internet penetration is low and the day to day obstacles people need to overcome are significantly more difficult than in the global north, a large proportion of the project time and resources will be dedicated to delivering offline services.
These provision of these services will tend to take a shape that fits into citizens’ current experiences. An example might be setting up an SMS short-code that allows people to contact a support team to call them back, in order to draft an FOI request on their behalf. They will then physically deliver hard copies of those FOI requests to the relevant ministries in Monrovia. This type of solution could be particularly beneficial for people who live outside the capital and do not have the time or resources to travel there to submit requests directly themselves.
There are two critical differences between the Design Thinking approach and other projects we have run with groups in the past. The first is that, with non-design centred partnerships, most groups start the process with a firm sense of the “type” of thing that they want from the outset – for example an instance of our Pombola platform that is used to power Mzalendo.com.
This is totally understandable, and in many cases what the funders of these projects are looking for, but it doesn’t necessarily mean that the users will get the most impactful solution.
Where we have sufficient funding to undertake this process available we intend for all of our new international projects to be starting with no expectations about the ultimate product – the outcome might even be that we don’t end up producing any software at all, because the best solution might consist of a desk, a phone and some radio advertising.
The second difference is that we have usually relied upon the local implementing partner to provide the insight and define the specification.
For our Design Thinking based projects we’ll have a closer relationship with the local partner and together we’ll identify potential end user groups we can talk to about their needs.
After doing a first round of in-depth interviews, the team then synthesise the information – essentially sharing what we’ve learnt with the rest of the group to pick out the most important points. The next stage is empathy mapping, where we figure out what people have said, thought, felt and done. This is a key stage in helping to identify the needs of the users.
It might seem simpler to ask them what they need, and often we did say something like “What would make this process easier for you?”. Yet actually analysing what they’re saying about the process and at what points they seemed frustrated or blocked – that tells us a lot more about points where we could change and hopefully improve the process than a straight up “What do you need?” question.
This is the stage we’re in at the moment with the Liberian project, though we have done some brief forays into Ideation – coming up with ideas for how to address the needs, and we are now starting to thinking about prototyping these ideas.
Of course, this method doesn’t mean we’ll completely stop using software solutions, or looking at A/B testing and Analytics as measures of the success of website. However we will also be looking at other measures of success or failure based on the product we’re building and the change we’re trying to achieve.
In the case of the Liberia FOI project, many of the users are likely to have no direct contact with the software themselves so we’ll need to design a monitoring system that measures the effect the changes have on their experience of making FOI requests.
One thing we’ve learnt is that a Design Thinking approach doesn’t only affect the first iteration of a solution. This may seem obvious, but from our brief work with this process we’ve seen that uses/users can be hard to predict at the outset – though in the case of the latter group we worked hard to spread the net widely in order to find potential users in Liberia.
So we’re interested to see, when we get to that point, what the prototype testing brings back and what new changes, improvements and tweaks need to be made.
More about our experiences with this process will be shared the lifetime of the project, as we learn, change and iterate ourselves.
Last year Dave Whiteland wrote about our first experience of using design thinking in creating the specification for a Freedom of Information Project in South Africa.
For those of you unfamiliar with the design thinking approach, it is worth me providing a little context (borrowing heavily from Dave’s text).
The design thinking way
Traditionally mySociety has built international digital projects by working closely with our local partner to define a specification. We then build to this specification and seek to continually improve it . We perform usability tests, we apply A/B testing, and we think hard about what our analytics tell us. The problem is that much of this is reactive, iterative design: it’s being applied after the core product has already been built.
Design thinking challenges this approach by suggesting that the user on which initial designs are often based is purely imaginary. As a result, the site inevitably includes the assumptions and prejudices of its creators. This won’t necessarily lead to a bad design — especially if the creators are benign and experienced — but it must fail, by definition, to account for the unexpected things that may motivate or concern actual users. The design thinking process attempts to change this by approaching the initial problem in a prescribed way and following a process that isolates genuine, existing requirements. This includes, in design thinking terms, processing the initial interviews into empathy maps from which requirements emerge, and which themselves become features that are rapidly prototyped in isolation from other parts of the system.
Our commitment to the process
mySociety is dedicated to maximizing the impact of its projects by tailoring solutions to the local context. As part of this, we have committed to only carrying out new large-scale international digital projects where we can follow these design principles. One challenge with this approach is that, unlike traditional projects, we are unable to provide funders with a clear description of what the project will deliver. At the start of the process the tools that will be built and the processes and infrastructure that will surround them are unclear. Furthermore, all this prototyping and piloting adds to the time these projects take to complete.
In Making All Voices Count we were incredibly fortunate to find a funder who was not only undeterred by these concerns but actively appreciated the value of the approach. Earlier this year they awarded us a scaling grant to work for a two-year period on a Freedom of Information project in Liberia.
On the ground in Liberia
Last week the project team met up in Monrovia to start the design process. Our partners on the project are the iLab Liberia and Public Works at Stanford, an offshoot of the d.school which focuses on applying design thinking to governance in the developing world. The staff working on the project are Luther Jeke, Carter Draper and Teemu Ropponen (Ilab); Jenny Stefanotti (Stanford); and Paul Lenz, Dave Whiteland and Jen Bramley (mySociety).
So what does this actually entail? Briefly, it first involves meeting with a large range of different stakeholders, users and potential users and building up an understanding of their current behaviours, their needs, challenges and perspectives. We interviewed more than 20 people, including the Information Minister, the Independent Information Commissioner, investigative journalists, Public Information Officers, FOI NGOs and community groups.
Each day we would break down our notes from these interviews into things said, thought, done or felt, and group them by type of stakeholder. This is a very active and visual process, resulting in sheets of paper being covered in hundreds of Post-It notes.
In working through this process it became clear to us that there were a number of common issues. Firstly, while the FOI law in Liberia is legally very strong, in practice adherence is pretty poor – partially due to simple process failures, and perhaps sometimes due to willful avoidance. Secondly, despite significant resources being invested in “sensitising” (educating) citizens about the law, very few FOI requests have ever been made by individual “average” citizens; rather they have been submitted by NGOs, journalists and activist groups. These two finding might not be hugely surprising, but others were perhaps less obvious. For example, almost without exception, the Public Information Officers that we met were deeply proud of their work and wished the FOI law was used more; even though the law allows for electronic or phone call requests to be made in reality each request must be hand-delivered, hard copy, and a receipt obtained; finally, even skilled and experienced investigative journalists can end up spending years chasing requests through a Kafkaesque bureaucracy that takes advantage of the requester’s ignorance of the detail of the law.
The next stage, that we are really only just starting, is to identify some approaches that might address these issues, and then to find a way to attempt to prototype the solutions. To save time and cost and in order to enable maximum flexibility these prototypes often take a “Wizard of Oz” approach – human intervention in lieu of building a technical platform to trial. An example would be rather than building an SMS gateway that interfaces with a computer system the prototype relies upon a person simply receiving the SMS on their phone and typing it into a form.
The week was incredibly intense and rewarding driven by Jenny’s fantastic energy in overseeing the whole process, and the great commitment and engagement from the iLab team – particularly given the developing situation with Ebola while we were there.
Freedom of Information: really?
You might ask: given the infrastructure, income and health challenges facing Liberia , does FOI really matter? Is it perhaps a right to be addressed when more essential needs have been met? Are people even worried about being able to request information from their representatives, due to the other factors at play?
I can make no more compelling case in response than one that was made to me by one of our interviewees:
“Liberia went to war over the mismanagement of natural resources in our country. FOI can help hold people to account and stop it happening again”