Stuart Lawson is a librarian, one of the editors at the Journal of Radical Librarianship, and a part of the open access movement which advocates that research outputs should be distributed for free and online, with an open license — for the good of all.
So it should come as no surprise to learn that some of Stuart’s recent research, informed by Freedom of Information requests to the UK’s Higher Education Institutes, has focused on research journals — and specifically, how much institutes are paying for subscriptions. You can see the requests here, the data released in this spreadsheet, and the resulting report here.
The study collected details of payments made by HEIs for access to academic journals from 2017-2019, focusing on ten publishers. The research team discovered that the total expenditure was more than £330 million.
We spoke to Stuart and asked for some background to this FOI-based investigation, beginning with an explanation of the original motivation behind it:
“Open access publishing means that research is available to everyone, but there are debates around how that model can be paid for. And since there is currently a mixed system where some publications are open access and some require subscriptions to access, libraries are continuing to pay a lot of money for subscriptions while also trying to find ways to fund open access.
“I am a librarian who wants all research to be published open access rather than behind a paywall, so I felt that it was important to know the financial costs of the current system.
“Previously, the amounts were unknown. It’s impossible to have conversations about the appropriate cost of scholarly publishing if we don’t know what those costs are in the first place!”
The need for FOI
Freedom of Information is, of course, a practical way to obtain data from public authorities, and to build up a nationwide picture. But in this case, it was vital.
“Using the legal tool of FOI was the only way to get this data, as institutions were not voluntarily releasing it.
“One publisher, Elsevier, even had a clause in the contracts signed by libraries that forbade them from telling anyone how much they were spending, unless they were required to via an FOI request.
“These ‘non-disclosure’ clauses are common worldwide in publisher contracts, but thankfully not widely used in the UK (except by Elsevier) because Jisc — the higher education body that negotiates most deals — have worked to remove them”.
Despite the reluctance that one might assume that this signified, Stuart says getting the required information was pretty straightforward once they’d submitted the FOI requests. In fact, the hardest part was the admin:
“A majority of HEIs provided the data promptly, although some refused in the first instance which meant I need to push back and sometimes requested an internal review of the handling of the request.
“Eventually most institutions provided the data, but the hold-outs caused a lot more work for me”.
Making requests in public
Why was WhatDoTheyKnow particularly suitable for this project?
“It was the best way that I knew of to make bulk requests to organisations. But more importantly for me, I wanted to make sure there was a complete public record of all responses so that when I compiled the data, others could verify it”.
That’s one of the reasons that WhatDoTheyKnow is set up to publish FOI requests and responses online, so we were glad to hear this.
And what is Stuart’s desired outcome from this study?
“For people to realise the high cost of subscription charges, and for libraries to question how much they are spending. And perhaps even cancel some of the deals and spend their money on enabling open access instead.”
It’s possible that this piece of research will be enough of an eye-opener to start making a change in this area. But Stuart’s realistic:
“I hope I don’t need to send these requests again in future years, but the situation is still moving quite slowly, so it might be necessary to use WhatDoTheyKnow once again!”
Image: Bruno Mira
Earlier this year, we were fortunate enough to be contacted by Brian Keegan, Assistant Professor in Information Science at the University of Colorado Boulder, who specialises in the field of network analysis.
Brian and his team were planning to mine the official biographies of every legislator published by the Library of Congress – going back to the first Congress in 1789 – and add the information as structured data to Wikidata. Having heard of our involvement with WikiProject Every Politician, they wanted to understand more about contributing.
The research team, which included professors from the Libraries, Political Science and Information Science departments, planned to combine this biographical data with more common data in political science about voting and co-sponsorship, so that interesting questions could be asked, such as “Do Ivy League graduates form cliques?” or “Are medical doctors more likely to break with their party on votes concerning public health?”. Their hypothesis was that the biographical backgrounds of legislators could play an important role in legislative behaviours.
However, the first big step before questions could be asked (or SPARQL queries made) was supporting undergraduate students to enter biographical data for every member of Congress (going right back to the first) on Wikidata. This has not generally made it into the datasets that political scientists use to study legislative behaviour, and as students began to enter data about these historical figures, it quickly became apparent why: non-existent nations, renamed cities, and archaic professions all needed to be resolved and mapped to Wikidata’s contemporary names and standardised formats.
Nine months on, the team and ten undergraduates have revised over 1,500 Wikidata items about members of Congress, from the 104th to the 115th Congresses (1995-2018) and the 80th– 81st Congresses (1947-1951), which is 15% of the way through all members dating back to the first Congress in 1789!
They started running SPARQL queries this summer.
Joe Zamadics, a political science PhD student who worked on the project explained the potential of combining these data: “One example we tried was looking at House member ideology by occupation. The graph below shows the ideology of three occupations: athletes, farmers, and teachers (in all, roughly 130 members). The x-axis shows common ideology (liberal to conservative) and the y-axis shows member’s ideology on non-left/right issues such as civil rights and foreign policy. The graph shows that teachers split the ideological divide while farmers and athletes are more likely to be conservative.”
The team are keen to highlight the potential that semantic web technology such as Wikidata offers to social scientists.
For the full Q + A with Brian and Joe see the mySociety Medium post.
When you send a Freedom of Information request through WhatDoTheyKnow.com, every part of the exchange is published online. Those who have browsed the site will know that you can read the correspondence around each request from beginning to end, including the initial enquiry, auto-replies, any holding letters, messages seeking clarification, and finally, the response — or refusal.
We built the site so that, when information was released, that information would be available for everyone. The result is the massive online archive, all searchable, that you can find on WhatDoTheyKnow today. That being the case, why do we bother publishing out all the rest of the correspondence? Why not simply publish the end result, that is, the actual information?
Well, we believe there’s value even in what you might consider the ephemera of everything else, not least that it helps demystify the various steps of the FOI process.
This week, an article by ‘FOIMan’ Paul Gibbons showed that the publication of this material can also help with research. He was able to look at 250 ‘refusal notices’ — that is, times when authorities had turned down requests for information — and pull out examples of best and worst practice.
The result will benefit us all, from those requesting information to those who process the requests: for the former, it sets out what to expect from a refusal, and for the latter, it highlights how to ensure that you are sticking to the law as well as ensuring a good experience for the public.
A refusal, as Gibbons points out, does not have to be a shutting of the gates in the face of the requester: it can help educate, point people towards a better means of obtaining the information they need, or even clarify for the FOI officer where withholding the information may in fact be inappropriate. We’re very glad to have seen our data being used in this way.
Today, mySociety, in partnership with Microsoft, launch Civic Tech Cities, a new piece of research looking at the technologies local governments implement to serve and communicate with their citizens. You can download it here.
Civic Tech: whose job is it?
Debating and making decisions on behalf of the people; managing services, disseminating information — all of these have been the agreed tasks of local government for a very long time. But has citizen-facing technology now also become a core function of government? And if so, how are they doing?
We often say that mySociety was originally set up to show governments how they could be using digital better, and that one day we hope to have done ourselves out of a job.
But perhaps it’s wrong to foresee a time when we’ll be able to pack up and go home. Perhaps those within government will never be able to escape internal bureaucracies and budget constraints to provide the software that their citizens will really benefit from; perhaps the provocative NGO, one step ahead with citizen-to-government technologies, will always be a necessary agent.
We won’t know for sure until we start researching beyond our own sphere.
A vital new area for research
When we set up the mySociety research programme, as you’d expect, our first priority was to look at the impact of the services we, and other organisations like us, were providing.
Around the same time, the term ‘Civic Tech’ was gaining traction, and it carried with it an implicit reference to applications made outside government, by organisations like us, cheekily providing the tools the citizens wanted rather than those the government decided they needed.
If our aim was to wake governments up to the possibilities of digital, to some extent it has been successful. Governments around the world, at all levels, have seen the financial and societal benefits, and are producing, buying in, and commissioning civic software for their own online offerings.
It is, then, high time that the sphere of government-implemented civic technologies were more closely examined: how effective are they? Who is using them? What changes are they wreaking on the relationship between citizen and government? How, indeed, are governments themselves changing as a result of this new direction?
Civic Tech Cities
Thanks to generous funding from Microsoft, we were able to conduct research that seeks to answer these questions, in the context of municipal-level council digital offerings in five US cities.
Emily Shaw, in collaboration with mySociety’s Head of Research Rebecca Rumbul, examined standalone projects in Austin, Chicago, Oakland, Washington DC and Seattle, to produce case studies that cast a light on the state of institutional civic tech in the current age.
The technologies chosen for scrutiny were diverse in some ways, but the challenges they faced were often alike: and we can all, whether inside or outside government, recognise common pitfalls such as failing to budget for ongoing maintenance of a service that was expected to roll happily along, untended, for the foreseeable future; or building a world-changing digital service that fails to gain traction because its potential users never get to hear about it.
It’s our hope that local governments everywhere will benefit from this in-depth look at the tools US municipal governments have put in place, from LargeLots in Chicago which sold disused land in disadvantaged neighbourhoods for a nominal $1 fee, to RecordTrac in Oakland, a request and response tool for those seeking information under California’s Public Record Act.
Better tools make better policy
Interestingly, one of the key findings of this report is that developing digital tools alongside policy, rather than bolting these tools on afterwards, results not only in better tools, but better policy too.
The user-centred design principles that have been central to the Civic Tech movement had a knock-on effect beyond the software development departments of municipal government. They began to shape the ways in which policy itself was developed, resulting in services that were more accessible and appropriate to the communities they serve.
Finally, it’s not just governments who will learn from this examination of best practices, potential problems and unexpected bonuses; we, and other NGOs like us, can gain crucial insights from the sector which, after all, is pursuing the same aim that we are.
You can read the research paper here. Many thanks to Microsoft for making it possible, and to Emily Shaw for putting in the time and effort to make it a reality.
Image: Jindong H
Suppose we sent an automated tweet every time someone made a successful Freedom of Information request on WhatDotheyKnow — would it bring more visitors to the site?
And, if you get a response to your first FOI request, does it mean you are more likely to make a second one?
These, and many more, are the kind of questions that emerge as we refine the advice that we’re offering partner organisations.
Our Freedom of Information platform Alaveteli underpins Freedom of Information sites all around the world. When we first launched it, our only priorities were to make the code work, and to make that code as easy as possible to implement. But, as a community emerged around Alaveteli, we realised that we’d all be better off if we shared advice, successes and ideas.
And that’s where we began to encounter questions.
Some of them, like how to get more users, or how to understand where users come from, are common to anyone running a website.
Others are unique to our partner structure, in which effectively anyone in any part of the world may pick up the Alaveteli code and start their own site. In theory, we might know very little more than that a site is running, although we’ll always try to make contact and let the implementers know what help we can offer them.
There were so many questions that we soon saw the need to keep them all in one place. At mySociety, we’re accustomed to using Github for anything resembling a to-do list (as well as for its primary purposes; Github was designed to store code, allow multiple people to work on that code, and to suggest or review issues with it), and so we created a slightly unusual repo, Alaveteli-experiments.
This approach also gives us the benefit of transparency. Anyone can visit that repo and see what questions we are asking, how we intend to find the answers, and the results as they come in. What’s more, anyone who has (or opens) a Github account will also be able to add their own comments.
Some of the experiments, like this one to analyse whether people click the ‘similar requests’ links in the sidebar, we’re running on our own site, WhatDoTheyKnow. Others, such as this one about the successful requests listed on every Alaveteli site’s homepage, are being conducted on our partners’ sites.
Our aims are to find out more about how to bring more users to all Alaveteli sites, how to encourage browsing visitors to become people who make requests, and how to turn one-off requesters into people who come back and make another — and then pass all that on to our partners.
We hope you’ll find plenty of interest on there. We reckon it’s all relevant, especially to anyone running an FOI website, but in many cases to anyone wondering how best to improve a site’s effectiveness. And we’re very happy to hear your ideas, too: if we’ve missed some obvious experiment, or you’ve thought of something that would be really interesting to know through the application of this kind of research, you’re welcome to let us know.
Researching in an unstable environment
It’s been nearly two years since the InfoLib Liberia project with iLab Liberia started. In that time the project has faced many hurdles, some predicted, and some completely unforeseen.
The iLab team have seen their country devastated by Ebola, only 11 years after the end of their second civil war, bringing tragedy and instability along with it. As you can probably imagine, the impact of curfews, fear and death in communities has made it difficult for people to continue with their daily lives. The social impact of such a disease is wide-reaching. Distrust, marginalisation and exclusion can be directed at those who show symptoms, or even who suffered and survived.
These are challenges that our local partners have had to contend with every day, both when holding training sessions and more crucially when researching the impact of the project on people’s lives.
However, by far the largest hurdle for this particular project has been a mixture of low internet penetration and lack of government will to release information. The team on the ground have been working tirelessly to create an ecosystem of requesting and training Public Information Officers (PIOs) to reply – even providing them with tablets to scan documents without needing electricity, let alone a computer. But if those officers have no access to the information that has been requested, their jobs become virtually impossible.
The project is now drawing to a close and we’re undertaking our final research survey. It seemed like a good time to take a look at what we’ve learnt about the impact of our joint Freedom of Information project in Liberia.
When designing the project we decided that impact could best be measured in terms of whether or not the project increased confidence in government transparency.
We carried out surveys in January 2016 and April 2016, to provide a baseline picture and then an assessment of impact at midline. The final survey is being conducted in August 2016 just as the project ends.
The first survey – the baseline – was carried out mainly in the rural areas. iLab Liberia teamed up with LFIC to survey 152 participants who had been involved in the FOI workshops that LFIC had held in the counties.
We had to attempt the second survey twice, as it turned out to be more challenging than we’d expected. We needed the participants from the first questionnaire to answer the same questions we’d asked them initially, in order to measure change — but it proved hard to locate all of them.
There were many factors which caused this, but the main one was economic drivers, forcing people to move to where the opportunities are. It’s a problem many researchers must run into working in the field.
Carter, the project lead at iLab Liberia told us:
“There are several reasons why this happens […]. People migrate a lot between markets, farms. Several persons who participated in the baseline could not be reached as they [had] travelled to other cities/counties. [Or] the job that allowed them to reside in that city/county is no longer available so they might have left seeking after another job.”
Our second attempt was more successful. We managed to contact a large percentage of the original participants in the survey: 112 of the 152.
We’ve found out some interesting things from doing this research. We saw that 74% of people who use the internet daily say it’s their main source of information, though it is still only a small percentage of the population who have access to the internet.
So the next biggest source of information? Radio! 85% of people with with no access to the internet give radio as their main source of information. Thinking of the migration of workers between cities and counties – you suddenly appreciate why Radio is such an important medium for getting hold of information. Thankfully, as you’ll remember from our original blog post, we’re covering both of these media in the InfoLib project.
In the months since we began studying the impact of this project we also learned that fear of making a request has dropped by 5% in the individuals surveyed . The amount of people who reported that they didn’t know how to ask for information dropped from 24% to 21%. This is pretty great news to us as it shows that our training and our encouragement is working – albeit slowly.
Finally we saw the percentage of people who believe government would be more transparent if citizens could see the information they hold rise by 3% to 93% of the surveyed respondents. Even if this figure hadn’t risen, this demonstrates a clear existing demand from the citizens of Liberia for the Government to release more information about its activities which is great news overall!
No project is without its challenges, and as you’ve seen above one of the big ones is ensuring that the same people respond from survey to survey. Not being able to pin down precisely the same set of people means that we can’t say with 100% certainty that we have a true measure in the difference in attitude.
As a result of the economic and social drivers mentioned above, the workforce in Liberia is very transient. This makes disseminating information through radio and internet mediums even more important. This research has shown that these are the primary sources of news and official information for the majority of Liberians, and continuing to improve knowledge about, and access to, information via these sources will empower the population further.
Finally, it can be challenging to demonstrate impact in projects like these, simply because research is not the main focus for our local partners. We partner with local groups because they are passionate, capable, and able to engage and mobilise citizens around a certain issue. We cannot expect small grassroots groups to have the resources or experience to conduct academic surveying, sampling or interviewing that could detect and definitively isolate the short term impact of a small project. This piece of research has provided some encouraging interim results, but most of all, it has provided valuable lessons to us at mySociety in trying to conduct this kind of impact research remotely and in partnership.
While we wait for the outcome of the final survey we can feel cautiously hopeful that this project has caused a small change in the way access to information operates in Liberia. infoLib will continue to run after the project officially ends, and mySociety will continue to support the work that iLab does in this area . However it may take longer than we had expected or hoped, to see the governmental shift towards releasing information.
Videos from TICTeC — our Impacts of Civic Technologies Conference — are now available for viewing.
So, whether you were there and you want to experience it all over again, or you just want to see what you missed out on, settle down and enjoy accounts from some of the most interesting civic tech projects around the world right now.
What is TICTeC?
Here’s mySociety CEO Mark Cridge explaining what TICTeC is, and why we run it:
And if that’s left you hungry for more, make sure you visit our YouTube channel to watch the other TICTeC videos.
You might also enjoy the slides, photos and other media that have all been collected together on the TICTeC 2016 page.
TICTeC is our annual conference on the impacts of civic technologies.
This year, we’ll be in Barcelona, Spain, with a diverse programme of speakers from all over the world.
Guy Grossman, assistant professor at the University of Pennsylvania’s political science department, will deliver one of two keynotes. See details of our other keynote speaker, Helen Milner OBE, here.
Guy has had a long history in civic tech research, with a special focus on Uganda and Sub-Saharan Africa, and we are delighted that he will be sharing his insights to our audience of practitioners and researchers.
Hi Guy! Tell us what you’ll be covering in your TICTeC keynote.
The state of academic knowledge about the promises and pitfalls of ICT applications designed to increase voice and accountability in low-income countries. Specifically:
- Under what conditions are citizens more likely to communicate with government officials using mobile-platforms? If we “build them – will they come”?
- Do mobile platforms exacerbate inequalities in political access, privileging groups (men, urban, better-off) that are heavier users of ICTs?
- What can be done to help increase the participation rates of marginalised populations (such as women and the poor)?
Low-income countries have leapfrogged past the landline era and directly into the mobile era. Citizens can now connect with each other and with their public officials in unprecedented ways.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited.
The massive penetration of mobile technology, even to remote areas of the least developed countries, has great potential to reshape both the social and political landscapes.
Even though mobile technology is so pervasive, our collective understanding of many first-order questions in this area is surprisingly limited. My goal at TICTeC is to help take stock of what we know and don’t know in this emerging field.
What are you hoping to get out of TICTeC?
I’m hoping to interact with practitioners, policy makers, technologists and fellow academics to figure out where the overlap is between practical needs and academic research.
I’m also hoping to get exposed to promising and innovative applications that I am unaware of.
Your civic tech research focuses especially on Uganda and Sub-Saharan Africa. What led you down this path?
I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries.
I care deeply about poverty and social and gender inequality, so working and studying in Africa has been a natural choice.
How did I get involved with civic tech? In the past decade I have witnessed an increasing number of mobile-based applications designed to enhance such things as agriculture productivity and financial transactions or match between buyers and sellers.
On the other hand, I found very few applications and platforms, if any, designed to improve governance and accountability in low-income countries. I came to believe that this was a great void that needed to be filled.
In fact, unfortunately, we are still quite far from saturation in the development of governance-related ICT applications.
What do you think are the big unanswered questions when it comes to civic tech?
Does participation in mobile communication spill into more traditional forms of political participation?
Here’s a very partial list of some of the answered questions when it comes to civic tech in the global south:
Can simple innovations in mobile technology be used to facilitate new meaningful forms of political participation?
If so, what types of mobile-based political communication are most likely to be adopted by voters?
What is the potential of ICTs to flatten political access to marginalised populations?
What are some of the ways to increase the usage of mobile-based political communication in the face of clear collective action problems?
How can ICT be used to overcome political economy constraints to growth (e.g. by resolving monitoring challenges in government bureaucracies, reducing the cost for citizens of tracking bureaucratic performance, or making it easier to detect corrupt practices in public-sector salary distribution)?
And, does participation in mobile communication spill into more traditional forms of political participation?
If you could make one recommendation to those developing new civic tech, and wanting to see real impact from it, what would it be?
Adopt a user perspective — make sure that the suggested application/platform addresses a real need that could not be addressed using a low-tech solution.
You won’t want to miss what Guy has to say at TICTeC, so make sure you book your tickets now.
In my last post I described how we’re taking stock of where and how we’re delivering against our theory of change to give greater influence to citizens over those with power.
Since starting at mySociety I’ve spent my time meeting lots of lovely people, getting to know the team, our funders, partners and peers and finding out how mySociety does what it does.
One thing I have learned is that despite our British roots, the majority of our work is now international, and we work with wonderful partners in over 35 countries around the world, from Ukraine to South Africa, Liberia to Norway. In each case they tend to be activists, journalists and NGOs who are passionate about better government, citizen empowerment, and fighting corruption.
Our success is defined by our partners’ success – so in order to best support our partners I’d like to talk a little bit about some of the practical steps we’re taking to consolidate what we already have and scale up what works.
Four Simple Goals
The core mission of mySociety remains the same: to invent and popularise websites and apps that enable citizens around the world to exert power over institutions and decision makers.
We see the need to both ‘invent’ and ‘popularise’ digital tools as equally important – digital tools can be useful in developing new approaches to difficult problems, but we must ensure they are both widely used and actually enable citizens to be capable of demanding better.
In order to best help our partners and to better understand the impact of our work we have four really simple goals that will direct our efforts over the next few years:
1. Encourage more people
to use our websites and apps
in more countries
2. Work with more partners
to help them get better at
using digital tools
3. Prove what works and
feed those learnings back into
the wider community
4. Take a lead role in
making technology more useful
to civil society
Planning For Success
In addition to running our successful UK sites TheyWorkForYou.com, WhatDoTheyKnow.com, WriteToThem.com and FixMyStreet.com, we’ll continue to work with our partners to improve our existing services, making them easier to deploy and better integrated together.
We’ve recently established a quarterly call for new proposals for potential new partners who wish to set up new sites of their own from our roster of services (FOI, Parliaments and Elections, FixMyStreet). This helps inform potential partners of what’s involved before getting started, and helps us better target our resources and plan for success upfront.
We’re also putting more effort into increasing the impact and usage of our existing sites and services, by providing targeted development support, training, direct funding and additional technical development. Helping to sustain each site through the difficult first year or two should be a major marker of success.
Proving What Works
One major thing that will change is putting our research much more front and centre to our work, in order to create a greater evidence base for the impacts of civic technology and ensuring we are able to talk about this widely and publicly.
You’ll see us carry out much more inclusive and comparative impact research on the use of civic technology encompassing individual, socio-political and sector-specific factors.
If you haven’t already read our latest research paper ‘Who Benefits From Civic Technology?’ then please do have a look. This is an important first step in laying down the case for impact, being honest about where more work is required and focusing our efforts to create a greater evidence base for civic technology as a whole.
Our long-term aim is to establish a global hub for impact research, and assist more civic tech organisations to assess and improve the impact of their own work. To this end we’ll be hosting our next TICTeC – The Impacts of Civic Technology Conference, in Barcelona on the 27th and 28th of April next year.
This will be an important opportunity to share and discuss research findings and key challenges from across the sector and we hope to see many of you there in person.
Where We Go Next
Over the past decade, through a process of experimentation, consultation and measurement, mySociety have created a portfolio of popular, proven online services, used by over 10 million people each year.
This is an amazing legacy to take on.
Over the next decade I hope that we’ll continue this work, and seek to further establish mySociety as one of the leading international civic technology institutions, providing much-needed global leadership and inspiration in our sector – if we could come to be seen as having a similar impact to that of an Article 19 or Human Rights Watch in our own field, then I think that will be a pretty good measure of success.
For the moment we’ll continue to focus on the practical steps we need to take in order to improve and build upon what we already have, but I’m excited about the plans we have for the future and I’ll share more details on what we have in store in the weeks to come.
How do you pin down the intangible?
More specifically, how do we understand something as nebulous as why people visit our websites? For better or worse, Google Analytics can’t provide brain scans to show our users’ motivations, so the only solution is to ask them.
Between May and August of this year, if you visited TheyWorkForYou, you might have seen a pop-up inviting you to answer a few questions. Some users also gained new site features. It was all part of a concerted drive to understand more about why you use the site, and part of our wider research programme funded by The William and Flora Hewlett Foundation.
Many thanks if you were one of those who responded. In all, the survey generated a couple of a million data points, which has certainly given our resident data-cruncher, Nick, plenty to wade through.
Why do people use TheyWorkForYou?
We know why we run TheyWorkForYou: it aims to make Parliament—often perceived as ‘not for the likes of me’—more accessible for everyone. But what we don’t know is whether or not we are achieving that aim.
You see, with our other sites, WhatDoTheyKnow, FixMyStreet or WriteToThem, there is a set path that indicates a successful visit. The user lodges an FOI request, reports a problem, or writes to their representatives.
TheyWorkForYou, on the other hand, does not promote an action. There are actions you can take, like signing up for email alerts, or adding an annotation to a debate—but these are secondary to the site’s primary function, which is simply to present information. There’s no measurable metric that can tell us how informed people are after they’ve read one of our pages, nor what they do with that information. Hence the survey.
People hate pop-ups
One of our first findings came as no surprise: yes, we hate pop-ups too*! And yet, we wanted to keep users on the site while they answered our questions, so that’s the interface we used.
75.7% of people dismissed the pop-up when it appeared. We’d probably have done the same. We’d like to meet the remaining 24.3% of people and shake their hands.
The pop-up asked users what they would go on to do with the information they’d found on TheyWorkForYou. And most people gave the same answer they would if a shop assistant had asked: in fact, 75.2% said they were ‘just browsing’, with no further plans.
That makes us really curious as to what provoked that browsing: are they regular visitors, or were they sent to TheyWorkForYou via a tweet or Facebook reminder? Maybe one of our email alerts was the trigger. Google Analytics will allow us to see which site referred visitors to us, but of course we can’t match that data with individual motivations.
A further 10.9% of users had come with the intention of contacting their MP: TheyWorkForYou’s MP pages feature a prominent button inviting you to do that, and clicking on it takes you to our site WriteToThem.
6.6% of people were using the site for work purposes. Only 3.1% of people had plans to share content via social media, a statistic we’d like to increase if we can.
During the research period, some users (not necessarily those who had seen the pop-up) saw an addition to person pages and debate pages: a “where next” button, which led to some suggestions on how to use the information they’d found on the site.
This button was not widely used: just 0.1% of people clicked it. As so often with research findings, this result leads to more questions: did people already know what they wanted to do? Did they not want to do anything? Were they, perhaps, annoyed at the suggestion they might need guidance?
Or was the button just poorly positioned? We may run some further testing to find out if a more prominent placing leads to more clicks.
Of those few people who did click through for advice, the most popular options were to use WriteToThem to contact their MP about what they’d just read, or to contact the Parliamentary Ombudsman—it seems that on the few occasions when people do welcome a next step, they do want to take action.
Follow the news
Meanwhile, we were also taking a concentrated look at our site analytics. The test period coincided with the Labour party leadership contest and David Cameron’s cabinet announcements, and we saw very clearly how current affairs have an effect on which pages people visit.
John Whittingdale (the new Culture Secretary)’s voting record and Jeremy Corbyn’s profile page were far and away the most visited pages by several thousand (they’re responsible for the spike on the left of the graph that you see below). Also popular were the profile pages of the other Labour candidates, and Corbyn’s voting record.
But the other pages visited were very diverse: an extreme example of the long tail effect.
We reckon it takes more than a few seconds to take in a parliamentary debate, so one of the things we measure on Google Analytics is how many people stick around for more than 7 seconds, and again, for more than ten seconds.
Actually, we get pretty good results for the 7-second cut-off. Recent work we did on the design of the site’s debate pages had an effect that was a real cause for celebration: a massive 46% drop in the bounce rate (ie people who leave the site after viewing just one page)
But only 38% then stick around long enough for the second trigger, which doesn’t indicate as much involvement as we might have hoped for.
What have we learned?
At the end of this experiment, we reckon we’ve learned four main things:
- We need to carry on thinking about how to encourage more engagement with TheyWorkForYou’s content
- Current affairs are a huge driver to the site, and we can build on this via social media—and especially by encouraging our users to share the content they’ve found
- We need to conduct more experiments to see whether people genuinely don’t want advice about how to use the information they’ve found, or whether they would take it if it was more prominently flagged up
- There’s also a motivation to dig in deeper to motivations: what made people come to the site, if they are ‘just browsing’? What exactly are they doing ‘for work’? And if they want to contact their MP, what was the trigger for that?
If you have thoughts about any of these, as regards your own usage of TheyWorkForYou, please do feel free to comment below. And if you’d like to find out more about our research programme there’s plenty of information here, along with news of our annual research conference, TICTeC.
*And we know who to blame.
Image: The U.S. Printing Co., Russell-Morgan Print, Cincinnati & New York [Public domain] via Wikimedia Commons.