How do you pin down the intangible?
More specifically, how do we understand something as nebulous as why people visit our websites? For better or worse, Google Analytics can’t provide brain scans to show our users’ motivations, so the only solution is to ask them.
Between May and August of this year, if you visited TheyWorkForYou, you might have seen a pop-up inviting you to answer a few questions. Some users also gained new site features. It was all part of a concerted drive to understand more about why you use the site, and part of our wider research programme funded by The William and Flora Hewlett Foundation.
Many thanks if you were one of those who responded. In all, the survey generated a couple of a million data points, which has certainly given our resident data-cruncher, Nick, plenty to wade through.
Why do people use TheyWorkForYou?
We know why we run TheyWorkForYou: it aims to make Parliament—often perceived as ‘not for the likes of me’—more accessible for everyone. But what we don’t know is whether or not we are achieving that aim.
You see, with our other sites, WhatDoTheyKnow, FixMyStreet or WriteToThem, there is a set path that indicates a successful visit. The user lodges an FOI request, reports a problem, or writes to their representatives.
TheyWorkForYou, on the other hand, does not promote an action. There are actions you can take, like signing up for email alerts, or adding an annotation to a debate—but these are secondary to the site’s primary function, which is simply to present information. There’s no measurable metric that can tell us how informed people are after they’ve read one of our pages, nor what they do with that information. Hence the survey.
People hate pop-ups
One of our first findings came as no surprise: yes, we hate pop-ups too*! And yet, we wanted to keep users on the site while they answered our questions, so that’s the interface we used.
75.7% of people dismissed the pop-up when it appeared. We’d probably have done the same. We’d like to meet the remaining 24.3% of people and shake their hands.
The pop-up asked users what they would go on to do with the information they’d found on TheyWorkForYou. And most people gave the same answer they would if a shop assistant had asked: in fact, 75.2% said they were ‘just browsing’, with no further plans.
That makes us really curious as to what provoked that browsing: are they regular visitors, or were they sent to TheyWorkForYou via a tweet or Facebook reminder? Maybe one of our email alerts was the trigger. Google Analytics will allow us to see which site referred visitors to us, but of course we can’t match that data with individual motivations.
A further 10.9% of users had come with the intention of contacting their MP: TheyWorkForYou’s MP pages feature a prominent button inviting you to do that, and clicking on it takes you to our site WriteToThem.
6.6% of people were using the site for work purposes. Only 3.1% of people had plans to share content via social media, a statistic we’d like to increase if we can.
During the research period, some users (not necessarily those who had seen the pop-up) saw an addition to person pages and debate pages: a “where next” button, which led to some suggestions on how to use the information they’d found on the site.
This button was not widely used: just 0.1% of people clicked it. As so often with research findings, this result leads to more questions: did people already know what they wanted to do? Did they not want to do anything? Were they, perhaps, annoyed at the suggestion they might need guidance?
Or was the button just poorly positioned? We may run some further testing to find out if a more prominent placing leads to more clicks.
Of those few people who did click through for advice, the most popular options were to use WriteToThem to contact their MP about what they’d just read, or to contact the Parliamentary Ombudsman—it seems that on the few occasions when people do welcome a next step, they do want to take action.
Follow the news
Meanwhile, we were also taking a concentrated look at our site analytics. The test period coincided with the Labour party leadership contest and David Cameron’s cabinet announcements, and we saw very clearly how current affairs have an effect on which pages people visit.
John Whittingdale (the new Culture Secretary)’s voting record and Jeremy Corbyn’s profile page were far and away the most visited pages by several thousand (they’re responsible for the spike on the left of the graph that you see below). Also popular were the profile pages of the other Labour candidates, and Corbyn’s voting record.
But the other pages visited were very diverse: an extreme example of the long tail effect.
We reckon it takes more than a few seconds to take in a parliamentary debate, so one of the things we measure on Google Analytics is how many people stick around for more than 7 seconds, and again, for more than ten seconds.
Actually, we get pretty good results for the 7-second cut-off. Recent work we did on the design of the site’s debate pages had an effect that was a real cause for celebration: a massive 46% drop in the bounce rate (ie people who leave the site after viewing just one page)
But only 38% then stick around long enough for the second trigger, which doesn’t indicate as much involvement as we might have hoped for.
What have we learned?
At the end of this experiment, we reckon we’ve learned four main things:
- We need to carry on thinking about how to encourage more engagement with TheyWorkForYou’s content
- Current affairs are a huge driver to the site, and we can build on this via social media—and especially by encouraging our users to share the content they’ve found
- We need to conduct more experiments to see whether people genuinely don’t want advice about how to use the information they’ve found, or whether they would take it if it was more prominently flagged up
- There’s also a motivation to dig in deeper to motivations: what made people come to the site, if they are ‘just browsing’? What exactly are they doing ‘for work’? And if they want to contact their MP, what was the trigger for that?
If you have thoughts about any of these, as regards your own usage of TheyWorkForYou, please do feel free to comment below. And if you’d like to find out more about our research programme there’s plenty of information here, along with news of our annual research conference, TICTeC.
*And we know who to blame.
Image: The U.S. Printing Co., Russell-Morgan Print, Cincinnati & New York [Public domain] via Wikimedia Commons.