How do you pin down the intangible?
More specifically, how do we understand something as nebulous as why people visit our websites? For better or worse, Google Analytics can’t provide brain scans to show our users’ motivations, so the only solution is to ask them.
Between May and August of this year, if you visited TheyWorkForYou, you might have seen a pop-up inviting you to answer a few questions. Some users also gained new site features. It was all part of a concerted drive to understand more about why you use the site, and part of our wider research programme funded by The William and Flora Hewlett Foundation.
Many thanks if you were one of those who responded. In all, the survey generated a couple of a million data points, which has certainly given our resident data-cruncher, Nick, plenty to wade through.
Why do people use TheyWorkForYou?
We know why we run TheyWorkForYou: it aims to make Parliament—often perceived as ‘not for the likes of me’—more accessible for everyone. But what we don’t know is whether or not we are achieving that aim.
You see, with our other sites, WhatDoTheyKnow, FixMyStreet or WriteToThem, there is a set path that indicates a successful visit. The user lodges an FOI request, reports a problem, or writes to their representatives.
TheyWorkForYou, on the other hand, does not promote an action. There are actions you can take, like signing up for email alerts, or adding an annotation to a debate—but these are secondary to the site’s primary function, which is simply to present information. There’s no measurable metric that can tell us how informed people are after they’ve read one of our pages, nor what they do with that information. Hence the survey.
People hate pop-ups
One of our first findings came as no surprise: yes, we hate pop-ups too*! And yet, we wanted to keep users on the site while they answered our questions, so that’s the interface we used.
75.7% of people dismissed the pop-up when it appeared. We’d probably have done the same. We’d like to meet the remaining 24.3% of people and shake their hands.
The pop-up asked users what they would go on to do with the information they’d found on TheyWorkForYou. And most people gave the same answer they would if a shop assistant had asked: in fact, 75.2% said they were ‘just browsing’, with no further plans.
That makes us really curious as to what provoked that browsing: are they regular visitors, or were they sent to TheyWorkForYou via a tweet or Facebook reminder? Maybe one of our email alerts was the trigger. Google Analytics will allow us to see which site referred visitors to us, but of course we can’t match that data with individual motivations.
A further 10.9% of users had come with the intention of contacting their MP: TheyWorkForYou’s MP pages feature a prominent button inviting you to do that, and clicking on it takes you to our site WriteToThem.
6.6% of people were using the site for work purposes. Only 3.1% of people had plans to share content via social media, a statistic we’d like to increase if we can.
During the research period, some users (not necessarily those who had seen the pop-up) saw an addition to person pages and debate pages: a “where next” button, which led to some suggestions on how to use the information they’d found on the site.
This button was not widely used: just 0.1% of people clicked it. As so often with research findings, this result leads to more questions: did people already know what they wanted to do? Did they not want to do anything? Were they, perhaps, annoyed at the suggestion they might need guidance?
Or was the button just poorly positioned? We may run some further testing to find out if a more prominent placing leads to more clicks.
Of those few people who did click through for advice, the most popular options were to use WriteToThem to contact their MP about what they’d just read, or to contact the Parliamentary Ombudsman—it seems that on the few occasions when people do welcome a next step, they do want to take action.
Follow the news
Meanwhile, we were also taking a concentrated look at our site analytics. The test period coincided with the Labour party leadership contest and David Cameron’s cabinet announcements, and we saw very clearly how current affairs have an effect on which pages people visit.
John Whittingdale (the new Culture Secretary)’s voting record and Jeremy Corbyn’s profile page were far and away the most visited pages by several thousand (they’re responsible for the spike on the left of the graph that you see below). Also popular were the profile pages of the other Labour candidates, and Corbyn’s voting record.
But the other pages visited were very diverse: an extreme example of the long tail effect.
We reckon it takes more than a few seconds to take in a parliamentary debate, so one of the things we measure on Google Analytics is how many people stick around for more than 7 seconds, and again, for more than ten seconds.
Actually, we get pretty good results for the 7-second cut-off. Recent work we did on the design of the site’s debate pages had an effect that was a real cause for celebration: a massive 46% drop in the bounce rate (ie people who leave the site after viewing just one page)
But only 38% then stick around long enough for the second trigger, which doesn’t indicate as much involvement as we might have hoped for.
What have we learned?
At the end of this experiment, we reckon we’ve learned four main things:
- We need to carry on thinking about how to encourage more engagement with TheyWorkForYou’s content
- Current affairs are a huge driver to the site, and we can build on this via social media—and especially by encouraging our users to share the content they’ve found
- We need to conduct more experiments to see whether people genuinely don’t want advice about how to use the information they’ve found, or whether they would take it if it was more prominently flagged up
- There’s also a motivation to dig in deeper to motivations: what made people come to the site, if they are ‘just browsing’? What exactly are they doing ‘for work’? And if they want to contact their MP, what was the trigger for that?
If you have thoughts about any of these, as regards your own usage of TheyWorkForYou, please do feel free to comment below. And if you’d like to find out more about our research programme there’s plenty of information here, along with news of our annual research conference, TICTeC.
*And we know who to blame.
Image: The U.S. Printing Co., Russell-Morgan Print, Cincinnati & New York [Public domain] via Wikimedia Commons.
Every year, thousands of people in the UK fall prey to financial scams. For the last six months, mySociety has been working with the Financial Conduct Authority to create an online tool, the FCA Warning List, to try and help potentially vulnerable financial consumers avoid scams.
The tool launched on Wednesday last week as part of a wider communications campaign aimed at educating the public on the typical characteristics of scams.
The homepage of the FCA Warning List
The sophistication of financial scams
When I’ve mentioned this project to people, they invariably think of emails from extravagantly-named lawyers in far-flung countries identifying you as the lucky beneficiary of a lost inheritance. These scams seem, to most people, obvious and the people who fall prey easy to ridicule: how can people possibly think this stuff is real?
But many scams are actually incredibly sophisticated and work on people over a long period of time, with even the most experienced investors at risk of being taken in.
An introductory approach is made by someone who knows your name on a realistic pretext. Might you be interested in this investment? No? Well, John, it was good to chat and I’ll call again in a few months. A few months later another call. Did you know that if you’d invested you’d have made 20% by now? I have another investment here that I think might interest you, let me introduce you to my colleague who knows the director at this firm personally.
Scammers work their victim as a team, they lure us in and use our cognitive flaws and emotional weaknesses to ensnare us.
Disconcertingly informative user research
During this project, we interviewed 15 or so potential users, recruited as having the characteristics of people likely to be targeted by scammers. About half of these people had come in contact with a scam. As far as we know, none of them had fallen victim but only one of them reported their experience to the authorities.
Almost all of them thought that a website to help them research investment opportunities—provided by an independent third party like the FCA—would be useful for them.
We’ve built an online tool that we hope will help investors avoid scams. A person using it gives it some basic facts about an investment opportunity (what is it an investment in? how did you hear about it?) and it gives them back some guidance specific to them.
For example, did you know that if you’re cold called about an investment opportunity, in all likelihood it’s a scam? Just that simple message alone has the potential to be very disruptive to scams.
The tool also allows users to search a list of firms that the FCA knows to be operating without their permission: if someone’s talking to a firm on that list about an investment, that’s probably a pretty bad idea.
Finally, it gives people other suggested actions at the end of the process. Been cold called? You should report it. Considering an investment in an unregulated commodity like bamboo? You should research further before investing.
We also made a widgetty version of the tool homepage that can be easily embedded into other websites.
Scammers rob people of cash, but they can also take people’s self-respect and damage their personal relationships, often at a time of life when they are particularly vulnerable.
We’re excited about the launch of the Warning List and are proud to be supporting the FCA in its mission to protect consumers by disrupting more of the scams that have such a destructive effect on people’s lives.