When a user on FixMyTransport tells us about a journey they’ve had, we ask them if it was on a bus, train, underground, or ferry. This is a simple question, but of course our lead developer Louise thought carefully about how to ask it.

As programmers, we sometimes approach collecting data from the user just like filling in a form — a paper form. So, we could ask the bus/train/ferry question by using a list to select from (in HTML, that’s a select tag) — perhaps spread out, or as a drop-down list — or maybe as a set of radio buttons. But in this case we have deliberately broken away from idea of the form.

Instead, FixMyTransport uses four big buttons. They look like this:

Effectively, this means we have a whole webpage devoted to one single question. That’s perhaps not what you’d expect if you’re building an online form. Often it seems easier not to break a task across several pages. But here we have a single page with a single question on it.

There are some important things to note about this page:

• the four buttons are big and colourful and beautiful (we can thank Supercool for this, who did the design work on the initial FixMyTransport site)
• it’s a very easy question, even though the answer is critical (because it affects the kind of data we need to ask next)
• it’s a reasonable question — the user isn’t surprised or confused to be asked it

This page has been very successful. We know this because we study our web analytics (that is, how people use our sites) as well as running usability testing. It’s true to say nobody gets stuck on this page, nobody drops out, and in fact most people don’t even think about it (that’s the “reasonable” thing, above).

After the latest session of FixMyTransport usability testing, run by our developer Mark, we had a discussion with other team members. We agreed this was one of our favourite FixMyTransport pages. It’s pretty, it does the job, it moves the user forward (see the earlier blog post about “why the FixMyStreet homepage asks one easy question”)… and best of all it shows that some of us were wrong to be cautious about introducing another step into the problem-reporting process.

Incidentally, Mark recommends the book Rocket Surgery Made Easy if you’re interested in running your own usability tests.

In summary, sometimes a super-simple choice is strong enough to be presented as a single webpage all by itself. Don’t fall into the trap of thinking web forms should be like paper forms.

When you report a problem on FixMyStreet.com, the site displays a map for you to click on to indicate its exact location. Of course, you can zoom in and out of that map, but when it is first displayed, FixMyStreet needs to use an initial ‘default’ zoom level. Ideally, this is a zoom level that reduces the number of clicks required before a user can pinpoint the location of their problem.

And here’s where we encounter a tricky problem. The world is a varied place – some towns are very dense with buildings and streets crammed close together. In these areas you need to default to a zoom that’s quite ‘close in’, otherwise it can be hard to locate your problem.

But out in the countryside, we have the opposite problem. You can have huge areas where there’s nothing but blank fields or moorlands. If the default map zoom is ‘close in’ here then users are likely to see a big map full of nothingness, or maybe just a single stretch of unidentifiable road.

So, what is to be done?

The answer is this – every time you search for a location in FixMyStreet the website does a check to see whether the location you typed is in an area where a lot of people live, or very few people live.

mySociety has been storing this population density data in a webservice which we call Gaze. If the area you searched for is in a densely populated area we assume that it must be an urban location, and the map starts with a helpfully zoomed-in map. But if you’re in a sparsely populated area then it’s probably rural, so FixMyStreet starts zoomed out, making it easier to get an overview of the whole area.

Where do we get the data from? Our late colleague Chris discusses this in a blog post from 2005 — the short answer is NASA SEDAC and LandScan. It’s an interesting example of how unexpected things can happen when data is made public — if population density wasn’t available to us, we wouldn’t have been able to put this small but clever detail into FixMyStreet’s interface.

When you report a problem on FixMyStreet, we ask you to click on a map to pinpoint its location. For example, if you want to tell the council a tree has fallen over, or there is a hole in the road, you can click exactly where it is. This is an easy way to provide the most accurate information in the report that we then send to the council.

If you’re a programmer then you probably think that it’s obvious to use a map for such problem reporting. We agree: maps are ideal for this, and it’s a shame that so many councils still aren’t doing it this way.

However, even though it’s very useful to have an accurate location for a problem, with FixMyStreet there are several reasons (below, we list just four) why someone might not be able to simply “click on the map”. In these cases, the map is no longer helpful — it’s a barrier. So we have to ask: is an accurate location so important that we should refuse to accept reports without one?

It turns out, for FixMyStreet, we don’t really need that accuracy. Sure, it’s best if we have it, and over 95% of reports do contain pinpoint locations, because most people do click on the map. But if we don’t have that location data, then we can still make a fair guess from the postcode or area name that the user has already provided (that’s how we knew which map to display, after all). So we allow the user to submit the report without clicking on the map.

Consequently, every time someone reports a problem without using the map, it means they’ve not given up, or clicked on the wrong place just to submit the form. In fact, they’ve reported a problem because we removed what would otherwise have been a barrier.

So, here are some reasons why we didn’t make clicking on the map mandatory in FixMyStreet:

• Map-reading isn’t a skill everybody is comfortable with
  When you’re building and testing FixMyStreet, you tend to imagine people will be reporting problems in places they know well. It’s easy to find somewhere on a map you are familiar with. But FixMyStreet users could just as easily be reporting a problem they passed on their way to work, on holiday, or at a party. So if they can’t read maps well it might be difficult or frustrating to locate a unfamiliar road junction or building.
• Navigating a map requires more challenging user skills
  FixMyStreet is easy to use, deliberately — in fact you can report a problem just using your keyboard. Zooming and panning a map element is much harder than any other part of the process. Remember that if you’re building a website like this yourself you are already comfortable with using complex controls that lots of people — FixMyStreet-using people — are not. This is true even before considering other limitations such as small-screen devices or visual disabilities.
• The map might not be helpful
  We rely on third-party maps. Most of the time, they are excellent. But what if the map is out of date? What if the problem is on a new road which hasn’t been added yet? What if the user remembers that the pothole was outside a distinctive shop or remarkable tree, only to find such landmarks aren’t on the map?
• The problem might not have a location
  Potholes are easy: they have a fixed position on the road. What about smells or flooding? These problems sometimes simply don’t seem to have an obvious pinpoint location.

In summary: we think clicking on a map is the best way to ask for a location from FixMyStreet users. But if we forced everybody to do it, some problems would never be reported, and some people would never become FixMyStreet users.

Remember how we put one simple question on the FixMyStreet homepage? This is possible because we know what most people coming to the page will want to do. So, because we’re confident about their intention, there’s something else we do too. We set the focus to that input. That is, we use JavaScript to put the cursor into that input box.

This means that you can go to www.fixmystreet.com and start typing your postcode straight away. There’s no extra click to put your cursor in the input box.

“Folk Andal. Castañuelas” by calafellvalo (CC BY-NC-ND licensed from Flickr)

Does it matter? On any single visit, if we didn’t do this, then probably this happens: you simply click in the box, and start typing, and do not even think about it. No problem! But sometimes you don’t notice that the cursor is not in the text box, and you start typing, and those letters do not appear. You look up, and you are a tiny bit annoyed; you click in the box and type it in again. That’s all.

But remember we’ve had well over one hundred thousand different users access FixMyStreet since it went live. On a site that gets that busy, even if we annoyed only 1% of those users on the homepage, it adds up to a lot of frowning. A user who is annoyed on the homepage is more likely to give up — and remember that the homepage often is not the first page they came to, so some users will already be getting tired of unnecessary clicking. This is why we do believe it is worth getting things like this right, and that it does matter.

In fact, in this case there are no other input boxes on the homepage: all the other interaction is by clicking on a link. This means that everything that page offers is just one click away (and yes, the way browsers handle forms means you can press Enter instead of clicking “GO” when you’ve finished typing, but we count that as one click).

This is a very small detail, and almost nobody would notice it if we did not point it out. But actually, that is one of the reasons we’re posting these design tips: good design, especially usability design, usually means you don’t notice when an interface is doing the right thing.

This tip is easy, and it builds on two earlier ones: don’t force your users to make any more clicks than absolutely necessary. Identify what they’ll probably be trying to do, and let them begin that as soon as possible, especially on the homepage.

"Click beetle (Agrypnus murinus)" by Rachel Scopes
(CC-BY-NC-ND-licensed from Flickr)

There are three things to think about here:

• Is there anything you can do to remove an extra click for a user? Do you really, really need that information from them? Or do you really need them to press ‘OK’ here?
• Too many clicks and users will abandon their task especially on sites where people are engaging with government or politics for the first time
• Most users began their task before they arrived at the site

The second point is especially relevant to users of civic sites like the ones we build. This is different from other websites. For example, if you go online to book a flight, or pay tax, or buy flowers because you forgot your mother’s birthday, even if the website is difficult or frustrating to use, you will stay on it until you have finished. The result is more important than the barriers: you need to finish. But people who are thinking about reporting a pothole they just missed on their bicycle that morning, or writing to their parliamentary representative for the first time, or making a freedom of information request — well, these people are already making an unusual effort. If you do anything at all to make that harder for them, then they can and they will give up. Civic sites must make the task so easy that nothing discourages users from finishing the mission they started.

The third point is more subtle. Earlier this year, Francis Irving explained how clicking on Google was part of the product. This means that, by the time a user has arrived on your site, they may already be many clicks and key-presses into the process. The more you prolong this effort, and especially if you prolong it with unnecessary or annoying extra clicks, the more likely it is that they will give up.

A user who gives up does not accomplish the task you wanted them to do.
A user who gives up might never bother to try again.
Civic engagement can be that fragile, especially for first-time users.

As we’ve already pointed out in a previous blog post, it’s likely that someone will arrive at your site on the “wrong” page, and so they will click again to get to your homepage. That click has already prolonged their journey. Click-fatigue starts to set in. You must help them to start doing something — instead of still clicking around trying to get to the place where they can start — as soon as possible. This is why you should think of the homepage as the centre of your website: it’s not the cover page or an introduction — it’s where people already on your site go to get things get done.

Or, to put it another way, by the time users have got to your homepage, perhaps they won’t have any spare clicks left.

Of course, we have to be pragmatic about this and make sensible guesses about what users will be trying to do — it’s silly to try to cover everything. We study our web analytics and do user-testing, but even if you can’t do that, it’s usually fairly easy to identify the key tasks. We don’t just think of the homepage as an introduction, or a cover page, or a portal (although it can also sometimes serve those purposes): ideally, for the most common tasks, it must be the start. The task does not need to be finished — that can be several pages further on. As we showed in an earlier blog post, we just need to get users started.

Let users make a useful click — even better, a successful click — and they stop thinking about giving up, and start getting things done.

If you’re building any civic or democratic website, we know that you’ll want to make its homepage beautiful and unique. After all, you want to make a good impression on your users, and first impressions are the ones that count.

Go ahead! Dive in and make it special… but also take our advice and don’t burn all your energy on building a perfect homepage. The reason why is that many of your users will come to your site without ever seeing your homepage.

The main reason for this is the influence of search engines, Twitter and Facebook. These have already changed users’ behaviour during the short history of the World Wide Web: there was a time when people could be expected to start on the front of your site and work inwards, clicking on the links you provided, step by step. But several  things now work against this: search engine results link to pages deep within your website; links are easily and routinely shared (email, Twitter, Facebook, Pinterest, even QR codes); and people are becoming accustomed to simple URLs that they might compose by hand, such as mysociety.org/contact. Realistically you must expect a user’s first page to be any* page on your website.

This means that, on every page except the homepage, your users may have arrived without any context or history — and, importantly, without the benefit of all the fabulous design time you’ve put into that glorious homepage.

“Teleport” by mercurialn (CC BY-NC-ND licensed from Flickr)

Spend some time thinking about how you are treating those people. Imagine they’ve been teleported into a strange land; they know, roughly, what planet they’re on, but otherwise it’s all new to them. That’s how bewildering this could be to a user who’s clicked on a link that drops them deep into your website. It’s a common experience and it’s easy to get right, but if you get it wrong, they will just as easily teleport out again. And they will think: “that is not somewhere I want to visit again.”

None of this is very radical. But the point is that it’s very easy to overlook — especially because, if you’re one of the developers working on a site, you already know too much about how it works to be objective when you look at what you’ve made.

Here are some things you should consider:

• Do your pages instantly say what they’re for? A new user landing on an unfamiliar page will grope for some explanation – “This is a page about a politician”, or “This is a page about problems in this street”. You need to make sure you spell it out somewhere easy to see.
• Can the user easily take an action from this page? If the user was looking for more than just a quick hit of information, the chances are that they want to get something done. Don’t fall into the trap of assuming that, just because they’ve landed on this page, this is the page where they want to be. For example, if you look at any one problem report on FixMyStreet and you will find a big “Report a problem” link in yellow at the top of the page.

So you should of course create a beautiful, functional homepage, but you should never forget that almost every page on your website will be a homepage for someone.

* Technical note: as you’d expect, there are a few exceptions to this (admin pages and so on), but in general all our sites deliberately offer their data pages with URLs that allow such indexing.

“Clock of the long now (prototype)” by piglicker (CC-BY-NC-ND-licensed from Flickr)

One of the democratic software tools mySociety has built is an online petitions tool for use by governments. We built the first version for the British Prime Minister’s office, and now it is used by a variety of British local governments.

When a user wants to make a new petition, they fill in a web form, containing the petition details.

During this process, a user must choose when they want their petition to end.

The question we are exploring today is this: what’s the best way to ask for that information?

From a developer’s point of view, the end of a petition is obviously a date. This is because the petition software really needs to test whether or not today’s date comes after the petition’s end-date. So an easy — and accessible — way to ask for this data would be as a date input, which could even helpfully trigger a calendar pop-up menu or similar widget.

But we don’t do it like that, for a couple of reasons, both of which you might miss if you were to build this from scratch. Instead, we ask:

For how long would you like your petition to accept signatures? (e.g., “1 month”; maximum 1 year)

That’s not a date input. But why?

The first reason is that the user is thinking about running a petition as something they plan to do for a period of time. Put simply, if you’re asked about your next holiday, you’ll probably imagine what you’ll be doing (“I’m going to spend two weeks lying on the beach”) rather than the end of it (“I’m going to stop lying on the beach on the 25th of August”).

In terms of interface design, this means that users will tend to think about how long something lasts differently than from how a computer or a coder will tend to think about it. The general rule is: even for something as simple as this little web form, if it’s possible to create an interface that matches the way that a user thinks about the world, then it will be less confusing for them than one that does not. Usually this means the system has to be a bit more complicated to program, in order to convert what the user types to what the computer needs.

The second reason that we ask for a duration, not a date, is that in the case of the petition, the user doesn’t actually know when their petition is going to start. Although they’re entering it into the website right now, there will nearly always be a delay before the government approves it. So most of the time, by asking for a duration, we’re avoiding raising this issue at all, and the user simply says what they mean, and it works. A petition with a duration of “3 months” will run for three months, once it goes live. A date wouldn’t survive an unspecified delay like that.

So, we ask for a duration and parse the result to turn it into a date when it goes live… but there’s an extra twist. What if the user really does know the date their petition must end? Maybe it’s a petition about closing the main road for a New Year’s Party, and it’s got to be discussed at the council meeting that’s being held on the 5th of December. Well — OK! If the user puts a date into the duration field, even though we didn’t ask for a date… we accept it. We can do some validation to make sure it’s a sensible date, of course, but basically if someone does puts a date in there, then that’s fine too.

What we hope this has shown you is that even something as simple as asking a user for an end date can be done easily (just use a date input there) or thoughtfully. The system wouldn’t break if we did it the other way. But making things as convenient as this means that we are allowing users to focus on the thing we’re trying to let them do, instead of drawing attention to how we are making them do it.

Simple things are the most easily overlooked. Two examples: a magician taking a wand out of his pocket (see? so simple that maybe you’ve never thought about why it wasn’t on the table at the start), or the home page on www.fixmystreet.com.

The first thing FixMyStreet asks for is a location. That’s so simple most people don’t think about it; but it doesn’t need to be that way. In fact, a lot of services like this would begin with a login form (“who are you?”) or a problem form (“what’s the problem you want to report?”). Well, we do it this way because we’ve learned from years of experience, experiment and, yes, mistakes.

We start off by giving you, the user, an easy problem (“where are you?”) that doesn’t offer any barrier to entry. Obviously, we’re very generous as to how you can describe that location (although that’s a different topic for another blog post). The point is we’re not asking for accuracy, since as soon as we have the location we will show you a map, on which you can almost literally pinpoint the position of your problem (for example, a pothole). Pretty much everyone can get through that first stage — and this is important if we want people to use the service.

How important? Well, we know that when building a site like FixMyStreet, it’s easy to forget that nobody in the world really needs to report a pothole. They want to, certainly, but they don’t need to. If we make it hard for them, if we make it annoying, or difficult, or intrusive, then they’ll simply give up. Not only does that pothole not get reported, but those users probably won’t bother to try to use FixMyStreet ever again.

So, before you know it, by keeping it simple at the start, we’ve got your journey under way — you’re “in”, the site’s already helping you. It’s showing you a map (a pretty map, actually) of where your problem is. Of course we’ve made it as easy as possible for you to use that map (and, yes, we also let you skip it if for some reason a map doesn’t help you). You see other problems, already reported (maybe you’ll notice that your pothole is already there — and we haven’t wasted any of your time making you tell us about it, even though you probably didn’t realise we intended right from the start to show you other people’s problems before you reported your own). Meanwhile, behind the scenes, we now know which jurisdictions are responsible for the specific area, so the drop-down menu of categories you’re about to be invited to pick from will already be relevant for the council departments (for example) that your report will be going to.

And note that we still haven’t asked you who you are. We do need to know — we send your name and contact details to the council as part of your report — but you didn’t come to FixMyStreet to tell us who you are, you came first and foremost to report the problem. So we focus on the reporting, and when that is all done then, finally, we can do the identity checks.

Of course there’s a lot more to it than this, and it’s not just civic sites like ours that use such techniques (most modern commerce sites have realised the value of making it very easy to take your order before any other processing; many governmental websites have not). But we wanted to show you that if you want to build sites that people use, you should be as clever as a magician, and the secret to that is often keeping it simple — deceptively simple — on the outside.

"Boundary Stone 1" taken by Tim Green (CC-BY-licensed from Flickr)

This is the first part of a two-part blog post about some of our work on making it easier to deploy FixMyStreet and MapIt in new countries.  This part describes how to generate KML for a given boundary in OpenStreetMap. Update: the second part is now available.

As mentioned in a previous post, we’re looking at ways of making it smoother to deploy FixMyStreet for a new country, city or use-case.  Essentially there are two fundamental bits of data that you need for this:

1. a mapping between a latitude and longitude (or postcode) to all the adminstrative areas that cover that point.
2. a mapping between problem type and adminstrative areas to an appropriate email address for reporting that problem.

The first of those is typically provided by a service called MapIt, an open source GeoDjango web application written by Matthew Somerville.  In the UK we are fortunate that official boundary data and the postcode database have now been released under an open data license from the Ordnance Survey.  However, in other many other countries similar data is unavailable, or not available under reasonable licensing conditions.  In such cases, though, all is not lost thanks to the extraordinary work of contributors to the OpenStreetMap project.  OpenStreetMap contains high-quality administrative boundary data for many countries of the world, and we know that data submitted to the project is available under the Creative Commons Attribution-ShareAlike license, so we can reuse it in a web service like MapIt.

The first step towards being able to build an instance of MapIt based on OpenStreetMap boundary data is to be able to generate a shapefile that represents a boundary in the project.  (In OpenStreetMap’s data model, boundaries are represented as either ways or relations, and those that we are interested in are tagged with boundary=administrative.)  Matthew had previously written code to generate KML files for boundaries in Norway in order to help to set up an instance of MapIt for Norway, which is used by FiksGataMi.  However, that script was quite specific to the organization of boundaries in that country, and it did not deal with more complex boundary topologies (e.g. enclaves), or different representations of boundaries (e.g. multiply nested relations).

So, Matthew and I wrote a new version of the code to extract a boundary from OpenStreetMap and generate a KML representation of it. The new version uses the Overpass API instead of XAPI, since it allows us to specify multiple predicates in the query and recursively fetch the ways and nodes that are contained in a relation.  Once all the ways that make up a relation have been fetched (ignoring those with roles like “defaults” or “subarea”), the script tries to join each unclosed way to any other with which it shares an endpoint.  We should end up with a series of closed polygons – the script exits in error if there are any unclosed ways left.  We can then directly create KML from these polygons, the only subtlety being that we need to mark certain boundaries as being an inner boundary (i.e., creating a hole in a boundary) if they had the role “enclave” or “inner” in an OpenStreetMap relation. For example, the South Cambridgeshire District Council boundary has a Cambridge City Council-shaped hole in it:

Similarly, the script has to cope with multiple distinct polygons, such as the boundary of Orkney.

If you want to use this code to generate a KML representation of a closed way or boundary relation from OSM, just clone the MapIt repository and run bin/boundaries.py:

$ bin/boundaries.py --help
Usage: boundaries.py [options]

Options:
  -h, --help            show this help message and exit
  --test                Run all doctests in this file
  --relation=<RELATION_ID>
                        Output KML for the OSM relation <RELATION_ID>
  --way=<WAY_ID>        Output KML for the OSM way <WAY_ID>

For example, to generate a KML boundary for the Hottingen area of Zürich, you can do:

$ bin/boundaries.py --relation=1701449 > hottingen.kml

In the next blog post in this series, we will discuss extracting such boundaries en masse and creating a service based on them.

DIY mySociety is all about making our code – and our experience – available to people who want to build similar websites in their own countries. We thought it would be helpful to list some examples of sites already using mySociety code, so you can see the variety of different possible outcomes.

It might seem like a simple task, but identifying sites in this way isn’t as straightforward as you might think – we don’t always know when people pick up our open source code! If we’ve missed any, please do comment below and we’ll add them.

There are also many sites around the world which were directly, or indirectly, “inspired by” ours. In these cases, the site’s owners have written their own code from scratch. That’s a subject – and a list – for another post. For now, here are all the international sites using mySociety’s code that we know about.

Alaveteli: our Right-to-Know Platform

WhatDoTheyKnow.com – our original Freedom of Information site
FYI.org.nz  – New Zealand Freedom of Information site
Pravodaznam – Bosnia and Herzegovina Freedom of Information site
Queremossaber.br – Brazil Freedom of Information site
Informatazyrtare.org – Albania Freedom of Information site
Tuderechoasaber.es – Spain Freedom of Information site
AskTheEU – Europe Freedom of Information site

Get the Alaveteli code here.

FixMyStreet: our fault-reporting Platform

FixMyStreet.com – our original fault-reporting site
Fiksgatami – Norway FixMyStreet
FixOurCity – Chennai FixMyStreet
FixMyStreet.br – Brazil FixMyStreet, based on both our code and FixMyStreet.ca from Canada

Get the FixMyStreet code here

Parliamentary monitoring and access to elected representatives

TheyWorkForYou – our original parliamentary monitoring site
WriteToThem – our original ‘contact your representative’ site
Mzalendo – Kenya parliamentary monitoring site
Open Australia – Australia parliamentary monitoring site
Kildare Street – Ireland parliamentary monitoring site
Parlamany – Egypt parliamentary monitoring site
Mejlis – Tunisia parliamentary monitoring site

A community of people, waiting to help

Inspired by the examples above? If you’re thinking of going ahead and building your own site, we’re here to support you with our easy-to-understand guidebooks and our friendly mailing lists. In our online communities you’ll find many of the people who built the sites listed here. There’s no-one better to ask questions, because they’ve been through the process themselves, from early conception right up to completion.

If you are one of those people who has been through the whole process of building, launching and running a site like these (with or without our codebase), and lived to tell the tale, please shout in the comments below. And especially if you’re open to people approaching with questions. Perhaps add a note to say where you prefer to have those conversations – whether that’s via your favourite mailing lists, Twitter, email or simply in the comments to this post.

One last thought – it’s interesting to see that our code can be used for areas as small as a single city (FixMyStreet Chennai) or as large as a confederation of states (AskTheEU.org). In short, it’s scalable! How will you use it?

Image by Windell Oskay, used with thanks under the Creative Commons licence.