We won’t insist on being addressed this way, but you can now append ‘BAFTA and Emmy nominated’ to our name. We were very chuffed to be nominated for two television awards in the last month: the BAFTA for Digital Creativity in Television Craft, and the Emmy for best Digital Non-fiction Programme.
‘TV?’, you might be thinking, ‘I thought mySociety were all about digital stuff.’ Well, increasingly, of course, the lines are blurred. Television programmes come bundled with their own website, Twitter hashtag, or app. These days, TV is less about being a passive viewer, more about becoming part of an active, engaged conversation online.
Last year, we worked with Channel 4 and TV production company Tiger Aspect to create the app and the website tools that accompanied their programme about empty houses – The Great British Property Scandal. A repurposing of the software that underlies FixMyStreet, the app enabled viewers to report empty homes; the site petition amassed 119k signatures – so the audience certainly got involved.
We were, of course, delighted to have been recognised, along with C4 and Tiger Aspect. In the end, we didn’t need the space we’d hastily cleared on the mySociety mantelpiece, but as the BAFTA went to the incomparable Paralympics, we really can’t begrudge it.
And of course, if you’re a TV company looking for help with your digital tie-ins, we’re happy to help.
FixMyStreet.com has always tried to make it as simple as possible to report a street problem. When we built FixMyStreet for Councils, we wanted to simplify things for local authority employees too.
So, as well as offering the option to integrate with council back-end systems, we also put together this nifty dashboard (right – click to see full-size). It’s one of several extra features councils get when they purchase the FixMyStreet for Councils package.
What do councils need?
- At-a-glance statistics, for all kinds of reporting. Perhaps the local newspaper have asked how many potholes have been fixed this year, or internal staff need a report on which types of problem are most rapidly fixed.
The top half of the dashboard allows for this sort of analysis. The drop-down category list means you can filter the view to show one category of problem – say, fly tipping – or all of them. Results are shown across a variety of timeframes.
FixMyStreet for Councils allows councils to designate their own progress statuses, beyond our standard ‘fixed’ and ‘open’. So, in this case, the statuses include ‘in progress’, ‘planned’, ‘investigating’, etc. Each of these is shown separately.
- A realistic picture of how long it takes to deal with issues. The ‘average time to council marking as fixed’ is a great measure of just how much time it is taking to get reports resolved.
Perhaps just as important, though, is the ‘average time to first council state change’ – that could just mean the report has been acknowledged, or that its status has changed to ‘under investigation’ – but these are still valuable mileposts for keeping residents informed of progress.
- Quick access to problems, as they’re reported. At the foot of the dashboard, there are links to all problems reported within the council boundaries.
There’s an option to filter them by any of the statuses, as above.
- Access for multiple people, in different locations. The dashboard is web-based, so it can be accessed by any employee with internet access – or several at once.
- But at the same time, complete security. It’s password-protected, so it’s only accessible to those who have been granted access.
- A responsive provider. mySociety believe that the launch of new software is only the beginning of the story.
When people start using new products, they often do so in surprising ways. They often ask for features that would never have occurred to us, and indeed might never have previously occurred to them.
We will remain in active development, of the dashboard, and of FixMyStreet for Councils as a whole. We’ll be soliciting feedback, and listening to it very carefully.
The FixMyStreet for Councils dashboard is only available to councils as part of our FixMyStreet for Councils package – find out more here.
Mapumental can turn vast datasets into visual tools that everyone understands. Faced with highly complex, yet crucial data from the Fire Protection Association, we had a chance to really put our technology through its paces.
Just how quickly could fire engines reach a given postcode in case of a fire? It’s a question that’s pivotal to decisions made by both the emergency services and the insurance industry.
But previously, it has been a challenge to present the data simply, because it involves so many variables.
Every region has its own factors, each of which will impact on fire engine response time. The number of vehicles at each station, the hours during which the station is manned, and the response policy of each individual fire authority will all play a part – and that’s before you even consider how geography might affect things.
Dr. Jim Glockling is Technical Director at the Fire Protection Association and Head of the Risk Insight, Strategy and Control Authority (RISCAuthority), an organisation for the advancement of risk management within the fire and security sectors. Jim approached mySociety with this question: how could we map this crucial, yet complicated data in a way that could be understood by RISCAuthority members at a glance?
It was clearly a job for Mapumental. Our transit-time mapping software was originally built to visualise public transport journey times, but its beauty is that ‘layers’ of data can be swapped out, allowing it to be used for all kinds of purposes.
Read more about mySociety’s data visualisation services here.
Assessing a property or postcode
And here’s the result of our pilot project. The maps on the right answer the following questions (click each image to see it at full size):
How quickly could 4 fire engines get to AL10 0XR ?
How does that change if the severity of the fire just requires one fire engine?
A user inputs a postcode, and can assess exactly how quickly a fire could be tackled in that area. The different levels of severity are measured by how many response vehicles are required, and changes in this number are immediately reflected on the map.
Assessing the general area
Which areas can four fire engines get to within 9 minutes 30 seconds at midday on a Saturday?
It’s also possible to assess the region’s overall response capability, without inputting a postcode. The user sets severity levels (number of fire engines, or High Volume Pumping or Aerial Appliance (ladder) is needed), the time and day of the week.
Where can an aerial appliance get to within 15 minutes at 2am on a weekday?
The FPA tool immediately highlights the areas that are accessible within the chosen parameters, drawing on the underlying data of journey times and information such as vehicle numbers and hours of operation for each individual fire station in the region.
Simplicity itself
With RISCAuthority, we tested the concept using data from one fire authority – Hertfordshire. mySociety’s task was to create a usable, elegant web interface that was as simple as possible to use, while still giving insurers the key data they needed.
The project called on everything we knew about clean design, usability and data structures. A key part of what makes Mapumental’s data visualisation so intuitive are its sliders: this enables the user to quickly explore variables on a map.
A tool with purpose
Dr Glockling explains: “Whilst not necessarily used as a component of insurance pricing, this information helps insurers administer risk control and fire protection advice to their customers in the context of what the Fire and Rescue Services will be able to achieve on their behalf.”
The response time is just one factor that insurance surveyors will take into account when they are assessing a building. “Where response and arrival times are not coherent with protecting the viability of the business in the event of fire, additional forms of in-built protection and control might be recommended, such as the installation of sprinkler systems.”
“In the longer term it is hoped such information will impact beneficially on the annual cost of fire in the UK.”
Results
The pilot tool was well received by the FPA community, and the plan is now to work with RISCAuthority to roll it out to more fire authorities shortly, and then nationwide.
Dr Glockling explains the pilot study helped them to understand two factors:
-
Would they get buy-in from both insurers and Fire and Rescue services on the viability and usefulness of the project?
-
Was it possible to present such a massive amount of data in a format that was readily palatable to the intended audience?
He says, “Mapumental’s team displayed an immediate understanding of our requirement. Delivery was to time and the result has perfectly satisfied the de-risking ambition. The working relationships were very good throughout and we intend now to extend the pilot to full UK rollout.”
During this phase, we will be inputting still more detail to the data, including information on the types of fire engine available to each region, and the plotting of fire stations on the map.
The tool will be a valuable resource for the FPA and the insurance industry, and we really look forward to the roll-out later this year.
Mapumental specialises in visualising complex geographic data sets on intuitive, easy to use map tools. If you have a data visualisation project that will benefit from Mapumental, just get in touch. Or read more about mySociety’s data visualisation services here.
Photo by William Murphy (CC)
Over the last 6 months or so, mySociety has been doing increasing amounts of work with local councils, not only helping them with problem reporting and online petitions, but also advising them on the impact of digital by default and how changing customer expectations are affecting digital service provision. To paraphrase Tom, for an ever-increasing number of customers, “local councils don’t have websites, local councils are websites”.
More specifically, we’ve been helping councils use user-centred techniques to kick-start the process of digital transformation: taking existing services that cause unnecessary frustration, figuring out how they should work for the customer in an ideal world, identifying the process changes needed, and helping make them happen.
How do you know where to start?
Most consultancies in this area will publicise their patented 5-step approach, or shower you in platitudes about talking to users and involving service managers, but I thought it would be more useful to walk through in detail what we actually do on a project like this. In this post, I’m going to describe only the first step (I’ll talk about others in future posts): given all the stuff that councils do, how do you know where to start?
Clearly, not every council service is susceptible to digital transformation. If you work in children’s services or benefits advice, your service is more likely to rely on cups of tea and conversation than on your website. But there are high volume transactions that involve exchanges of information or of money that do not, or rather *should not*, require any human intervention. Unfortunately, because of mistakes in how websites are structured and processes organised (that often go right back to decisions about management structure and procurement priorities), unnecessary demand is placed on contact centres.
What are your users trying to do?
So if you want to know what mistakes you’re making with your online presence, the first place you should look is the volume of calls to your contact centres and what questions the callers are asking. Here’s a complete list of all the places you can look for useful data on what your customers are actually trying to do and what you might be doing wrong:
- Contact centre logs: the records of what people who call you are actually asking about. This is the best place to look to identify the areas where your web presence is under-performing.
- Internal site search terms: the things people type in most often in the search box on your website. Generally speaking, use of search on a website is an indicator that your navigation and page structure have failed. Therefore the search terms people use on your site are another very interesting indicator of things you’re not doing well enough.
- Referring search terms: the most frequently used search terms that drive traffic to your website. What are people looking for and what words have they actually put in to Google (or indeed any other search engine) for to arrive at your website?
- Popular pages: data on the most frequently visited pages and sections of your website doesn’t tell you what you should improve or how, but it does give you a feel for where the demand is.
If you look at all of those things, you’ll have a lot of data to go through and make sense of. If you’re short on time, focus on the first one – it’s the juiciest source of insights.
Talking to service managers
Another approach we pursue in parallel to this one is to talk to a group of service managers and ask them for their opinions: if the decision on where we should focus our redesign efforts was up to them, what single thing should we start with that would make the biggest difference? How this actually happens in practice is that we get a group of people in a room together and ask them to write down (almost certainly on post-it notes) the top 3 – 5 services that they think are in need of a digital redesign. We then discuss and consolidate all of these before grouping them, trying to identify those that are the most susceptible to automation and where the complexity of the change needed internally is low enough to be approachable.
Decision time
The final part of figuring out where to start is to make a decision: which of these areas are you going to start redesigning first? You now have two sources of data on where to start: the results of your analysis of customer behaviour and the views of your employees who are closest to the action. Here we’ll make a recommendation, but leave the final decision to our council client: they know their organisation a lot better than we do.
With a focal point for the transformation efforts decided on, so begins the daunting-yet-exciting task of researching and designing the changes to be made: the bit where you actually talk to users, make prototypes or mockups of what the service’s digital touchpoints should look like (no specification documents here please) and then figure out together what process changes need to be made for it all to work in practice. Which, of course, are the topics for future blog posts.
I don’t think it is too controversial to make the following – rather boring – assertions: Greenpeace is part of the environmental movement. Oxfam is an international development charity. Human Rights Watch is part of the human rights movement. Obama for America is a political campaign. Facebook dominates the social networking sector. I hope none of these simple, descriptive statements has caused you to turn purple with semantic rage.
But what primary movement or sector is mySociety part of? Or Avaaz? Or Kiva? Or Wikileaks? When I ask myself these questions, no obvious words or names race quickly or clearly to mind. There is a gap – or at best quite a bit of fuzziness – where the labels should go.
This lack of good labels should surprise us because these groups definitely have aims and goals, normally explicit. Also, it is unusual because social and political movements tend to be quite good at developing names and sticking to them. If you were given a time machine you could tell a Victorian that you were ‘pro-democracy’ or ‘anti-slavery’ and the locals would have no trouble understanding you. Terms like ‘gender equality’, ‘small government’, ‘cancer research’, ‘anti-smoking’, even ‘anti-capitalist’, can comfortably be used by news media companies without fear of baffling the audience. The public can also easily understand terms that referred to methods of achieving change, rather than goals, terms like ‘political TV advertising’, ‘protests’, ‘petitions‘ and ‘telethons’.
But now let’s look at some of the common terms that are used to talk about the (very) wide field of digital social change projects. These include ‘digital transparency’, ‘hacktivism’, ‘peer production’, ‘edemocracy’, ‘clicktivism‘ and ‘open data’. But if you tried to slip one into a newspaper headline, the terms would definitely fall beneath the sub-editor’s axe before they could make it to print. They are too niche, and too likely to confuse readers.
The first thing to note about most of these terms is the way that they refer to methods, rather than goals of social change. But this isn’t completely unprecedented, and isn’t a reason to dismiss these terms out of hand. The name ‘Chartists‘ does indeed refer to people who used the publication of a charter as a political tool, but the name signified a huge bundle of values, methods and goals which went way beyond the deployment of that document.
Nevertheless, to me it still just doesn’t feel like the broad, loosely coupled fields of human endeavour which stretch from Anonymous to JustGiving have decent labels yet – especially not labels that signify the ways in which two things can be both similar and different (e.g. ‘rail station’ and ‘bus station’). And this worries me because consistent names help causes to persist over time. If the field of AIDS research had been renamed every 6 months, could it have lasted as it did? Flighty, narrowly used language confuses supporters, prevents focus and is generally the enemy of long term success.
So, why does this dearth of decent sector labels exist, and can we do anything about it? The short version is, I don’t know. But I do know that the easy answer, ‘It’s all too new to have names’ cannot be right any more, not now that millions have signed petitions, joined Avaaz, donated to Obama online and so on.
I don’t know why the category terms in these sectors are so weak and changeable, but I am posting today because I would love to hear the thoughts of other people who might have some ideas as to the causes, and possible solutions. Here are some theories about the lack of good labels, off the top of my head:
-
I think some of the terms currently in circulation were coined in anticipation of the development of possible projects, not after retrospectively reviewing them. So the category terms sometimes seem to define what a field might look like, rather than what it ends up looking like (think ‘edemocracy‘, from a decade ago). This means the terms often feel like they don’t describe real projects very well.
-
In the traditional (for-profit) internet industry a certain amount of money can be made from coining or becoming associated with new terms (think of IBM and ‘smarter cities’). Because there is a profit motive, there may be a structural incentive to rapidly create new terms which displace older ones which haven’t been widely adopted yet. There are probably similar incentives in some academic fields too – career rewards for coining a key term.
-
Terms in these fields we work in are usually minted one at a time – ‘only children’ as opposed to born as whole families of interconnected terms. This is unlike the sciences which, since Linnaeus came up with his elegant way of naming living things, have been good at developing naming systems, not just one-off names. Organic chemistry and inorganic chemistry are related, but different in important ways – the names helpfully show that. To explain how 38 Degrees and mySociety are similar in some ways but different in other very significant ways needs a way of naming things that can signal both commonality and difference.
-
The knowledge-sharing disconnect between the academic and activist/practitioner communities is really, truly terrible, everywhere except data-driven voter-targeting. People who run services or campaigns normally never hear about what the brightest academics are saying about their own work. And if they do try to pay attention to the ideas coming out of academia then the signal to noise ratio is too bad and the filters are too few and too busy having day jobs.
-
And, of course, I should namecheck the sceptic’s probable theory: this would argue that good, clear terms don’t exist because all these widely differing organisations are nothing more than meaningless feel-good bunk, so language slides off them like an egg off Teflon. I don’t subscribe to this theory, of course, but it’s worth noting because I’m sure some people would provide this answer to my question.
I am planning to write a follow-up blog post to this containing some suggested terms we might use to reflect what the many digital projects out there have in common, and how they are different.
But before I do, I would like to hear people’s thoughts on whether this is a real problem at all, and if so why that might be, and what we might do about it. Who knows, maybe someone will even write a blog post about it, like we’re back in 2003 or something…
National Health Service changes in England
Today (1st April 2013) marks a significant change in the way that the NHS in England is structured. Strategic Health Authorities (SHA) & Primary Care Trusts (PCT) are abolished, and their responsibilities are being taken on by newly created Clinical Commissioning Groups (CCG), the National Commissioning Board, Public Health England and local authorities.
The split is roughly along these lines:
- Clinical Commissioning Groups commission elective hospital care, urgent and emergency care, community healthcare and mental healthcare & learning disability services for the local areas they cover
- The National Commissioning Board covers primary care contracting (GP Contracting, Dental, Pharmacy), specialised services, offender healthcare, secure mental health care and some armed forces healthcare
- “Top-tier” and unitary Local Authorities take on responsibilities for these aspects of public health: sexual health services, drug and alcohol treatment, health checks, school nursing programmes, giving up smoking programmes and services to prevent childhood obesity
- Public Health England is a national body which will work closely with local authorities’ public health teams, carrying out a range of activities to protect and improve the nation’s health, eg to co-ordinating work to combat infectious diseases such as flu or infections acquired in hospitals such as MRSA, or to carry out national publicity campaigns to prevent ill health
This means quite a bit of change to the public authority listings on WhatDoTheyKnow:
1) PCTs and SHAs are now marked as “defunct” to prevent new requests from being made (see below for more details).
2) We’ve now listed all the new CCGs, but we’re missing email addresses for around 15% of them. It’s clear that many CCGs are not quite ready to welcome FOI requests. Even though they went live today, there are a fair number of websites still under construction (I’ve seen lots of “lorem ipsum” text today), with no contact details. We aim to get these all up-to-date in the next few weeks as they get up to speed.
3) The National Commissioning Board and Public Health England have been added to the site
4) We’ll be adding local Health and Wellbeing Boards, Healthwatch organisations & Local Education & Training Boards soon.
Police Service changes in Scotland
Under the banner of reducing duplication and cost-saving (BBC article), police services in Scotland are being completely re-organised with 2 new central bodies replacing all the regional police forces and boards:
Fire Service changes in Scotland
Similar changes are taking place with Scotland’s fire services:
Other joiners & leavers…
The following is a round-up of other changes taking place today…
Say hello to:
And goodbye to:
And although they’re officially changing, it’s pretty much business as usual for:
Defunct public authorities
We flag old public bodies that no longer exist as “defunct” to prevent new requests from being made. In most circumstances FOI officers transfer across in-flight requests to the relevant replacement authority. If you need to follow-up a request to a defunct public body (e.g. if there’s no further contact from an authority), the website will let you, however the “old” authority is no longer under any obligation to reply. You may need to re-send your request to a new public authority which will restart the 20-day clock…
Please help us!
Given the scale of change, if you find any incorrect information for these public authority listings, please let us know! Also please get in touch if you find an email address for any of those we’re still on the hunt for…
The Open Democracy Advice Centre (ODAC) in South Africa will be using mySociety’s Alaveteli software in their latest project – and, with a bit of match-making from mySociety, the preparation period has been rigorous.
Alaveteli is our open source Freedom of Information platform. It underlies our own UK-based WhatDoTheyKnow, and right-to-know sites around the world. Alaveteli sites make it easy for citizens to ask questions of those bodies who operate under Freedom of Information law and, significantly, they automatically publish all responses.
Before any coding or implementation began, we got ODAC together with the “Governance Collaboratory”, an initiative from the d.school in Stanford University that seeks to apply the “design thinking” approach to projects that intend to make government more open, more effective, and more accountable. We’ve observed quite a few Alaveteli installs, but while we’re always on hand to offer support and answer development queries, we’ve never prepared the ground quite like this.
Gabriella Razzano of ODAC welcomed Jeremy Weinstein and Jenny Stefanotti, both from the d.school, to Cape Town for an intensive few days of assessing how the design thinking approach could shape the project. Two staff from mySociety also went along — Paul (our Head of International Projects) and Dave (one of our developers) — because we’re keen to understand how the d.school’s approach might improve the way we go about building our new projects.
Now, at mySociety we already know a thing or two about building civic systems that engage with the public, because we have considerable experience in the field. We are expert at combining user experience and current tech to create simple, usable interfaces (see our DIY blog for some example details). We conduct usability tests, we apply A/B testing, and we think hard about what our analytics tell us. But actually much of this is reactive, iterative design: it’s being applied after the core product has already been built.
Design thinking challenges this approach by suggesting that the user on which initial designs are often based is purely imaginary. As a result, the site inevitably includes the assumptions and prejudices of its creators. This won’t necessarily lead to a bad design — especially if the creators are benign and experienced — but it must fail, by definition, to account for the unexpected things that may motivate or concern actual users. The design thinking process attempts to change this by approaching the initial problem in a prescribed way and following a process that isolates genuine, existing requirements. This includes, in design thinking terms, processing the initial interviews into empathy maps from which requirements emerge, and which themselves become features that are rapidly prototyped in isolation from other parts of the system.
This is uncomfortable for those of us used to building loose iterations from the bottom up and refining them later. It means introducing empathy and rapid, offline prototyping much earlier in the process than we’d normally expect. Certainly in the commercial world it’s common for a company to prototype against their target consumers early on. But for civic projects such as mySociety’s, it’s often much harder to identify who the users will be, for the impressive yet overwhelming reason that often we are building our platforms for everybody. This can lead to generalisations which may miss specific issues that could make a huge difference to some users.
The d.school advocates a “learning by doing” way of teaching, so the days we spent in Cape Town were a busy mix of practice as well as theory. We interviewed people who had a variety of reasons to want to make Freedom of Information requests, including an activist who’s already used South Africa’s Freedom of Information legislation to make requests regarding housing projects, the head of a rape crisis centre, and law students who may well become a nation’s most empowered activists. From these interviews we isolated specific needs, which at this stage were nearly all unconnected to any digital or web requirement. Jeremy and Jenny then led us through the process of rapid, analogue prototyping intended to address those needs.
Inevitably we could only scratch the surface in the few days we had available, but we hope ODAC will be able to apply the process to the development of their project, just as we aim to use it to benefit the work on ours.
Image credit: Procavia capensis (Rock-dwelling Hyrax or Dassie) by Arthur Chapman, released under CC BY-NC-SA on Flickr.
They tell visitors that dassies such as these live atop Table Mountain. We went up there and saw none. Similarly, Freedom of Information requests exist in South Africa under the Promotion of Access to Information Act 2000 (PAIA), but most people have never seen one — fewer than 200 PAIA requests were made nationally in 2012. This tenuous comparison allows us to illustrate the blog post with a cute picture of fuzzy mammals.
This is the third of our recent series of Open311 blog posts: we started by explaining why we think Open311 is a good idea, and then we described in a non-techie way how Open311 works. In this post we’ll introduce our proposed extension to Open311, and show how we use it in FixMyStreet.
The crux of our suggested improvement is this: normal people want to know what has happened to their problem, and Open311 currently isn’t good enough at telling them whether or not it has been dealt with. To be more specific, our additions are all about reports’ status change, by which we mean something like this:
That pothole?
I just totally fixed it.
That’s robot-311 from the previous post, if you’ve dropped in here without reading the previous posts. Once again we’re blurring the distinction between client and user (the girl you’ll see below) a little, to make things simpler to follow.
Create→send→fix→update
Every month in the UK, thousands of problems are reported on www.fixmystreet.com and, moments later, sent on to the councils who will fix them. Here’s what happens with a problem report for something like a pothole or a flickering streetlight:
- You create the report on FixMyStreet.
- FixMyStreet sends that report to the right department at the right council.
- That body puts it into its own back-end system.
- Later, when the council fixes the problem, FixMyStreet is updated, and everyone knows it’s fixed.
On the face of it, you might think we need only care about 1 and 2. But really, FixMyStreet isn’t just about dispatching reports, it’s about helping to get things like potholes actually fixed. And neither citizens nor local governments benefit if work gets done but nobody finds out about it – which is part 4 on the list above.
What do we mean by “status change”?
The example at the top of the page shows the robot effectively changing a problem’s status to “fixed”.
Actually, statuses can be simple, such as either OPEN or CLOSED, or more detailed, such as “under investigation”, “crew has been dispatched”, “fixed”, and so on. But since we’re only concerned here with the status changing, that specific vocabulary deployed doesn’t really matter – it can be anything.
In situations where FixMyStreet is not integrated with council systems (i.e we just send email problem reports) FixMyStreet problems still frequently get marked as fixed, because anyone can change the status of a report just by visiting the page and clicking the button. Obviously, though, we prefer to have FixMyStreet directly connected to the local government back-end databases, so that news of a fixed report can be automatically bubbled from the back-office up into FixMyStreet and out onto the net.
And here’s where the problem lies: Open311 doesn’t quite support this business of getting problem updates from the back office out to the public. So first, we’ll show you how it can be done today, using Open311, and we’ll explain why this isn’t a good option. Then we’ll show our preferred solution, which we’ve proposed as an extension.
Looking at everything just to spot one change (bad)
One way to notice if any problems’ statuses have changed is to use Open311 to ask for every single service request, and see if any of them have a different status since the last time you checked.
Tell me all the service requests you’ve ever received
OK:
request 981276 the pothole on the corner by Carpenter Street is now CLOSED (I filled in the pothole)
request 988765 the pothole by bus stop on Nigut Road is now CLOSED (I filled in the pothole)
request 998610 gaping hole at the end of Sarlacc Road is now OPEN (the pothole fell through)
request 765533 where the street was cracked outside Taffey’s Snake Pit is now CLOSED (I filled in the pothole)
. . .
continues for thousands of requests
Um, OK. Now I’ll look at all these and see if any have changed since I last asked *sigh*
Obviously there are some problems with this. Even though Open311 lets you ask for quite specific service requests, you have to ask for all of them, because by definition you don’t know which ones might have changed. Remember, too, that problems can potentially change status more than once, so just because it’s been marked as CLOSED once doesn’t mean it won’t become OPEN again later. This exchange is very wasteful, very slow and ultimately (with enough reports) may become de facto impossible.
Asking for just the changes (good)
So here’s a better way of doing it. We’ve actually been doing this for some months, and now seems the time to share.
The client asks the server for just the updates on a regular basis, so any requests that have recently changed get updated on FixMyStreet automatically, usually just a few minutes later.
Have you changed the status of any of service requests today?
Yes, request 981276 was CLOSED at 3 o’clock (I filled in the pothole)
Or, more practically for keeping FixMyStreet up to date:
Have you changed the status of any of service requests in the last 15 minutes?
This is handled by our extension to Open311, GET Service Request Updates. There’s also an optional equivalent call for putting updates into the server (POST Service Request Update), which would apply if the client changed the status after the service request had been submitted.
Note that the server identifies the problem with its own reference (that is, 981276 is the council’s reference, not a FixMyStreet ID, for example). This is important because not all these requests necessarily came from this particular client. Remember that all service requests are available through the Open311 GET Service Requests call anyway (as shown above). So the server doesn’t send each service request back in its entirety: just its ID, the new status, when it changed, and a brief description.
In practice the client wouldn’t usually ask for “today”. In fact, we typically send a request asking for any updates in the last 15 minutes, and then at the end of the day ask for the whole day’s updates, just to check none were missed.
The technical bit
From a client’s point of view, this is simply an extra call like others in the Open311 API. So it’s just a request over HTTP(S) for XML (or JSON, if required).
We deliberately make the client poll the server for updates and pull them in, rather than expecting the server to push updates out. This frees the server from any obligation to track which clients (for there may be more than just one) care about which updates. The requests themselves are sent with unique IDs, allocated by the server, so the client can dismiss duplicates. It’s also robust in the event of connection failures, so if there are timeouts or retry logic, that’s for the clients to worry about, not the server. Basically, this is all to make it as light on the server as possible: the only real issue is that it must be able to provide a list of updates. This usually means adding a trigger to the database, so that when a problem’s status is updated a record of that update is automatically created. It’s the table of those “service request update” records that incoming requests are really querying.
We have published our own recommendation of this mechanism, which FixMyStreet already implements, alongside the FixMyStreet codebase.
Is that it?
Yup, that’s it.
This extension is in addition to the Open311 specification — it doesn’t break existing implementations in any way. Obviously this means FixMyStreet’s Open311 implementation is compatible with existing Open311 servers. But we hope that others working on Open311 systems will consider our extension so that clients are kept better informed of the status of the problems being fixed.
Why are statuses so important that it is worth extending the Open311 spec?
mySociety didn’t originally build FixMyStreet because we wanted to get potholes fixed. We built it because we wanted nervous, politically inexperienced people to know what it felt like to ask the government to do something, and to be successful at that. We wanted to give people the buzz of feeling like they have a bit of power in this world, even if the most tiny amount.
If the government fixes a problem and the citizen doesn’t find out it’s a double loss. The citizen becomes disillusioned and weakened, and the government doesn’t get the credit it is due. Everyone loses. We think that Open311 is a key mechanism for making large numbers of people feel that the government does respond to their needs. It just needs a bit of an upgrade to do it better. We hope very much that the wider community tests and endorses our extensions, and it can be folded in to the next official version of the Open311 standard.
In the previous blog post we explained why we think Open311 is a good idea. In this post we’ll explain what it actually does.
Open311 is very simple, but because it’s fundamentally a technical thing it’s usually explained from a technical point of view. So this post describes what Open311 does without the nerdy language (but with some nerdy references for good measure). At the end there’s a round-up of the terms so you can see how it fits in with the actual specification.
We’re using an unusual example here — a blue cat stuck up a tree — to show how applicable Open311 is to a wide range of problems. Or, to put it another way, this is not just about potholes.
So… someone has a problem they want to report (for this discussion, she’s using a service like FixMyStreet).
There’s one place where that report needs to be sent (in the UK, that’s your council). That administrative body (the council) almost certainly has a database full of problems which only their staff can access.
I have a problem :–(
the “client”
I fix problems!
the “server”
In this example, FixMyStreet is an Open311 client and the council is an Open311 server. The server is available over HTTP(S), so the client can access it, and the server itself connects to the council’s database. In reality it’s a little bit more complicated than that (for now we’ll ignore clients that implement only part of Open311, multiple servers, and decent security around these connections), but that is the gist of it.
Although it’s not technically correct to confuse the client with the user, or the server with the council, it makes things a lot easier to see it this way, so we’ll use those terms throughout.
Service discovery
To start things off, the client can ask the server: what services do you provide?
Until the client has asked the server what problems it can fix, it can’t sensibly request any of them.
What services do you offer?
I can:
POT: fix potholes
TELE: clean public teleports
PET: get pets down from trees
JET: renew jetpack licenses …
FixMyStreet can use the response it gets from such a service discovery to offer different categories to people reporting problems. We actually put them into the drop-down menu that appears on the report-a-problem page.
In the Open311 API, this is handled by GET Service List. Each service has its own service_code which the client must use when requesting it. Note that these services and their codes are decided by the server; they are not defined by the Open311 specification. This means that service discovery can easily fit around whatever services the council already offers. The list of services can (and does) vary widely from one council to the next.
Service definitions
Some services require specific information when they are requested. For example, it might be important to know how deep a pothole is, but it’s not relevant for a streetlight repair.
Tell me more about the PET service!
I can get pets down from trees, but when you request the service, you *must* tell me what kind of animal the pet is, OK?
In the Open311 API, this is handled by the GET Service Definition method. It’s not necessary for a simple Open311 implementation. In fact, it only makes sense if the service discovery explicitly told the client to ask about the extra details, which the server does by adding metadata="true" to its response for a given service.
Requesting a service
This is where it gets useful. The client can request a service: this really means they can report a problem to the server for the body to deal with. Some submissions can be automatically rejected:
My hoverboots are broken :–( I need BOOT service!
404: Bzzzt error! I don’t fix hoverboots (use service discovery to see what I *do* fix)
Hey! Blueblue is up a tree! I need PET service (for cats)!
400: error! You forgot to tell me where it is.
If the report is in good order, it will be accepted into the system. Open311 insists that every problem has a location. In practice this is usually the exact position, coordinates on planet Earth, of the pin that the reporter placed on the map in the client application (in this case FixMyStreet.com).
I need PET service (for cats)! Blueblue is stuck up the biggest tree in the park :–(
200: OK, got it… the unique ID for your request is now 981276
In the Open311 API, this is handled by POST Service Request. You need an API key to do this, which simply means the server needs to know which client this is. Sometimes it makes sense for the server to have additional security such as IP address restriction, and login criteria that’s handled by the machines (not the user).
Listing known requests
The server doesn’t keep its reports secret: if asked, it will list them out. The client can ask for a specific report (using the ID that the server gave when the report was submitted, for example) or for a range of dates.
Did anyone ask you for help yesterday?
Yes, I got two requests:
request 981299: TELE dirty teleport at the cantina (I’m waiting for a new brush)
request 971723: POT pothole at the junction of Kirk and Solo (I filled it in)
In the Open311 API this is handled by GET Service Request(s). The client can indicate which requests should be listed by specifying the required service request id, service code, start date, end date or status.
Does Open311 work?
Oh yes. On the Open311 website, you can see the growing list of places, organisations, and suppliers who are using it.
The technical bit
In a nutshell: Open311 responds to HTTP requests with XML data (and JSON, if it’s wanted). There’s no messing around with SOAP and failures are reported as the HTTP status code with details provided in the content body.
You can see the specification for Open311 (GeoReport v2). It doesn’t feature blue cats, but if you look at the XML examples you’ll be able to recognise the same interaction described here. And remember the specification is an open standard, which means anyone can (and, we think, should) implement it when connecting a client and server in order to request civic services.
Coming next…
In the next blog post we’ll look at how FixMyStreet uses Open311 to integrate with local council systems, and explain why we’re proposing, and utilising, some additions to the Open311 specification.
Illustrated especially for us by René Carbonell.
Helô!
Alaveteli (the software that runs WhatDoTheyKnow) is capable of being translated into any language, and we’ve finally switched on the ability to use the website in Welsh today. Many apologies for the long wait as this has been on our to-do list for well over 2 years…
As you can see, we don’t yet have a complete Welsh translation, and it’s just a start: we’ve done the help pages, and around 6% of the rest. To take a look at what’s been done, just click the “Cymraeg” link at the top of any page.
We’d love it if you could help us get to 100% by adding translations (or correcting any mistakes we’ve made!) at Transifex. You can read more about working with translations for Alaveteli, here and here, or just get in touch if you need a helping hand getting started or have any further questions.
And finally, a massive thank you & diolch to the translators who have already helped us get this far!
