If you visit FixMyStreet and suddenly start seeing spots, don’t rush to your optician: it’s just another feature to help you, and the council, when you make a report.

In our last two blog posts we announced Buckinghamshire and Bath & NE Somerset councils’ adoption of FixMyStreet Pro, and looked at how this integrated with existing council software. It’s the latter which has brought on this sudden rash.

At the moment, you’ll only see such dots in areas where the council has adopted FixMyStreet Pro, and gone for the ‘asset locations’ option: take a look at the Bath & NE Somerset installation to see them in action.

What is an asset?

mySociety developer Struan explains all.

Councils refer to ‘assets’; in layman’s language these are things like roads, streetlights, grit bins, dog poo bins and trees. These assets are normally stored in an asset management system that tracks problems, and once hooked up, FixMyStreet Pro can deposit users’ reports directly into that system.

Most asset management systems will have an entry for each asset and probably some location data for them too. This means that we can plot them on a map, and we can also include details about the asset.

When you make a report, for example a broken streetlight, you’ll be able to quickly and easily specify that precise light on the map, making things a faster for you. And there’s no need for the average citizen to ever know this, but we can then include the council’s internal ID for the streetlight in the report, which then also speeds things up for the council.

Map layers

So, how do we get these assets on to the map? Here’s the technical part:

The council will either have a map server with a set of asset layers on it that we can use, or they’ll provide us with files containing details of the assets and we’ll host them on our own map server.

The map server then lets you ask for all the streetlights in an area and sends back some XML with a location for each streetlight and any associated data, such as the lamppost number. Each collection of objects is called a layer, mostly because that’s how mapping software uses them. It has a layer for the map and then adds any other features on top of this in layers.

Will these dots clutter up the map for users who are trying to make a report about something else?

Not at all.

With a bit of configuration in FixMyStreet, we associate report categories with asset layers so we only show the assets on the map when the relevant category is selected.

We can also snap problem reports to any nearby asset which is handy for things like street lights as it doesn’t make sense to send a report about a broken street light with no associated light.

Watch this space

And what’s coming up?

We’re working to add data from roadworks.org, so that when a user clicks on a road we’ll be able to tell them if roadworks are happening in the near future, which might have a bearing on whether they want to report the problem — for example there’s no point in reporting a pothole if the whole road is due to be resurfaced the next week.

Then we’ll also be looking at roads overseen by TfL. The issue with these is that while they are always within a council area, the council doesn’t have the responsibility of maintaining them, so we want to change where the report is going rather than just adding in more data. There’s also the added complication of things like “what if the issue is being reported on a council-maintained bridge that goes over a TFL road”.

There’s always something to keep the FixMyStreet developers busy… we’ll make sure we keep you updated as these new innovations are added.

—

From a council and interested in knowing more? Visit the FixMyStreet Pro website

FixMyStreet sends users’ street reports to councils across the UK.

But if you’re one of the staff that receives these reports, you might sometimes wish for more insights: which issues are most commonly reported in your area? What’s a bigger problem, dog fouling or potholes? How quickly do reports get fixed, and how does this compare with other councils’ performance?

To make it easy to discover the answers to all these questions, we’ve just rolled out a new stats dashboard on FixMyStreet — and it’s free to access if you work for a council.

Council staff can now view and download information that answers questions such as:

• How many FixMyStreet reports have been made in your area across various time periods?
• How many reports have been made in each category?
• What are the average times between reports being made and being marked as fixed, and how does this compare to other councils?
• Which categories of report are most common within your area?
• Which categories of report does FixMyStreet send to your council, and which email addresses does it use?

From this exclusive area, you can gain a more in-depth understanding of how FixMyStreet is being used in your area, while also getting a taste of the fuller functionality available with FixMyStreet Pro.

So, if you work for a council, head over to the dashboard page now, and start exploring.

This year’s Impacts of Civic Technology conference, TICTeC, is shaping up to be one of the best yet. Tickets have almost sold out, so if you’re hoping to attend, don’t delay.

Running on April 18-19 in Lisbon, TICTEC will, as usual, provide an unparalleled opportunity to meet the people building and using Civic Technologies that improve lives, solve problems and address social ills. The schedule is now on the TICTeC website, where you can also get acquainted with this year’s speakers.

Keynotes Martha Lane Fox and Prof Jonathan Fox will set the tone for a full programme, with speakers and delegates including representatives from Google, Facebook, and scores of cutting edge practitioners from many countries.

This will be your chance to hear from recently-elected French MP Paula Forteza; and Civic Tech thinkers from MIT, NYU and UCL. More international angles are added by representatives from Buenos Aires City Government, Rome’s ‘Roma Capitale’ initiative, and several speakers from Nigeria whose attendance has been made possible thanks to a grant from the MacArthur foundation.

During two days of diverse presentations and workshops, attendees will examine what works — and what doesn’t — in the fields of digital democracy, accountability, anti-corruption and transparency tech. There’s just one rule for those making a presentation at TICTeC: it’s not enough to present a new digital initiative; you must also bring the research that enquires into its efficacy.

A few tickets are still available, but hurry — we’re nearly sold out.

Don’t worry too much if you can’t attend in person. Every session will be filmed, with videos shared online after the event. Keep an eye on the mySociety blog or YouTube channel to be the first to know when they’re available  — or sign up for the newsletter. To track the conference in real time, follow the hashtag #TICTeC.

—
Image: Alejandro (CC by/2.0)

Does publishing a correspondence with MPs make it more likely that promises will be upheld, and citizens’ voices heard? Thanks to a piece of software we’ve just installed on a partners’ site, we may be about to find out.

As you may know, mySociety supports several partners’ projects worldwide: one of these is People’s Assembly, which, like our own TheyWorkForYou, makes it easier for citizens to find out who their representatives are and what they’re doing in Parliament.

PMG, who run the site, saw the potential of the Open Source WriteInPublic software, which was made by our friends in Chile Fundación Ciudadano Inteligente. Like mySociety’s own UK tool WriteToThem, WriteInPublic allows users to easily contact their representatives; where it differs is that the whole correspondence is published online. It’s a way of holding representatives to account, and making sure that promises or assertions are not forgotten.

Messages to MPs

Here in the UK, of course, MPs only deal with correspondence from their own constituents, but in South Africa, citizens may legitimately write to any MP. Messages are far more frequently about policy rather than personal issues, which might go some way to explaining why a WriteInPublic tool targeting MPs is a more viable prospect than it might be, say, in the UK.

PMG are yet to promote the tool through their newsletter and social media channels, but of course, users are discovering it for themselves on the homepage. In the five weeks since launch, more than 270 messages have been sent to MPs. These can be seen on the MPs’ pages, in a new ‘messages’ tab: here’s an example.

Informing Committees

The new tool doesn’t just invite users to write to their MPs directly; People’s Assembly now sports two invitations on its homepage: one to write to an MP, and another to contact a Committee.

PMG have previously had some success in surveying their users over key issues of party funding: the survey results were sent to a sitting Committee, and the chairman reported that they were “very helpful for the Committee’s discussions” and were “used as a reference point to gauge public opinion especially where discussions were deadlocked”.

The group are keen to extend this kind of engagement, and this second tool allows citizens to send a message to a Committee dealing with specific issues such as public works or the police. PMG are planning to continue surveying their users, while also pointing them at the tool as a way of getting public input into the bill-making process.

In the spirit of Democratic Commons, the underlying contact data for the MPs tool (though not the Committees one) is also now being used by Wikidata and our EveryPolitician project, so it’s freely available for anyone to use. For us it’s a win-win when data can not only serve an immediate purpose, but will also go on to provide a resource for anyone else who needs it.

—
Image: WOCinTech (CC by/2.0)

We’ve submitted evidence to the recent inquiry on whether Parliament should introduce a more formal system of voting by proxy. You can read our submission here, and see submissions from other organisations and individuals on Parliament’s website.

Voting by proxy is the practice of allowing someone else to cast your vote for you. In Parliament, when MPs go on extended leave, for example when they have a baby, there is no formal system in place; rather, arrangements are often made informally and, potentially, inconsistently.

A Member may approach a whip to request that they are paired with an MP from the opposition who will not be voting either, thus effectively cancelling out the votes that would have been cast. Apparently, there is also an informal tradition of allowing infirm or incapacitated (for example, because they are carrying a baby in their arms) representatives to vote from outside the chamber, but only when present within the precincts of the House. We were interested to see a remark in David Lammy MP’s own evidence:

I would also hope that the Committee might consider some way to end the practice notorious from the late 1970s of bringing seriously sick Members into Westminster in order to vote. This would carry severe reputational risk if repeated nowadays.

Why are we interested?

The inquiry is a direct result of the recent debate, on International Women’s Day, in which Harriet Harman led the call for a more formal system of voting by proxy for members on extended leave (and particularly on ‘baby leave’).

We agree that it’s important Parliament formalises this system, and we fully support any measure that will make life easier for parents, or those on extended leave for other life-changing reasons. And of course, we’re very much in favour of any initiative which will make parliamentary arrangements more transparent and accessible to the general public, which after all is the whole reason TheyWorkForYou exists.

But we also have a further interest in this subject. As you may recall, we were called out by MPs (and subsequently members of the public) for misrepresenting representatives on leave, since our site TheyWorkForYou was not displaying this information, leaving potential for members of the public to believe that such MPs were not attending to their duties.

In response to this, we are now able to manually add notes to the profile pages of those MPs who request it. However, as we outlined in our prior posts it’s not an ideal solution for a number of reasons, as summarised in our inquiry response.

We’re hoping that once the proxy voting system is formalised, the relevant information (that is, who is on extended leave, that a proxy is voting in their place, and the name of the proxy) will be released along with Parliament’s existing data outputs. You can read more about that in our response, but in short, this would allow us to display the information consistently and automatically, as we do for virtually all the rest of the information on TheyWorkForYou.

But it won’t only be useful for us. It’ll allow for the data to be displayed on Parliament’s own website, and of course will be of help to any website or tool which deals with Parliamentary activity and makes it easier for everyone to understand.

—
Image by Jessica Taylor (Parliamentary copyright, reproduced with the permission of Parliament). “Ayes to the right, noes to the left”. When there is a vote in the Commons, MPs leave their seats and walk into either the Aye or No lobby.

All mySociety websites have strong security: when you think about some of the data we’re entrusted with (people’s private correspondence with their MPs, through WriteToThem, is perhaps the most extreme example, but many of our websites also rely on us storing your email address and other personal information) then you’ll easily understand why robust privacy and security measures are built into all our systems from the very beginning.

We’ve recently upped these even more for FixMyStreet. Like everyone else, we’ve been checking our systems and policies ahead of the implementation of the new General Data Protection Regulation in May, and this helped us see a few areas where we could tighten things up.

Privacy

A common request from our users is that we remove their name from a report they made on FixMyStreet: either they didn’t realise that it would be published on the site, or they’ve changed their mind about it. Note that when you submit your report, there’s a box which you can uncheck if you would like your report to be anonymous:

FixMyStreet remembers your preference and applies it the next time you make a report.

In any case, now users can anonymise their own reports, either singly or all at once. When you’re logged in, just go to any of your reports and click ‘hide my name’. You’ll see both options:

Security

Security for users was already very good, but with the following improvements it can be considered excellent!

• All passwords are now checked against a list of the 577,000 most common choices, and any that appear in this list are not allowed.
• Passwords must now also be of a minimum length.
• If you change your password, you have to input the previous one in order to authorise the change. Those who haven’t previously used a password (since it is possible to make a report without creating an account), will receive a confirmation email to ensure the request has come from the email address given.
• FixMyStreet passwords are hashed with an algorithm called bcrypt, which has a built in ‘work factor’ that can be increased as computers get faster. We’ve bumped this up.
• Admins can now log a user out of all their sessions. This could be useful for example in the case of a user who has logged in via a public computer and is concerned that others may be able to access their account; or for staff admin who share devices.

—
Image: Timothy Muza (Unsplash)

 

 

 

 

When we started building WhatDoTheyKnow Pro, our toolkit for professional users of FOI, we knew that there was one feature which would be a game-changer for such users: the ability to send a request to multiple authorities at once.

In this blog post, we examine what we wanted the tool to do, how we are guarding against abuse, and finally we’ll give a step by step walkthrough of the interface.

Investigative news stories or in-depth pieces of research often require information from a multitude of different sources. By gathering statistics or information from multiple authorities, journalists, activists and researchers can build up a previously-unseen picture, for example of how widespread a particular problem is, or where there are inequities in medical provision across the country.

It’s something that many professional users of FOI are doing already, usually with the aid of their own homemade spreadsheets on which they keep track of requests made, dates by which replies should be expected, which bodies have responded, which need chasing, and of course the information held in the responses themselves.

The standard WhatDoTheyKnow website already provides several helpful features that you just don’t get with a DIY system: it has all the right email addresses for authorities, for example; it guides you through the FOI process; and it will send you an email reminder when the deadline for response arrives — even taking bank holidays into account.

But we knew that in order for our batch request feature to woo people away from their spreadsheets, it needed to do more than those homebuilt systems, some of which have been refined over several years and work well, even if a bit clunkily, for their owners.

Power and responsibility

One important consideration was uppermost in our minds when it came to batch requests: it costs authorities time and money to respond to each request, and of course that multiplies with batch requests. We are keen to promote responsible use of FOI, so we want to fold appropriate safeguards and guidance into whatever system we build.

As mentioned, with WhatDoTheyKnow Pro we’re focusing on features that are genuinely useful for professional users of FOI, but we also want to help those users make better, more focused requests — ones that are more likely to get useful responses and see the light of day as news stories. So it was important that, in making it simple to send multiple requests, we also help users find the most suitable authorities to send their requests to.

With that in mind, here are some balances we’ve put in place:

• Users are limited in how many batch requests they can send within any one month — so there’s no chance to go too wild.
• There’s a limit to the number of authorities that can be added to a single batch: we set this to be the number of local authorities in the UK, which is a logical sector to survey in this way.
• Before users do a batch mailout, we encourage the sending of an initial request that goes to just a few authorities. This safeguard can reveal where a request is flawed, so for example, if the data you get back is not what you need or in the wrong format, you don’t have to send to the full list all over again.
• We provide advice on cost limits to encourage succinct batch requests.
• Authorities have the facility to report a request which is unsuitable for review by our administrators.
• We’re rolling out the batch request functionality gradually to vetted WhatDoTheyKnow Pro users so that we can gradually learn how people use it in practice, and course-correct as necessary..

Testing and improvements

So far, the batch feature is only available to a select group of test users, who are giving us feedback on how they’re finding it. There’s certainly nothing like having your code being used by real people to help you see where improvements might be made!

That said, it’s been a very gratifying process. With the help of our test users, we’ve seen that the batch request functionality has the potential to be immensely helpful to professional users of FOI; even genuinely game-changing. We are certain that with the sending tools, we’ve created a service that really adds value for this sector.

We’re now in the next phase, and turning our attention to improving the functionality that helps users deal with incoming responses when they come in. This already exists in a basic form, and thanks to our testers, we’ve identified which improvements we need to make. We’re already working on incorporating them. But that is definitely material for the next update  — for now, let’s take a look at just how the batch request function works.

Using batch request

There are three parts to making any request, whether you’re doing it yourself or using WhatDoTheyKnow Pro:

1. Creating the request
2. Managing the responses
3. Analysing the results

The batch request functionality builds on our super-simple FOI workflow tools for WhatDoTheyKnow Pro, extending them to make larger investigations much easier.

Creating the Request

The first step is compiling a list of authorities to send the request to. From the compose screen, you might search on a keyword (for example, ‘hospital’, ‘Birmingham’, or ‘Birmingham hospitals’) and then add the authorities you’re interested in.

Each authority is added to a recipient list and WhatDoTheyKnow Pro creates a ‘mail merge’ setup. You’ll see how many authorities you’re writing to in the compose interface.

You can then draft your request. The special `Dear [Authority name]` salutation gets automatically replaced with each of the selected authorities when you send your batch.

Finally, before sending you can choose a privacy duration.

At this point you can either go straight ahead and send your request, or save the draft and come back to it later.

Managing Responses

Once you’ve sent your batch request, you’re going to receive a lot of replies from authorities. This is where WhatDoTheyKnow Pro’s functionality really comes into its own, keeping all that clutter out of your email inbox.

Here’s what it looks like: the first thing you’ll see is a high-level progress bar showing you the overall progress of your batch. There are three main states that help you manage the requests in the batch:

• In progress (yellow): This means that there’s no action needed by you – you’re waiting on the authority to respond with an acknowledgement or the information you’ve requested.
• Action needed (red): When a request in the batch receives a response from the authority, you’ll need to check it out. We mark the response as “action needed” for you to review and decide what to do next.
• Complete (green): Once there’s no further action needed – either you’ve got the information you asked for, the authority didn’t have the information, or they’ve refused and you don’t want to challenge them – the request moves to the ‘complete’ state, so you know you don’t need to think about it until you come to analyse the data.

Clicking the title of the batch reveals the individual requests and their progress status. From there, you can click through, read the response and update the status.

Analysing Results

Now you’ve got all your data, it’s time to compare the results from different authorities.

Sometimes authorities will reply in the main correspondence.

Other authorities respond with one or more attachments. You can view these inline or download them to your computer.

If you’re dealing with a batch sent to lots of authorities, sometimes it’s easier to just download everything. You can download a Zip file containing all the correspondence and attachments for each request via the “Actions” menu. From there you can pull out the attachments that contain the raw data and plug the numbers or answers in to your spreadsheet so that you can compare across authorities.

You can sign up to WhatDoTheyKnow Pro today and receive 1 month free with the voucher code BLOGMARCH18. Make some requests to try out the FOI workflow tools for professionals, and get in touch to request to join the waiting list for batch access.

If your FOI requests have made the news, let us know! Send us links to your published stories and we’ll throw in an extra month of WhatDoTheyKnow Pro for free. Your stories help us improve WhatDoTheyKnow Pro.

—

Image: Peretz Partensky (CC by-sa/2.0)

To help us mark WhatDoTheyKnow’s tenth anniversary, we asked volunteers, supporters and users to tell us which Freedom of Information requests from the site’s first ten years particularly stuck in their minds.

The result was an eclectic mix of stories that really show the breadth of how WhatDoTheyKnow has been used. They have very little in common — unless you count the imagination and tenacity of those using FOI to try to uncover significant information.

Doug Paulley, WhatDoTheyKnow volunteer

A exposé that helped bring in the living wage for carers

Doug is one of the team of volunteers who give up their time to keep WhatDoTheyKnow running, using their experience and knowledge of FOI to moderate the site, give users guidance, and help set policy. Doug is also an extremely active user of FOI, having used the act to uncover many examples of discrimination and malpractice over the years.

He highlights the story of a care home talking the talk, but very much failing to walk the walk when it came to paying its staff the living wage.

“The exposure brought about by FOI played a significant part in the campaign for Leonard Cheshire, care home operator with 2,100 residents, to significantly increase carers’ wages to (just short of) the voluntary living wage. Journalist Heather Mills covered the story in Private Eye.” Read the whole story here.

Owen Blacker, mySociety trustee

Missing historic information on Cold War targets

Owen co-founded FaxYourMP, the earliest version of mySociety’s  WriteToThem, and has been an important part of the organisation ever since — he’s now one of our trustees and a non-executive director. He recalls the building and launch of WhatDoTheyKnow and indeed was one of its earliest registered users.

Owen particularly remembers a pass-the-parcel like series of FOI requests in which he was handed from one organisation to another:

“I went round in circles trying to find out some Cold War information that nobody claims to know any more. In 1980, the entire Civil Service, nationwide, ran a dry run of a Cold War nuclear attack on the United Kingdom, called Operation Square Leg. I’m slightly concerned that we spent a lot of money planning the civil contingencies of a Cold War attack — a sensible things to do, arguably — but no longer know where we were expecting to be hit or at what megatonnage.” Owen links to the requests from this blog post.

Will Perrin, Indigo Trust

Safer streets and better data handling

Will is not only a trustee at Indigo, supporting mySociety’s work with parliamentary monitoring organisations in sub-Saharan Africa, but he’s also a trustee of London’s King’s Cross Community Projects. Indeed Kings Cross — a locality in which Will has a personal stake, with a long record of community action — is the subject of two of the three FOI requests he singled out:

First was the Kings Cross Walkability audit which revealed just how hostile to pedestrians the area was back in 2008. At the time, Will wrote in his blog: “Crossing the road in Kings Cross is a nightmare and now we have an official report commissioned by TfL that sets it out in black and white.”

Today he recalls its impact: “This document underpinned the police taking a corporate manslaughter case against TfL to the Criminal Prosecution Service with regard to a cyclist’s death in 2011. The case did not proceed but was instrumental in changing TfL’s attitude to cyclists’ rights.

“Then this request revealed a massive overspend by Network Rail in refurbishing its own offices at Kings Cross”.

Finally, Will’s third choice of request had wider implications for the country as a whole:

“The National Police Chiefs’ Council revealing that there was no governance system in place for the Automatic Numberplate Recognition System (ANPR) and the existence of Met’s ‘Olympic Data Feed‘ led to a new governance system being instilled; some 2 billion records were deleted along with the introduction of a vastly reduced retention period.” Annotations at the foot of this request give a little more background.

Matthew Somerville, mySociety developer

A long-standing pillar of mySociety’s development team, Matthew wrote the core code behind many of mySociety’s most notable websites and tools, including FixMyStreet and TheyWorkForYou. He spends his working days coding for mySociety’s useful tools, and much of his free time coding his own useful tools, if his website is anything to go by. What was his most memorable FOI request?

“It was a request asking Royal Mail for information about all their postboxes, made by Tom Taylor.  I had to write a crowd-sourcing tool to locate them, as the information provided included street name but no actual location; they then (from another FOI request a few years later) released the co-ordinates as well.”

The data is mapped here. Why is this request significant?

“I’m not sure it’s really significant, but I do get plenty of people telling me they’ve used the site, and it’s something Royal Mail never got around to providing (even though that was their reasoning for refusing to release it)…”

—

So there we are: a handful of the 458,219 requests that WhatDoTheyKnow has processed to date. There are so many stories around FOI requests: each of them represents someone’s burning question; many of them result in a response that’s important, or fascinating, or historic. And that’s what makes WhatDoTheyKnow so rewarding to work on.

 

We’ve talked a lot about our new service for journalists and other professional users of Freedom of Information — but it’s not always the professionals who uncover the news stories.

This week, we mark WhatDoTheyKnow’s tenth anniversary. As part of the celebrations, we thought we’d look back on the news stories that came about because of requests made through the site. Many of these began with an FOI request submitted by a user with no links to the press, and were picked up by news outlets because the response was of public interest.

From the restrictions on what names can be given to a baby in this country, to an accidental torpedo release, and via a geographically-accurate Tube map, it makes for fascinating reading. You can see them all here.

On 22 February 2008, we posted an announcement on this very blog: “the new mySociety Freedom of Information site is now live”.

More than 450,000 requests later, WhatDoTheyKnow is marking its tenth anniversary: as part of the celebrations, we’ve put together a timeline showing how WhatDoTheyKnow has intersected with the history of FOI in the UK since we first gained our right to information in 2005. If nothing else, you may enjoy looking at the site’s rather more primitive design back in its early days.

The past decade has seen legal challenges, contributions to Parliamentary inquiries, and the development of our code for use in other countries (26 and counting). It has proved that an ambitious project can be kept going thanks to the efforts of unpaid but skilled and dedicated volunteers.

Most of all, though, it’s seen you, the general public, submitting requests for information, and sharing the responses you receive. That was always the idea, and, it turns out, it’s a pretty sound one.