Transparent screen, by Kathy McLeod, used under the Creative Commons licence, with thanks.

At mySociety we like transparency – it’s baked into most of our projects.

TheyWorkForYou attempts to make it easier to find out what your MP has been doing in Parliament. WhatDoTheyKnow tries to make it easier to find out what’s going on inside other public bodies. FixMyStreet and the upcoming FixMyTransport also use transparency to help get problems resolved.

We think transparency is a good thing for many reasons, but one of its rarely mentioned virtues is how valuable transparency can be for the people within the organisations which are transparent.

Transparency can be useful because it means people outside an organisation can make critical, constructive suggestions about how you can improve, and it lowers the odds that people in one part of your own organisation will be ignorant of the activities of people in other parts.

Research

To that end we commissioned Tobias Escher of the Oxford Internet Institute to conduct detailed analysis into two of our major websites – TheyWorkForYou and WriteToThem.

We were not highly prescriptive in our instructions, and we certainly didn’t ask Tobias to ‘discover’ pre-determined findings. All we did was ask Tobias to find out who was coming to the sites, what they were doing, and whether or not the sites could be considered to be succeeding.  We didn’t do it for a PR stunt:  we did it so we could learn from our mistakes, and so that we could share those learnings with others who might benefit.

His detailed, quantitative analysis holds the sites up to mySociety’s own stated aims, for the first time. And we’ve published both documents, in full, below.

Swings and Roundabouts

It was great to discover that we have, indeed, attained some of our goals by running these sites. For example, one of the reasons we set up TheyWorkForYou and WriteToThem was to make representatives accessible to people who were newcomers to the democratic process. It was therefore heartening to read that 60% of visitors to TheyWorkForYou had never previously looked up who represents them, and two in five users of WriteToThem have never before contacted one of their political representatives.

But, as you would expect with any properly neutral evaluation, it’s not all good news. Our sites aim to reach a wide range of people, but compared to the average  British internet user, WriteToThem users are twice as likely to have a higher degree and a higher income. It also seems that users are disproportionately male, white, and over 35. These figures and many more are available within these highly readable papers – Tobias did a terrific job in gathering and analysing a huge amount of data, and then making it easy to understand.

Stories

These reports are rich with data, from how visitor numbers boomed during the MPs’ expenses scandal to which MPs most people sign up to receive alerts about. You can also read how a budget airline almost brought a site to its knees in 2007; what part Joanna Lumley plays in our history; and how many visits to TheyWorkForYou actually come from within Parliament itself.

TheyWorkForYou and WriteToThem have inspired many people around the world to set up similar (and not so similar) sites inspired by the vision of using the Internet to lower barriers to democracy. However, until now we’ve never seen a really clear-eyed assessment of what seems to work, and what doesn’t.

If you’re at all interested in using the Internet to engage people with democratic systems, Tobias Escher’s excellent research papers will make a compelling read. Thank you Toby!

The reports

Download Tobias Escher’s research report on WriteToThem.com

or TheyWorkForYou research report by Tobias Escher

…and do come back and tell us what you found interesting.

PS

We hope to publish two evaluation reports like this at the start of each new year from now on. Next year’s sites will probably be FixMyStreet and WhatDoTheyKnow. Do get in touch if you’d like to input!

At mySociety we love our site FixMyStreet – it’s the epitome of a web tool that gives simple tangible benefits whilst generating a little accountability at the same time. Reports through the site were up 40% last year, so it’s clear that users quite like it too.

FixMyStreet has been copied in many different countries, which makes everyone in mySociety very happy, too, even apparently appearing in a slide deck the White House uses to show innovative services. However, it turns out that the cheerfully minimalist, almost wantonly unfashionable user-interface has an unfortunate down side: most people who copy the site look at it, think “That looks easy!” and then cheerfully start coding their own clone.

Deceptive simplicity

Alas – the very simplicity that makes the site good hides the fact that making a site like FixMyStreet really work well is actually way harder than it looks. What will you do when a government email inbox fills up? What about when administrative boundaries change, due to an election or restructuring? How do you know you’re not scaring users away with careless wording? All the hard-won lessons from these questions have been baked into the FixMyStreet codebase, and we’re only too keen to talk to people about them.

We were therefore particularly pleased when the Nowegian Unix User’s Group (NUUG) came to us to ask if we could help improve FixMyStreet to make it easier for them to install. Over the last month mySociety Senior Developer Matthew Somerville has been working hard with Petter Reinholdtsen and Christer Gundersen of NUUG, and here’s what they’ve managed in just a handful of weeks.

• The launch of a Norwegian FixMyStreet called Fiksgatami, covering nearly every corner of Norway’s 300,000 square kilometers.
• Problems reported anywhere within Norway will be correctly directed to any of the 400+ responsible municipalities, thanks to Petter and Christer’s amazing data sourcing skills.
• As a necessary side-effect of developing this, Norway now has a free, public administrative web service gazeteer – http://mapit.nuug.no. If Norway is anything like the UK this will soon become an indispensable service for many other web sites and mobile tools.
• The standard mapping is now OpenStreetMap, pulled together by the brilliant Norwegian OpenStreetMap community*. We couldn’t take a technological step backward, and so whilst the site uses OpenLayers if you have JavaScript, the map continues to work just fine without as well.
• The open source FixMyStreet codebase has been upgraded to make it easier to translate into other languages, easier to use different mapping with, and easier to install. These efforts will continue, as we realise this has been one reason why others have made their own versions.
• All this has been done without forking, so various major upgrades we have planned for the UK version will be exportable later in the year.

NUUG’s Fiksgatami is the epitome of what makes civic open source at its best so unmatchably good. It was developed incredibly quickly: just a month to create what is effectively a fully fledged, best-of-breed nationwide e-government service – albeit an unofficial one. Thanks to the hard work of the public servants who fix problem reports, it will make small but meaningful improvements to the lives of a lot of people in Norway.  And it has made the free FixMyStreet codebase better and easier for other people to use to help them do the same thing in other countries.

I know that at mySociety we are all looking forward to working with NUUG again. And I hope that this story inspires others to look at our code, and to work with mySociety to make FixMyStreet a service that can help everyone who would benefit from it.

* We’ll be rolling out updated mapping (including OSM) and more in the UK, soon.

I recently found these requests by James Muldoon covering FOI statistics for the London Boroughs for 2009.  As we regularly carry out analysis of WhatDoTheyKnow’s percentage share of FOI requests to central Government Departments of State, I thought it would make for an interesting comparison to do the same for the 33 Metropolitan borough councils, plus the City of London.

Below is a graph of the market share for WDTK.

Overall, the share for 2009 was 8.1%.  During the year, the share did fluctuate quite a bit, and the requests on WhatDoTheyKnow were significantly lower in the 2nd quarter for some reason.

Q1: Jan-Mar 2009 – 9.4%
Q2: Apr-Jun 2009 – 5.1%
Q3: Jul-Sep 2009 – 9.5%
Q4: Oct-Dec 2009 – 8.3%

The City of Westminster has a much higher number of FOI requests compared to the other boroughs, mostly apparently due to a large motorcyclist parking campaign/protest.  73% of all requests made to Westminster via WhatDoTheyKnow in 2009 contained the words “parking”, “motorcycle” or “Verrus” (203 out of 278).

I will soon start looking for FOI statistics for Local Authorities outside London, either on WDTK, or via their disclosure logs.  The Ministry of Justice encourages Local Authorities to regularly publish statistics on their FOI data.

Data caveats

• Brent – excluded from totals & comparison as the underlying FOI request is still outstanding.  The ICO is apparently investigating.
• Camden –  Q1-2009 data excluded from totals & comparison due to partial refusal to the FOI request by Camden (FOI Act Section 12, costs of complying too high)
• Southwark – excluded from totals & comparison.  They said in their FOI response:  “due to a serious malfunction of our reporting database we have no access to the data stored centrally”.  The data has been re-requested by James to see if the malfunction has been fixed.

The Ministry of Justice recently released the latest statistics on freedom of information implementation in central government for the first quarter of 2010.  We can use this data to roughly calculate the share of FOI requests made via mySocety’s Freedom of Information website WhatDoTheyKnow.com.

According to the Ministry of Justice 6,857 requests were made to “Departments of State” between January and March 2010, of which 707 were made via WhatDoTheyKnow – a 10% share.  Defra and DCSF had the highest percentage of WhatDoTheyKnow requests in the first quarter, at 23% and 24% respectively.  We have made the departmental breakdown of the statistics for the past two years available as a Google Spreadsheet.

Precise calculations of the percentage share are not possible, due to the way each department defines a valid request.  For example, some FOI requests to the UK Border Agency (included in Home Office data) or the FCO made via WhatDoTheyKnow would not have been counted as FOI requests, but instead as “routine requests for information”, falling outside the scope of data collection for the statistics produced by the Ministry of Justice.  In addition, requests for data readily available (i.e. exempt under Section 21 of the Freedom of Information Act) are not counted either by the MoJ, but will be included in the WhatDoTheyKnow statistics.  Overall, these form a minority of requests made by WhatDoTheyKnow, so the percentage share we quote is fairly accurate.

WDTK share of central departments’ FOI requests

The graph shows that the total numbers and percentage of requests have been fairly consistent over the past year, around 10-13%.  Share of requests increased significantly during 2008 as the site grew in popularity following its launch.

The large increase in total FOI requests between Q4 2008 and Q1 2009 may partly be due to the increased popularity of WhatDoTheyKnow and also increased visibility of Freedom of Information to the general public through more mentions of FOI law in the media.  Key stories in the press included Jack Straw’s veto against disclosure of the Iraq War Cabinet Minutes and Parliament’s aborted attempt to hide details of MPs’ Expenses. Requests made via WhatDoTheyKnow accounted for around 20% of the total increase in FOI requests.

It’s worth bearing in mind that the above graph only covers the 22 central Departments.  WhatDoTheyKnow.com lists over 3,750 authorities including local government councils, police forces, NHS Trusts, non-ministerial departments and many more, and also covers Scotland’s public authorities. There is no national monitoring of how many FOI requests are made to such bodies, or how well they perform when responding to requests.

In total, 6,565 requests were made via WDTK in the first quarter, 87% of which were to non-central Departments.  It’s unlikely that the site will ever get full coverage as the majority of FOI requests originate from companies carrying out market research, journalists, political parties and charities, most of whom prefer to make requests in private in order to maintain exclusivity over the released data.  WhatDoTheyKnow would like to include these companies in future, possibly by offering a managed FOI service on a fee basis, including an embargo period prior to the released information being made fully available on the internet – more information is available from the WhatDoTheyKnow team.

Articles based on previous statistics releases:

• Fraction of FOI Requests Made via WhatDoTheyKnow.com Increasing Fast – October 2009
• What Percentage of FOI Requests are made via WhatDoTheyKnow – July 2009

The Hansard Society have just published a report entitled MPs Online: Connecting with Constituents. I’m only going to talk about one part, the part that mentions the mySociety project WriteToThem in a section on MPs’ use of email.

We’re surprised and disappointed to see our methodology for collecting data on how well MPs respond to constituency mail being called “unreliable”, especially from a paper that makes a number of simple mistakes of its own in just a few lines on one page.

• On page 5, they state that WriteToThem has been “tracking responsiveness to emails via their website for three years”. Most importantly for the theme of the report, we don’t just send emails – we send faxes to a number of MPs (and other representatives) who do not accept or want messages via email.
• The figures given for survey responses in the table are backwards; 2007 and 2005’s figures should be interchanged – how could we get more survey responses than messages (again, not necessarily emails) sent? 🙂
• They claim there is “no quantification of the response categories provided” – the raw data used to automatically generate these categories is given in the adjacent column (“very high” simply means a response percentage of 80% or more, for example; our code is all open source).
• We exclude MPs with very small sample sizes, and take a range of steps to make sure the data is not abused.
• We have four years of statistics now, not three; our stats for 2008 were published nearly six months ago.

The Hansard Society, to the best of my knowledge, never got in touch with us to request any clarification or ask about our data or methodology, which we would have been more than happy to supply.

mySociety’s Freedom of Information website WhatDoTheyKnow is designed to appear simple and straightforward to users. That appearance belies the fact that behind the scenes a significant amount of effort goes into making sure both those making freedom of information requests and those answering them have a positive experience of the site. While the site is almost entirely automated sometimes human involvement is necessary. This article highlights those key “edge cases” which are dealt with by the staff and volunteers who make up the WhatDoTheyKnow team.

In the last year 15,233 freedom of information requests have been made via WhatDoTheyKnow.

Message Delivery
444 messages on 360 requests (2.3%) had to be manually placed on the correct request as a result of authorities not sending replies to the email address given. The errors are introduced as authorities apparently manually transcribe email addresses from incoming email into correspondence management systems. There have been suggestions some may even print out and scan-in emails into such systems. WhatDoTheyKnow’s code has been improved in light of experience, common errors are now detected automatically and in many cases the system suggests which request the message was intended to be directed to.

In terms of outgoing messages just 52 (0.3%) requests over the course of the year were marked as receiving an error message in response and users marked 94 (0.6%) as requiring administrator attention. These are generally either transient errors which simply require a message to be resent or prompt us to check and update the contact details we hold for a particular organisation. Regularly there are problems with authority’s spam filters and we have to encourage them to change the way their filters are set up to allow messages from WhatDoTheyKnow.com through.

Gone Postal
119 (0.8%) requests were at some point marked as “Handled by Post”. In many of these cases users eventually persuaded authorities to release the information in electronic form. Where information is supplied outside the site users can add annotations describing the information released, then can link to copies of the data they have posted online, or as has been done in respect of 14 requests (0.1% of the total, 11% of those handled by post) they can supply the information to WhatDoTheyKnow to upload manually. When the site was being designed there was a worry that authorities would reply to many requests by post. This has not occurred, in part perhaps because the freedom of information act contains a provision (section 11) requiring the requestor’s preferred means of communication to be used where it is reasonable. A requestor using an @whatdotheyknow email address is clearly expressing a preference for a reply to be made electronically via the site.

Libel
One of the major challenges facing the site is keeping it operating in the face of the UK’s libel laws. Unlike in other countries, such as the US, we cannot publish statements on our users’ behalf without taking the risk of being sued for libel ourselves. Even simply republishing FOI responses from public authorities is not without risk in the UK. While we don’t actively police the site a lot of administrator time is taken up dealing with cases where potentially libelous or defamatory comments have been brought to our attention. Cases can be very complicated and involve a great deal of correspondence. mySociety is lucky to have the services of a specialist internet and technology barrister with expertise in libel who provides his services free of charge. We try and act in such a way as to maximise transparency while ensuring that the existence of WhatDoTheyKnow and mySociety are not threatened by legal risks.

In the last year there have been only seven significant cases where requests have been hidden from public view on the site due to concerns relating to potential libel and defamation. Three of those cases have involved groups of twenty or so requests made by the same one or two users. While actual number of requests we have had to hide is around 70 (0.4% of the total) even this small fraction overstates the situation due to the repetition of the same potentially libellous accusations and comments in different requests. In all cases we have kept as much information up on the site as possible. Our policy with respect to all requests to remove information from the site is that we only take down information in exceptional circumstances; generally only when the law requires us to do so.

Personal Information
Sometimes people accidentally post personal information to the site; for example they make a request which is not a Freedom of Information request but a subject access request under the Data Protection Act. We are happy to remove such requests. On occasion we get requests from both our users and public sector employees asking us to remove their names from the site. As we are trying to build up a FOI archive we are very reluctant to remove information from the site, our policy is only to remove names in exceptional circumstances. Often information, such as an out of office reply, which a public body or civil servant considers irrelevant and asks to be removed is in fact critical to the correspondence thread and timeline of a response.

Copyright and Control of Information Released
The fact information is subject to copyright and restrictions on re-use does not exempt it from disclosure under the Freedom of Information Act (though there is a closely related exemption relating to “commercial interest”). Occasionally public bodies will offer to reply to a request, but in order to deter wider dissemination of the material they will refuse to reply via WhatDoTheyKnow.com. Southampton University have released information in protected PDF documents and the House of Commons has refused to release information via WhatDoTheyKnow.com which it has said it would be prepared to send to an individual directly.

Mantaining and Expanding The List of Authorities
WhatDoTheyKnow lists around 3,000 public authorities, there is a regular turnover of changes in contact details. Our coverage, while large, is not comprehensive so we have requests to add bodies such as parish councils, schools, and doctors surgeries which we have not yet attempted to add in a systemic manner based on official sources of information.

We have also had to carefully consider what we do when for handling the various situations where an authority becomes defunct and its responsibilities are taken over by another body for example as a result of reorganisations of local government and the creation and merging of government departments.

Providing Advice and Assistance
The team at WhatDoTheyKnow.com often provide advice to users. We encourage users to keep their requests focused so as to reduce the chance of any problems due to libel or requests being classed vexatious. On occasion we suggest appropriate authorities for users to direct requests to, provide advice to those unhappy with the response to their request, and answer a broad range of other queries as they arise such as if particular bodies are subject to the act or not. Increasingly we link to authority’s publication schemes which are intended to let people know what information an authority has and how it can be accessed.

Spam
Lastly, like all websites which allow people to post content online WhatDoTheyKnow.com occasionally suffers from spam in various forms. Most is dealt with automatically but some has to be removed by hand. With spam, like the other aspects of running the site, the site’s code and processes are constantly being developed and improved to reduce the fraction of cases requiring any manual intervention.

This article was prompted in part by a team in New Zealand considering launching their own version on the site asking us what’s involved.

Statistics were recently released on the performance of UK central government departments with respect to their handling of freedom of information requests. The latest figures are for the second quarter of 2009. We have been able to use these to calculate the fraction of all requests which are made via mySociety’s freedom of information website WhatDoTheyKnow.com.

• 13.1% of all FOI requests to “Departments of State” in the second quarter of 2009 were made via WhatDoTheyKnow.com. In absolute terms this was 753 out of 5769 requests; this is up from 8.5% in the first quarter of 2009.
• 32.3% of FOI requests to the Home Office (which includes the UKBA and the IPS) were made via WhatDoTheyKnow in the second quarter of 2009. In absolute terms this was 206 out of 638 requests.
• The latest figures show that in twelve of the UK’s twenty-one Departments of State more than 10% of FOI requests were made via WhatDoTheyKnow.

What these statistics mean is that an ever increasing fraction of the information released in response to freedom of information requests is being archived and made publicly available by WhatDoTheyKnow.com. Hopefully this will reduce the number of duplicate requests being submitted and ensure the information released is made available to the widest possible audience which in-turn should increase the chances it is acted on.

Only forty-three central government bodies have their freedom of information performance monitored centrally. This is a tiny fraction of the three thousand or so bodies currently listed by WhatDoTheyKnow.

Raw Data

• Freedom of Information Act: statistics on implementation in central government – UK Ministry of Justice
• Detailed statistics on WhatDoTheyKnow’s share of requests on a per department basis.

The volunteer team behind our Freedom of Information (FOI) site WhatDoTheyKnow.com, has used statistics released by the Ministry of Justice to discover the proportion of all FOI requests being made via the site.

They found that in the first quarter of 2009, 8.5% of all requests made to central government departments were made using WhatDoTheyKnow. In absolute terms that was 514 of 6019 requests.

The breakdown by department is given in the below table. Notably, one in five FOI requests to the Home Office (122 of 643) were made via WhatDoTheyKnow.

WDTK = WhatDoTheyKnow; Source for total FOI request statistics : Statistics for Q1 2009 (released on the 25th of June 2009); Extended table covering all monitored bodies available.

The Ministry of Justice only monitors, and provides statistics on, 44 bodies’ compliance with the Freedom of Information Act; WhatDoTheyKnow currently lists 2910. We cover a wide range of local bodies including Primary Care Trusts, Local Councils and the Police. There is no national monitoring of how many FOI requests are made to such bodies, or how well they perform when responding to requests.

If you want to see such performance statistics, please help categorise more of the responses made via the site. It can be quite addictive!

Thanks to Richard Taylor for doing this research – see his blog post for some more details, including some information about Scotland.

A few moments ago the team rolled out changes to our biggest and best known site, TheyWorkForYou.com meaning that every visitor to any page of the site will be greeted with a call to arms on the issue of some MPs voting this Thursday to conceal their expenses. And after the vote, we’ll be prominently publishing who voted which way – there should be a couple of million visitors at least before the next election.

Our explicit goal is to have a lot of constituents from around the country let their MPs know they won’t be impressed with a ‘yes’ vote or an abstention (the same thing in this case), and to build our Facebook group to the point where the mainstream media starts to take notice of this Net driven discontent.

Please do everything you can to get as many people as possible writing to their MPs and joining that Facebook group. We’re doing our bit – please do yours. Together we can stop the encouraging trend of more openness in our Parliament scrunching into reverse.

This post refers to the statistics we published for 2007. If you would like to see the latest responsiveness figures on WriteToThem, please visit www.writetothem.com/stats/.

—
New statistics, made by Francis, for your interest and elucidation. They’ll make their way onto TheyWorkForYou shortly.