Constituency boundaries are changing at the next general election in England, Wales, and Northern Ireland. After some amount of fiddling (I’ll go into technical details in another post, but it wasn’t as easy as just importing some shapefiles), as a slightly early Christmas present, TheyWorkForYou now has a section where you can enter your postcode to find out what constituency you are currently in, and what constituency you will be voting in at the election, along with maps of before and after:
http://www.theyworkforyou.com/boundaries/
This service is also available through the TheyWorkForYou API. This is a facility we have been asked for frequently, more so as we approach the forthcoming election; the large amount of boundary changes have led to confusion from our users and elsewhere, so this will hopefully prove useful.
One site that will need the boundaries before the election is DemocracyClub – join to help make this coming election the most transparent ever!
Side effects of the above process include updated council boundaries, so those councils on WriteToThem that we’ve had switched off since May due to lack of boundary data are now back; a more up-to-date postcode dataset; and the beginnings of parish council support (as in they’re now in the database, but the front-end doesn’t know what to do with them yet).
I hope you all have a happy Christmas and New Year.
Thursday 20th January, in Manchester, mySociety, Democracy
Club, and friends invite you to join us in the Briton’s
Protection from 7:30 for conversation on democracy,
engagement and tea.
mySociety runs projects such as TheyWorkForYou.com,
FixMyStreet.com and WhatDoTheyKnow.com, while DemocracyClub.co.uk is the new
independent volunteer network for the upcoming
election to keep track of how parties campaign in your local
community when they think no one elsewhere will notice.
Come join us in the pub (20th Jan, 7:30), or sign up online at
http://www.democracyClub.co.uk
The Briton’s Protection is just behind the Bridgewater Hall,
postcode M1 5LE. You’ll find us in one of the two back
rooms, wearing a mySociety hoodie or carrying a clear sign.
Tom looks like this: http://www.mysociety.org/about/
Wednesday 20th January, in Manchester; mySociety, Democracy Club, and friends invite you to join us in the Briton’s Protection from 7:30 for conversation on democracy, engagement and tea.
mySociety runs projects such as TheyWorkForYou.com,FixMyStreet.com and WhatDoTheyKnow.com, while DemocracyClub.co.uk is the new independent volunteer network for the upcoming election to keep track of how parties campaign in your local community when they think no one elsewhere will notice.
Come join us in the pub (20th Jan, 7:30), or you can sign up to Democracy Club at http://www.democracyClub.co.uk
The Briton’s Protection is just behind the Bridgewater Hall, postcode M1 5LE. You’ll find us in one of the two backrooms, wearing a mySociety hoodie or carrying a clear sign. Tom has his picture here: http://www.mysociety.org/about/
A Facebook event is now open.
Like mySociety? Like pubs? Why not come to our pre-Christmas pub meet, this coming monday?
The Banker
2 Cousin Lane, London, EC4R 3TE
21st December from 6.30pm onwards.
Leave a comment on this post if you’re coming. New faces and old hands equally welcome.
Over the last weekend of November 2009 a group of 21 mySociety staff, volunteers and trustees went to a house outside of Bristol to wrestle with the question of what mySociety should build over the next 12 months. This was the fourth time we’ve done it, and these meetings have become a crucial part of our planning. This year, we were talking not just about what new features to add to our current sites, but also about the possibility of building an entirely new website for the first time in a couple of years. The discussions were lively and passionate because we know we have a lot to live up to: not only is our last major new site (WhatDoTheyKnow) likely to cross the 1 million unique visitors threshold this year, but we understood that there were people and organisations who weren’t there who would be counting on us to set the bar high.
A chunk of the weekend involved vetting the 227 project ideas that were proposed via our Call for Proposals. I’m going to write a separate post on our thoughts about that process, but if you look at the list below you may spot things that were submitted in that call.
One nice innovation that helped us whittle down our ideas from unmanageable to manageable numbers was a pairwise comparison game to help us prioritise ideas, build custom for the occasion by the wonderful and statistically talented Mark Longair. In other words, we used the technique that powers KittenWar.com to help decide our key strategic priorities for the next year: after all , if we don’t, who will?
By the end of the weekend we had not battened everything down – there are too many uncertainties around how much time we will have, and some key ideas that need more speccing. However, we were able to put various things into different buckets, marked according to size and degree of certainty. So here goes:
1. Things which were decided at the last retreat, which we are definitely building, and which (mostly) need doing before next year’s stuff starts getting built
- A top level page for each bill on TheyWorkForYou
- Future business (ie the calendar) for events in the House of Commons, including a full set of alerting options.
- Video clips on MP pages on TheyWorkForYou
- Epicly ambitious election data gathering and quiz building with the lovely volunteers at DemocracyClub
2. Small new things that we are very probably doing because there was lots of consensus
- Publish a standard that councils can use to post problems like potholes in their databases to FixMyStreet and other similiar sites.
- Template requests in WhatDoTheyKnow so that users are strongly encouraged to put in requests that are well structured.
- After the next general election, email new MPs with various bits of info of interest to them including their new login to HearFromYourMP, their page on TheyWorkForYou, explanation of how WriteToThem protects them from spam and abuse, a double check that their contact details are correct, and a introduction to the fact that we record their correspondance responsiveness and voting records.
- Add to WhatDoTheyKnow descriptions about what kind of public authority a specific entity is (ie ‘school’, ‘council’) and the information they are likely to hold if FOIed.
- Show divisions (parliamentary votes) properly on debate pages on TheyWorkForYou, ie show the results of a vote on the same page as the debate where the issue was discussed, with full party breakdowns on each division.
- Add “How to benefit from this site” page on TheyWorkForYou, inspired by OpenCongress.org
- Help Google index TheyWorkForYou faster by creating a sitemap.xml file that is dynamically updated.
- Using the data we expect to have from DemocracyClub’s volunteers, send a press release about every new MP and to all relevent local newspapers
- Incorporate a council GeoRSS problem feed into FMS
3. Slighty more time consuming things we are very probably doing because there was lots of consensus
- 1 day per month developer time that customer support guru Debbie Kerr gets to allocate as she see fit.
- Premium account feature on WhatDoTheyKnow to hide requests so that journalists and bloggers can still get scoops and then share their correspondance later.
- Add Select Committees to TheyWorkForYou, including email alerts on calls for evidence.
- Take professional advice on how to handle PR around the election
4. Much more time consuming things and things around which there is less consensus. NB – We do not currently have the resources to do everything on this list next year – it is an ambitious target list.
- Primary New site: TBA in a new post
- Add a new queue feature to WhatDoTheyKnow so that users can write requests, then table them for comments from other users and expert volunteers before they are sent to the public authority
- Relaunch our Volunteer tasks page on our sites, keep it populated with new tasks, specifically allocate resources to handhold potential volunteers. Allocate time to see if any of the ideas that we didn’t build could be parcelled into volunteer tasks.
- Secondary New site (if we have a lot more time than we expect): Exploit extraordinary richness of Audit Comission local government target data in a TheyWorkForYou-like fashion.
- FixMyStreet to become international with a) maps for most of the world b) easy to follow instructions explaining how to supply mySociety with the required data to us to enable us to turn on FixMyStreet in non UK countries or areas. This data would includ ie gettext powered text translation files, shapefiles of administrative boundaries, and lists of contact data.
- Add votes and proceedings to TheyWorkForYou (where they reveal statutory instrument titles that are not debated but where the law gets changed anyway)
- Carry out usability testing on TheyWorkForYou with then help of volunteer Joe Lanman – then implement changes recommended during a development process taking up to 10 days.
- Add to TheyWorkForYou questions that have been tabled in the house of commons but which haven’t been answered yet.
- Add a new interface for just councils so that they can say if a problem on FixMyStreet has changed status.
Phew. And that’s not even counting the projects we hope to help with in Central and Eastern Europe, our substantial commercial work, or the primary new site idea, which will be blogged in Part 2.
Many MPs and Lords use the Freedom of Information Act to obtain information from public authorities despite the fact they are able to table parliamentary questions. Occasionally they make their requests via mySociety’s freedom of information website WhatDoTheyKnow.com which ensures both the request, and its response, are freely available online. Surprisingly the freedom of information route can result in the release of more, and better quality, information than a written Parliamentary Question.
For example on the 12th of November 2009 Eleanor Laing the Conservative Shadow Minister for Justice submitted the following written Parliamentary Question:
To ask the Secretary of State for Health how many staff in his Department were employed on the management of freedom of information requests submitted to his Department in each year since 2005; and how much his Department spent on the management of such requests in each such year.
The response contained the number of staff per year as requested but with respect to the spending the parliamentary response stated: “The information requested on expenditure could be provided only at disproportionate cost.”
A very similar request for information had been made many months previously, in July, by WhatDoTheyKnow.com user and FOI campaigner Heather Brooke. The response to the FOI request contained more information, and more precise information, than Eleanor Laing had obtained via her parliamentary question. When the request was made via WhatDoTheyKnow how much staff substantially involved in answering requests were paid was disclosed, in detail.
While the costs of complying with a particular request are capped by regulations under the Freedom of Information Act, data on total costs of FOI compliance such as that released by this request allows the average costs of dealing with a request to be calculated.
MPs using WhatDoTheyKnow
Do let us know in the comments if you’ve spotted any more!
Today we have a strange story about a department that appears to think that it has a duty not to release information under FOI if it makes people angry.
It all starts in January 2009 the Department for Children, Schools and Families (DCSF) appointed an expert by the name of Graham Badman to conduct a review of elective home education in England. It probably goes without saying that this is an issue far from our concerns, and an issue that mySociety has no views on – what makes us interested is the process that followed.
Shortly after the publication of the report, Elaine Walton, a user of mySociety’s freedom of information website WhatDoTheyKnow.com requested copies of communications between the Department for Children, Schools and Families and Nektus Ltd. the company through which it appears Mr Badman was paid for his work.
According to email replies to Ms Walton, the DCSF located two relevant invoices which show how much money was paid, but refused to disclose them. Strangely, though, they were not refused on grounds of commercial confidentiality, but rather on something more unusual. Here are the exemptions they cited:
- Section 40 – Personal Information
- Section 38 – Health and safety
Health and Safety? A little investigation reveals more.
When Ms Walton appealed against this decision, an internal review was carried out within the DSCF. The internal review’s findings stated that Mr Badman was likely become a victim of harassment if certain personal details were made public, hence a health and safety concern, and hence no publication of these invoices. Fair enough – nobody would be in favour of revealing private, sensitive information that would endanger anyone’s life or family, especially in the presence of a known threat. But take a look at this:
“That the Department had initially been drafting a response that included the release of invoices with only personal data redacted. But before the draft was complete it was apparent that there was a campaign of harassment and vilification against Graham Badman and other individuals/organisations that had contributed to the Report. In the light of this, at the weekly review meeting of FOI cases, it was considered that the balance of public interest might have shifted towards withholding.”
What is very curious here is the admission that the department had been thinking of releasing the invoices with personal data hidden (ie no home address, bank details etc). But then because of a campaign of harassment, it was decided that they wouldn’t publish anything at all. So not just no personal information, but no dates, no amounts of money, nothing.
What is so unease-making about this FOI decision is that it appears to be saying that departments may conceal information on how much public money has been spent on something because releasing that information will make some angry people even angrier. Surely this can’t be right – if it were every budget would be conducted in complete secrecy. We would encourage the Information Commissioner’s Office to take a look.
We know from our inboxes that there are people all over the world who would love to start sites like TheyWorkForYou.com, FixMyStreet.com, or WhatDoTheyKnow.com in their own countries. Building and running these sites is hard, though, and takes time, money, and love. Until now we haven’t been able to do much for these keen correspondents beyond sharing our ideas, sharing our code, and wishing people the very best of luck. We’re happy to say that for at least some of these people, things are about to change for the better.
If you live in Central or Eastern Europe, we’re now in a position to help you get effective democracy and transparency websites built. mySociety have teamed up with the Open Society Institute (OSI) and together we are now looking for determined people with great ideas for new digital transparency and accountability services in their countries.
Over the next few months we are running a Call for Proposals, similar to the one we recently ran in the UK. The big difference is that this time we’re not looking for projects that we will build. We’re looking for projects you want to build, but that for lack of funds or lack of the right skills, you can’t get started yourself.
Each month the Open Society Institute and mySociety will work closely together to select a series of projects to fund and mentor. Crucially, the call isn’t solely for existing NGOs: the process is absolutely open to submissions from individuals or groups with no prior direct experience of working in the transparency and accountability sector, but who have a good idea that addresses a problem they see in their country. We will, however, look more favourably on applicants with access to the advanced programming skills required to build sites like this.
The criteria are simple, though demanding:
- The projects have to generate some kind of meaningful transparency, accountability, or democratic empowerment of another kind.
- The projects must seize the unique benefits that the Internet brings with it, such as scalability, two way communication, easy data analysis and so on.
More details are available over at our new CEE site, but even if you don’t live in one of the eligible countries please help us spread the word about this exciting new opportunity!
mySociety’s Freedom of Information website WhatDoTheyKnow is designed to appear simple and straightforward to users. That appearance belies the fact that behind the scenes a significant amount of effort goes into making sure both those making freedom of information requests and those answering them have a positive experience of the site. While the site is almost entirely automated sometimes human involvement is necessary. This article highlights those key “edge cases” which are dealt with by the staff and volunteers who make up the WhatDoTheyKnow team.
In the last year 15,233 freedom of information requests have been made via WhatDoTheyKnow.
Message Delivery
444 messages on 360 requests (2.3%) had to be manually placed on the correct request as a result of authorities not sending replies to the email address given. The errors are introduced as authorities apparently manually transcribe email addresses from incoming email into correspondence management systems. There have been suggestions some may even print out and scan-in emails into such systems. WhatDoTheyKnow’s code has been improved in light of experience, common errors are now detected automatically and in many cases the system suggests which request the message was intended to be directed to.
In terms of outgoing messages just 52 (0.3%) requests over the course of the year were marked as receiving an error message in response and users marked 94 (0.6%) as requiring administrator attention. These are generally either transient errors which simply require a message to be resent or prompt us to check and update the contact details we hold for a particular organisation. Regularly there are problems with authority’s spam filters and we have to encourage them to change the way their filters are set up to allow messages from WhatDoTheyKnow.com through.
Gone Postal
119 (0.8%) requests were at some point marked as “Handled by Post”. In many of these cases users eventually persuaded authorities to release the information in electronic form. Where information is supplied outside the site users can add annotations describing the information released, then can link to copies of the data they have posted online, or as has been done in respect of 14 requests (0.1% of the total, 11% of those handled by post) they can supply the information to WhatDoTheyKnow to upload manually. When the site was being designed there was a worry that authorities would reply to many requests by post. This has not occurred, in part perhaps because the freedom of information act contains a provision (section 11) requiring the requestor’s preferred means of communication to be used where it is reasonable. A requestor using an @whatdotheyknow email address is clearly expressing a preference for a reply to be made electronically via the site.
Libel
One of the major challenges facing the site is keeping it operating in the face of the UK’s libel laws. Unlike in other countries, such as the US, we cannot publish statements on our users’ behalf without taking the risk of being sued for libel ourselves. Even simply republishing FOI responses from public authorities is not without risk in the UK. While we don’t actively police the site a lot of administrator time is taken up dealing with cases where potentially libelous or defamatory comments have been brought to our attention. Cases can be very complicated and involve a great deal of correspondence. mySociety is lucky to have the services of a specialist internet and technology barrister with expertise in libel who provides his services free of charge. We try and act in such a way as to maximise transparency while ensuring that the existence of WhatDoTheyKnow and mySociety are not threatened by legal risks.
In the last year there have been only seven significant cases where requests have been hidden from public view on the site due to concerns relating to potential libel and defamation. Three of those cases have involved groups of twenty or so requests made by the same one or two users. While actual number of requests we have had to hide is around 70 (0.4% of the total) even this small fraction overstates the situation due to the repetition of the same potentially libellous accusations and comments in different requests. In all cases we have kept as much information up on the site as possible. Our policy with respect to all requests to remove information from the site is that we only take down information in exceptional circumstances; generally only when the law requires us to do so.
Personal Information
Sometimes people accidentally post personal information to the site; for example they make a request which is not a Freedom of Information request but a subject access request under the Data Protection Act. We are happy to remove such requests. On occasion we get requests from both our users and public sector employees asking us to remove their names from the site. As we are trying to build up a FOI archive we are very reluctant to remove information from the site, our policy is only to remove names in exceptional circumstances. Often information, such as an out of office reply, which a public body or civil servant considers irrelevant and asks to be removed is in fact critical to the correspondence thread and timeline of a response.
Copyright and Control of Information Released
The fact information is subject to copyright and restrictions on re-use does not exempt it from disclosure under the Freedom of Information Act (though there is a closely related exemption relating to “commercial interest”). Occasionally public bodies will offer to reply to a request, but in order to deter wider dissemination of the material they will refuse to reply via WhatDoTheyKnow.com. Southampton University have released information in protected PDF documents and the House of Commons has refused to release information via WhatDoTheyKnow.com which it has said it would be prepared to send to an individual directly.
Mantaining and Expanding The List of Authorities
WhatDoTheyKnow lists around 3,000 public authorities, there is a regular turnover of changes in contact details. Our coverage, while large, is not comprehensive so we have requests to add bodies such as parish councils, schools, and doctors surgeries which we have not yet attempted to add in a systemic manner based on official sources of information.
We have also had to carefully consider what we do when for handling the various situations where an authority becomes defunct and its responsibilities are taken over by another body for example as a result of reorganisations of local government and the creation and merging of government departments.
Providing Advice and Assistance
The team at WhatDoTheyKnow.com often provide advice to users. We encourage users to keep their requests focused so as to reduce the chance of any problems due to libel or requests being classed vexatious. On occasion we suggest appropriate authorities for users to direct requests to, provide advice to those unhappy with the response to their request, and answer a broad range of other queries as they arise such as if particular bodies are subject to the act or not. Increasingly we link to authority’s publication schemes which are intended to let people know what information an authority has and how it can be accessed.
Spam
Lastly, like all websites which allow people to post content online WhatDoTheyKnow.com occasionally suffers from spam in various forms. Most is dealt with automatically but some has to be removed by hand. With spam, like the other aspects of running the site, the site’s code and processes are constantly being developed and improved to reduce the fraction of cases requiring any manual intervention.
This article was prompted in part by a team in New Zealand considering launching their own version on the site asking us what’s involved.
Come to the pub with us!
12th October 2009, from about 6.30, at the Banker just under Cannon Street station.
We won’t bite. Probably.
Statistics were recently released on the performance of UK central government departments with respect to their handling of freedom of information requests. The latest figures are for the second quarter of 2009. We have been able to use these to calculate the fraction of all requests which are made via mySociety’s freedom of information website WhatDoTheyKnow.com.
- 13.1% of all FOI requests to “Departments of State” in the second quarter of 2009 were made via WhatDoTheyKnow.com. In absolute terms this was 753 out of 5769 requests; this is up from 8.5% in the first quarter of 2009.
- 32.3% of FOI requests to the Home Office (which includes the UKBA and the IPS) were made via WhatDoTheyKnow in the second quarter of 2009. In absolute terms this was 206 out of 638 requests.
- The latest figures show that in twelve of the UK’s twenty-one Departments of State more than 10% of FOI requests were made via WhatDoTheyKnow.
What these statistics mean is that an ever increasing fraction of the information released in response to freedom of information requests is being archived and made publicly available by WhatDoTheyKnow.com. Hopefully this will reduce the number of duplicate requests being submitted and ensure the information released is made available to the widest possible audience which in-turn should increase the chances it is acted on.
Only forty-three central government bodies have their freedom of information performance monitored centrally. This is a tiny fraction of the three thousand or so bodies currently listed by WhatDoTheyKnow.
Raw Data
