Over 115,000 Freedom of Information requests.

Almost 225,000 FixMyStreet reports.

Close to 3,000 public transport problems.

Every word spoken in Parliament since 1935.

So, what would you like to know?

There’s no doubt about it, mySociety sites store a lot of data. And once you have that much data, you can start finding the answers to interesting questions. Questions like:

• Which public bodies receive the most FOI requests?
• Which county gets the most pothole reports?
• Which train routes are people complaining most about?
• Which MP has spoken for the longest cumulative time in the history of Parliament?

There are less obvious questions, too – how about:

• Which regions of the country are most likely to include bad language when submitting a form online?
• How many times does the Speaker have to interject, “Order, order!” in an average week?
• Which words are most spoken in Parliament, and which have only become popular in the last five years?
• What topics do people submit the most Freedom of Information requests about?
• Just how often does a UK citizen get so fed up about dog poop that they take action?

We reckon there are almost limitless stories in our data, waiting to be teased out. Some of them will be surprising, fascinating, or just plain funny. Some may even be potential front page news. So, we’ve invited journalists who have a particular interest in data, or indeed in any of the areas we work in, to come and have at it at our first ever mySociety Data Hackday.

Not a journalist?

Journalists aren’t the only ones with bright ideas, so if you’re reading this and there’s a burning question that springs to mind, leave a comment below. Given all these reams of data, what would you be looking for? We’ll add the best ideas to our list, and we’ll be reporting back on everything we find out.

Actually, I am a journalist!

There are still a few places, so if you’d like to attend, please drop us a line. Note: we will expect you to get stuck in! We will run the data, but you may be sifting through the results, looking for significant stories, and sharing your findings. Bring a laptop, and plenty of ideas.

If you can’t attend, but really wish you could, let us know what data you’d like us to run, and we’ll add it to the list.

ETA: Lanyrd page here.

Image credit: Johan Nilsson

FixMyTransport was launched a month ago. It is now well on its way to listing 1,000 individual complaints, suggestions and requests to the public transport operators of Great Britain.

As the sample size grows, we’re able to see just what provokes the country’s mild-mannered passengers into action. There are the diurnal irritations – the leaky station roof, constant announcements, smelly trains; there is the discomfort of overcrowding and overheated buses.

All of which are important, of course. And in this, FixMyTransport is achieving its aim of allowing people to make their reports to the operators, while at the same time creating small bunches of people who read those reports and think, ‘Hey, me too!’.

But FixMyTransport is not just for the little gripes. It’s uncovering some pretty big issues, too. Prime among these is the issue of accessibility: reports have come in of buses driving away rather than let a wheelchair user on; a disabled passenger who has surmised that it’s easier to invite friends to come to him rather than try to navigate London’s public transport system; a station from which those with restricted mobility can only travel in one direction.

What next?

The big question for us is, what happens now? Are these reports making anything better? In some cases, yes.

There is the train that will now stop at intermediate stations, and the pedestrian crossings that are no longer blocked by buses. Little things that’ll make a big difference to the people who reported them. But the pay-off is not always so immediate. Bigger issues are obviously not going to be fixed overnight. And some problems won’t be fixed, for a multitude of reasons – they don’t fit in with the operators’ plans, or they’re not budgeted for, or they just aren’t seen as sufficiently important.

How is FixMyTransport going to crack those? Well, it was set up so that you can show your operator that there is demand, that budgets need to be massaged, or that plans should change. If you’ve used the site, you may well have been on the receiving end of a comment from one of the team, nudging you to spread the word of your campaign, among friends, family, and fellow-passengers.

The fact that people can sign up to your page helps make FixMyTransport different from just contacting the operator directly. We also reckon it’ll make a difference when it comes to getting changes. Consider, for example, the campaign to get increased cycle parking at Cambridge station – with 176 supporters (and still growing every day), our biggest yet. It’s been picked up by local press, talked about on Twitter – and eventually, National Express won’t be able to ignore the public demonstration of a palpable need.

Well, that’s the plan. We know it’s early days, and that FixMyTransport represents a massive sea change for some operators who are not used to interacting openly and online.

If you’ve written an impassioned, well-reasoned request, gathered supporters and spread the word far and wide, and still hit a brick wall, we have other suggestions. FixMyTransport allows us to get you writing to your local councillor, to the local paper, or to relevant groups like Passenger Focus, Transport for All, and the Campaign for Better Transport. These groups have been bashing away at the big problems like accessibility for far longer than we have, and it makes sense to tap into their expertise.

We know that for some issues, it’ll be a long game – just as it’ll be a long game trying to get every operator fully signed-up to the notion of transparent online interaction. But we’ll keep trying, and we hope you will too.

So. Yesterday we officially launched FixMyTransport, a site that has been in ‘quiet beta’ for a few weeks. Not such a big event, you might think – after all, the site has been open for public use; the only difference was that we were announcing it.

I think we’ve all been gratifyingly taken aback by just how much use the site has seen in the last 24 hours. Thanks to mentions in some of the mainstream press, but equally because of a veritable outpouring of tweets and retweets, word spread quickly. We experienced a 550% rise in visitor numbers (the servers took it in their stride, we are glad to say). Over the course of the day, the number of reports on the site doubled, with more than 70 totally new campaigns being created and many more problems being sent to operators. With each report came more tweets, more blog posts and more users signing up to campaigns.

We’re seeing the idea we worked on become a reality, and that’s both exciting and full of surprises. We knew what we would use such a site for, but we had no idea which issues would most motivate our users (at the latest reckoning, it’s poor air conditioning, delays, and, above all, a lack of decent information).

If you haven’t had a chance to see what FixMyTransport is all about yet, take a look at some of these examples:

• A passenger who has to travel west when she wants to go east
• What sounds like a complete no-brainer of a suggestion
• A sensible placatory move Virgin could extend to delayed passengers
• A call for longer commuter trains

Many of these examples see users (not just the operators, but ordinary people) who know a lot more than we do about public transport in this country weighing in with useful insights, which is fantastic.

Don’t forget you can search your local region for reports. If you find one you agree with, lending your support is as easy as clicking a single button, and then spreading the word with a tweet, a Facebook status or however you see fit.

Excuse the puns – they are hard to avoid – but we have the sense that we’re at the beginning of a very exciting journey here. And we’re sure we’re going to enjoy the ride. Thanks to everyone who’s come on board so far.

Member of the National Secular Society Robert Christian used mySociety’s Freedom of Information site, WhatDoTheyKnow to ask all 227 English NHS “provider” Trusts about how much they spend on chaplaincy.

On the 28th of February 2011 the results of his research were published in an article on the National Secular Society website (full report [PDF]). He found that £29m of NHS funds were used to pay chaplains in 2009/10 and also observed a wide variation in the amount, as a fraction of total spend, that specific trusts were spending on chaplaincy.

The publication of the research prompted a number of articles in the UK media. eg. (Daily Mail, The Independent, The Mirror).

Mr Christian has commented:

“To have identified the right FOI contact for every provider NHS Trust in England would have been daunting if not impossible. I doubt that my study would ever have got off the ground without WDTK. I particularly valued the way that the site tracks which Trust has and has not yet responded. I liked the capability to thank each FOI lead after they had responded.”

The fact that making requests via WhatDoTheyKnow allowed Mr Christian to cite the source of his raw data was important to him. He added:

“The transparency of the raw data is, I think, one of the main strengths of the WDTK website for three reasons. First, I was able to hyperlink every piece of data back to its source – and that meant that it was easy for colleagues from the NSS to check the accuracy of the data (with so many Trusts a transcription error was always a possibility). Second, it ensured that if anyone had wanted to challenge the accuracy of the data they could be directed to see that the study was simply quoting the Trusts’ own information. Third, it means that the data is there for future reference to see if there are any changes over the coming years.”

mySociety and WhatDoTheyKnow are non-partisan and don’t get involved in campaigning except in specific areas relating to openness and transparency. We take no view on issues such as how much, if anything, the NHS ought be paying for chaplaincy. However we welcome campaign groups making use of our services.

Bulk Requests

WhatDoTheyKnow currently has around 2-4 “bulk requests” per month made via its site. At the moment we don’t provide any mechanism to make bulk requests automatically. We are considering adding such a system, for requests which have been sanity checked by the WhatDoTheyKnow team. The provision of such a system would probably be associated with a mechanism for preventing other “bulk requests” from being made without the site administrators’ explicit approval.

Making the requests is only a small part of the work involved in a study such as that carried out by Mr Christian. Chasing public bodies for responses, as well as collating and analysing the information released is likely to be much more time consuming than submitting the requests themselves. This is something Mr Christian agrees with, stating:

“If enquirers are not prepared to individually contact each organisation to ask the question, I would doubt their commitment to retrieve and analyse the information (as that is actually a much bigger task)”.

Clearly any facility for enabling requests to be made in bulk will have to incorporate safeguards to ensure responsible use.

Whereas Mr Christian has been happy to conduct his research in public, and still been able to generate media coverage following publication, we are aware that many campaign groups, and others such as journalists, like to make Freedom of Information requests in private.

Mr Christian has commented on the issue of “scoops” and the effect of conducting his research in public:

“The question of ‘scoops’ is an issue for journalists and in fact this problem did happen in this case. Someone appears to have trawled the WDTK know site and noticed what I was doing. A short piece was run by the Daily Express before we completed and published the study. So clearly this might be an issue. But the risk of a spoiler being run will tend to be low when the number of organisations being contacted is large. This is because the amount of work needed to collate and analyse the data is enormous and so casual trawling will show only that a question is being asked – not what the conclusions are.”

In order to get as great a fraction of the total number of FOI responses available on WhatDoTheyKnow we have also been considering an option for making requests in private, for a fee. The idea would be that once the findings were published then the FOI response could be opened up to the public providing access to the source material backing up the story.

Any views on our ideas for the future and on the way WhatDoTheyKnow has been used for this, and similar, research would be welcome in the comments below.

The two days leading up to election day are a hugely important time for less politically-obsessive voters. The parties know that a lot of people are only starting to seriously think how to vote today and tomorrow, and TheyWorkForYou saw its biggest spike ever the day before the election, way back in 2005.

This means it’s a super-important time to get trustworthy, non-partisan information in front of as many people as possible. And you can help by doing the following simple things:

1. Go to your constituency page on the TheyWorkForYou Election Quiz and take a good look at the answers. Is there anything surprising in the answers? Has anyone failed to respond who really shouldn’t? Is there anything funny in the responses? Make a couple of notes about what you think are the most interesting findings.

2. If you know the name of your local papers or radio stations, try to Google for the email or phone number of the news desk. If you don’t know the names, try sticking the name of your nearest town into a media database like this, to get a phone number or email address.

3. If possible, you should start your pitch by phoning rather than emailing. If you get a phone number for a news desk, give them a bell and say that you’re a volunteer from “The country’s largest non-partisan election information project”, and ask for the email of a specific person who might be interested in a story about what local candidates are saying.

4. Once you have an email address of a specific journalist, compose a locally specific email for them, along the following lines:

“Hi X,

I’m a resident of Z constituency, and this election I’ve been one of 6000 volunteers helping  to build an unprecedented project to get candidates across the country to go on the record, in conjunction with the website TheyWorkForYou.com. It’s a strictly non-partisan project, aimed at giving voters a really clear, spin-free view of what their candidates stand for. I’d really appreciate it if you could give it some coverage before election day.

In my constituency, N candidates have completed our survey. From this we can see some quite interesting things, namely:

* Candidate A thinks…

* Candidate B thinks…

Would you be so kind as to print a story encouraging people to check our their candidates via TheyWorkForYou.com, and mentioning some of the highlights I’ve included?

all the best,

Your name, email, phone”

5. An hour after you send the email through, give the journalist a call back to see if they need any more help.

6. If you do this, please leave us a comment on this post so we know who’s had a go!

Thank you for helping spread some non-partisan information this election time, and enjoy the election…

Update: The Telegraph posted a retraction yesterday.

You may have seen coverage on various websites saying that a civil servant was sacked after posting a comment on TheyWorkForYou.

We’ve no idea what this story is about, but we’re pretty certain it has nothing whatsoever to do with TheyWorkForYou. No journalist bothered to contact us before running the story.

• There is no comment on TheyWorkForYou containing the text quoted in that article, nor anything like it, nor has there ever been. Nor in fact (as we’ve checked), on HearFromYourMP, WriteToThem, or WhatDoTheyKnow.
• Only one comment has been left on any contribution by Hazel Blears in 2009, and it’s definitely not related to this.
• 27 comments were left on 13th May, the date the comment was apparently posted; we’ve read them all and they’re all nothing to do with this.

So frankly, we’ve no idea what’s going on.

What we do know is that the implication that mySociety would merrily hand over sensitive personal data that ends up in getting someone sacked, without fighting tooth and nail for their privacy every inch of the way, is a complete misinterpretation of the way we work and the things we hold most dear. No-one has ever contacted us to ask us to hand over such data, nor have we ever done so.

We think what might have happened is a simple mis-remembering of the website that contained the problematic comment. We’re hoping to get in touch with Lisa Greenwood so we can get full details before asking the various media companies that have run with this for a correction.

7,000 members on the Facebook group, over 93% of MPs contacted. Lots of news coverage: BBC, Daily Mail, Guardian, Telegraph, Times. John Mann MP makes a good point in a letter to the Guardian: “Few of my constituents care about the detail of how I spend their money as long as I do a good job, but nearly all of them care that they have the right to find out if they really feel the need to.”

Tom has updated the main blog post with a quote from President Obama’s speech that I thought was worth repeating, on why this is a much bigger issue than some bits of paper and some minor embarrassment: “And those of us who manage the public’s dollars will be held to account – to spend wisely, reform bad habits, and do our business in the light of day – because only then can we restore the vital trust between a people and their government.”

If you haven’t already, do write to your MP, and pick up the phone and call your local radio and TV news stations to let them know about this.

According to some, today is the day when the world is going to be saved. Not sure if I agree with that, but recognise that the inauguration of Barack Obama is of some importance :-)). And, naturally, so is the work carried out by mysociety.org. I therefore thought today would be a good time to point to some of its achievements in 2008.

So where to begin? Well, personally I think the work with the Commuting Time Maps is worth mentioning. Developed in collaboration with the Department of Transport it enables users to work out commuting distances from one point to another. This is arguably very useful information if you are house hunting, looking for a new office or if you are an estate agent wanting to provide clients with that extra information.

Or how about picking up an award for FixMyStreet at the SustainIT eWell-Being Awards. The judges said it was “[a]n excellent example of an independent website which empowers the general public in their dealings with their local council. It is a relatively simple application, yet highly effective and replicable.” Very well done indeed.

I know I have mentioned it before, but an obvious achievement is for the charity to have stayed alive and kicking for five years. The main man behind mySociety.org, Tom Steinberg, was around the time of the anniversary featured in an article in the Guardian. Check it out for some more in-depth information about Tom and the rise of mySociety.org!

Full details of all the achievements and general ongoings in 2008 can be found on the blog, especially the successes and 2008 pages.

Obviously it’s always great when any paper gives mySociety coverage – it helps get the word about our services out and helps more people get things done that help their lives.

However, today’s look at mySociety’s 5 years in the Guardian makes a few claims I think it’s important to challenge, so instead of writing to the readers editor I thought I’d just seize the power of Citizen Media(TM) to note them here.

First, has the No10 petitions site had “little notable impact” on government policy? Given that that project appears almost single handedly to have bounced Parliament into developing an online petitioning system and devoting debate time to major petitions, I’d say that it certainly has had some impact. But there is indeed a bigger problem of pointing at No10 petitions and going “That one changed policy.” It’s a problem of two halves: scale, and deniability.  Governments almost never acknowledge that they were forced into anything, ever. Policy announcements are almost always framed as if the right course of action was being followed all along. So apart from the fact that I don’t know how one could possibly assess the impacts of so many thousands of petitions without a huge research project, I would expect that even those that do have in impact will still usually be denied by the government, even when shifting policy. I would encourage No10 and the whole of Government to take a look at directly challenging this culture, and employ someone whose job it is to find out which petitions are having an impact, and shout about them in plain English.

Second, the majority of mySociety’s sites are programmed by staff and contractors, not volunteers. The volunteers are super-essential to mySociety running every day, but the sheer size of some of our projects makes it unlikely a volunteer could have built them without giving up their day job for many months. This needs mentioning to explain why it matters if our finances are precarious!

Next – do councils find FixMyStreet an irritation or an asset? Well, last time we did a count a few weeks ago, we had 4 complaining emails from councils, and 62 supportive ones, with several linking directly to us. As for the Customer Relationship Management at councils, we’d be delighted to send reports straight into their databases without going via email first, it’s just that only one council has set up such an interface so far. I hope that FixMyStreet can put pressure on councils and their suppliers to build a small number of standardised interfaces for the good of everyone. And yes, we are building FixMyStreet for iPhone and Android, and I’m happy to talk to anyone who wants to build UIs for any other phones.

There – hope that doesn’t come across as too ungrateful to Michael Cross et al. See you at the next birthday party, I hope!

Update: I also meant to mention that I’ve never been a ‘Downing Street Insider’. I was a junior civil servant in the Prime Minister’s Strategy Unit, which is not in Downing Street and more loosely affiliated than the name might suggest.

The Register has an article today Parliament’s take on Freedom of Information which describes an FOI request I made using WhatDoTheyKnow, and the House of Commons’ refusal to respond to it because the response would be automatically republished.

Hopefully the House will choose to waive copyright on the document, and send it soon – I still haven’t seen a good reason why they could or should not.

(Also, I haven’t changed my name to Francis Stirling, hopefully The Register will correct it soon!)