It’s been a while since we updated you on the progress of our next major project, FixMyTransport, but we’re still working hard behind the scenes. As you may recall, FixMyTransport will deal with public transport problems – delayed trains, vandalised stations, overcrowded buses, you name it. It’ll put problems in the public arena, while also reporting them directly to the relevant transport operator. Read more about the project here.

We will shortly be arriving at our final destination

Things are going to get exciting very soon. As launch date approaches, we’ll be starting a closed beta (mid July), rapidly followed by a full open public beta launch (end of July). During the closed beta we want to get as much feedback as possible from future users of the site, as well as pressure groups, transport operators, and anyone else who has anything valuable to contribute.

If you would like to be invited to beta test, and weren’t one of our alpha testers, please email us on team@fixmytransport.com. Alpha testers will, of course, be invited to test again.

Mind the (data) gaps

We got extremely useful feedback from our alpha testers, and a wealth of crowdsourced data from our community. Thanks to their efforts we now have contact details for the operators of about 50% of the routes in the UK. However, this leaves a lot of operators where we don’t know how to get in touch.

We really need your help to get them! If you can spare a few minutes, visit our spreadsheet and see if you can fill in any of the missing details.

The more contact details we can get hold of, the better experience FixMyTransport will offer to our users. As well as publishing passengers’ reports on the site, FixMyTransport sends them directly to the operators too, helping to get the issue fixed.

So, we especially need the email addresses for operators’ customer services departments. Finding these may be as simple as visiting the operators’ websites, or it may require a bit of sleuth-work on your part.  If advanced Googling gets you nowhere, we’ve found that simply phoning head office can get results.

Incidentally, the main operators are near the top of the sheet – those are the ones that will benefit the most users, although obviously the nearer completion we get, the better.

You’ll notice that the spreadsheet now includes a non-obligatory column for your name: this is to offer a small incentive. If you want to, tag your entries and at the end we’ll be offering goodies to the top contributors. Depending on your preference, this might be one of our highly sought-after mySociety hooded tops (they’re snuggly!), or a chance to become more involved in the project.

Those who helped in the first iteration, please note that although this sheet looks different, your details have been retained and indeed have been extremely useful as we build the site. Also –  if  you have already been a major contributor during our previous rounds of testing and data collecting, please holler so that we can give you proper credit.

Hold tight, please

Not long now… we hope you’re as excited as we are.

Ever got a problem fixed by reporting it on FixMyStreet? Written to your representative via WriteToThem? Here’s an opportunity to pay the favour forward to someone stranded on a wet Wednesday by the non-arrival of the number seven bus.

We’ve reached the point in FixMyTransport development where we can start asking for your help. We need to fill in the information we’ll use to report people’s transport problems to the companies that run bus and train routes. If you have five minutes to spare, please spend them adding a contact email address or two for your local bus companies to this spreadsheet:

http://bit.ly/aVZzlb

…then you can bask in the glory of a karmic balance restored*.

* Will also work if you accidentally ran over a kitten on your way to work this morning.

Matthew’s just updated ScenicOrNot, the little game that we built to provide a ‘Scenicness’ dataset for Mapumental, to include a data dump of the raw data. The dump will update automatically on a weekly basis, but currently it contains averaged scores for 181,188 1*1km grid squares, representing 83% of the Geograph dataset we were using, or 74% of all the grid squares in Great Britain. It is, in other words, really pretty good, and, I think, unprecedented in coverage as a piece of crowd sourced geodata about a whole country.

It’s available under the Creative Commons Attribution Noncommercial 3 Licence, and we greatly look forward to seeing what people do with it.

Score - 9.5: Across Loch Ericht to Sron a Chlaonaidh (by Geoff White on Geograph)

Harry Metcalfe and co have been working for us recently to build a new mini-site, ScenicOrNot.

The goal of ScenicOrNot is to be a gentle-ambling sort of quasi game that’s just compelling enough to keep clicking on, just in case the next picture is the dream valley in which you wish to be buried.

mySociety’s obviously not in the business of building games for their own sake, though. This is another crowdsourcing experiment to solve a specific problem – we need a scenicness map of the UK for a major upcoming mySociety project, and there ain’t one to be had any other way, for love or money.

So if you like mySociety, or just want to ogle the best and worst of this Island, please have a play.

We sometimes have incorrect or out of date addresses for sending Freedom of Information requests to. Now anyone can check our addresses. Click “view FOI email address” on the page for any authority, and enter two of those squiggly words to prove you are not a robot.

If you are using WhatDoTheyKnow, and suspect problems with a request, please do check the address we are using is correct. If you are from an authority, or work closely or know a particular authority, please also check the address.

The epic task of manually matching each of the 42,019 video clips of MPs was started way, way back, ooh, about 12 whole weeks ago. Two days ago the Number 1 rated volunteer timestamper in our league table, Abi Broom, completed the last clip in our database, bringing her personal tally to 8,543 clips.

Abi Broom, No1 timestamping league table champ

Last night we went out and met with Abi and Robert Whittakker, one of the other super-timestampers who had done over 2,000 himself.

As a result of their efforts, and those of hundreds of other volunteers, we have put all the video that we have of the House of Commons sitting over the last year online, next to the text of the debates. The many thousands of people per day who visit TheyWorkForYou can, as a direct consequence of this work, now see video of most of the debates for the last year. When people embed clips on their own sites, that’ll also be thanks in part to our volunteers.

Robert Whittaker, down in 5th place with a measly 2047 clips to his name

When Parliament starts again in the Autumn there’ll be another 300-400 clips a day to do, but we have a feeling the only problem doing them will be who gets to them first.

In the meantime, we’ll soon be working on another game-like toy to help create more data. Hint – it might have something to do with GroupsNearYou.

There’s been a lot written recently about the cognitive surplus, a phrase coined by Clay Shirky to describe the amount of human energy that can be deployed to create things if only barriers are lowered and incentives sharpened.

mySociety has recently been fortunate enough to see a little of this phenomenon through the explosion of volunteering activity which grew up around our TheyWorkForYou video timestamping ‘game’. For those of you not familiar, we needed video clips of politician’s speaking matched with the text of their speeches, and in just a couple of months a gang of volunteers new and old have done almost all of the video in the archive. Other, much larger examples include reCAPTCHA and the ESP game.

Reflecting on this, my friend Tom Lynn suggested that there was a gap in the market for a service that would draw together different crowdsourcing games, ensure that their usability standards and social benefit were high, and which then syndicate them out in little widgets, recaptcha style, to anyone who wanted to include one on a web page.

This is where Mozilla and Ubuntu come in. Anyone who uses Firefox knows what the home page is like, essentially the Google homepage with some Firefox branding. Ubuntu’s default browser homepage, post patch upgrade especially, is similarly minimalist and focused on telling you what’s changed.

Therein lies the opportunity – using pieces of these default home pages (maintained by organisations that claim to have a social purpose, remember) for more good than simply repeatedly reminding users about the the brand of the product. Traditionally that would mean asking people to donate or become volunteers, but the new universe of ultra-easy crowdsourcing games are challenging that assumption.

Here’s a scenario. One time in ten when I load Firefox, the homepage contains a widget right under the search box that contains an almost entirely self explanatory task that contributed to the public good in some way. This could be spotting an object on a fragment of satellite photo after a disaster, typing in a word that’s difficult to OCR, timestamping a video clip, or adding tags to an image or a paragraph of text. The widgets would be syndicated from the central repository of Cognitive Surplus Foundation ‘games’, and would help groups like Mozilla and Ubuntu to show themselves to millions of tech-disinterested users to be the true 21st century social enterprises that they want to be.

TheyWorkForYou now finds whenever an old version of Hansard is referenced (which they do by date and column number, e.g. Official Report, 29 February 2008, column 1425) and turns the citation into a link to a search for the speeches in that column on that date. This only really became feasible when we moved server, upgraded Xapian, and added date and column number metadata (among others), allowing much more advanced and focussed searching – the advanced search form gives some ideas. Perhaps in future we’ll be able to add some crowd-sourcing game to match the reference to the exact speech, much like our video matching (nearly 80% of our archive done!). 🙂

Kudos to Google and Yahoo! for spotting this change within a couple of days, as they’re now so busy crawling everything for changes that they’re slowing the whole website down… 😉

TheyWorkForYou video timestamping has been launched, over 40% of available speeches have already been timestamped, and (hopefully) all major bugs have been fixed, so I can now take a short breather and write this short series of more technical posts, looking at how the front end bits I wrote work and hang together.

Let’s start with the most obvious feature of video timestamping – the video player itself. 🙂 mySociety is an open-source shop, so it was great to discover that (nearly all of) Adobe Flex is available under the Mozilla Public Licence. This meant I could simply download the compiler and libraries, write some code and compile it into a working SWF Flash file without any worries (and you can do the same!).

Writing a Flex program is split into three main areas – MXML that lays out your application, defines any web services you’re using and so on; CSS to define the style of the various components; and ActionScript to deal with things like events, or talking to the JavaScript in the parent HTML. My code is probably quite shoddy in a number of places – it’s my first application in Flex 🙂 – but it’s all available to view if you want to take a peek, and it’s obviously running on the live TheyWorkForYou site.

To put a video component in the player is no harder than including an <mx:VideoDisplay> element – set the source of that, and you have yourself a video player, no worrying about stream type, bandwidth detection, or anything else. 🙂 You can then use a very useful feature called data binding to make lots of things trivial – for example, I simply set the value of a horizontal slider to be the current playing time of the video, and the slider is then automatically in the right place at all times. On the downside, VideoDisplay does appear to have a number of minor bugs (the most obvious one being where seeking can cause the video to become unresponsive and you have to refresh the page; it’s more than possible it’s a bug in my code, of course, but there are a couple of related bugs in Adobe’s bug tracker).

As well as the buttons, sliders and the video itself, the current MXML contains two fades (one to fade in the hover controls, one to fade them out), one time formatter (to format the display of the running time and duration), and three web services (to submit a timestamp result, delete a mistaken timestamp, and fetch an array of all existing timestamps for the current debate). These are all called from various places within the ActionScript when certain events happen (e.g. the Now button or the Oops button is clicked).

Compiling is a simple matter of running mxmlc on the mxml file, and out pops a SWF file. It’s all straightforward, although a bit awkward at first working again with a strongly-typed, compiled language after a long time with less strict ones 🙂 The documentation is good, but it can be hard to find – googling for [flex3 VideoDisplay] and the like has been quite common over the past few weeks.

Tomorrow I will talk about moving around within the videos and some bugs thrown up there, and then how the front end communicates with the video in order to highlight the currently playing speech – for example, have a look at last week’s Prime Minister’s Questions.

1. The Flash player
2. Seeking
3. Highlighting the current speech

Wow! The video timestamping site on TheyWorkForYou has been live for just under ten days, but you’ve already managed to timestamp almost 14,000 clips – that’s almost 40% of our entire video archive!

Thank you for all the good work – and especially to our top timestampers. David Jones, Alex Hazell and Lee Maguire are currently the top three in the overall rankings, but there are five more people who have timestamped more than 500 clips, another seventeen people who have done 100 clips or more, and more than 100 people who’ve done anything from 1 to 100 clips. And of course, there’s also a fair few anonymous people who haven’t yet registered, so their individual contributions to the “anonymous” total of 3349 clips are not recorded on the league tables. Remember, we’ll be handing out prizes to the top timestampers, so get registered before you timestamp your next video!

We’re starting to collect a list of notable clips that we can use to compile a “best of parliament” video gallery – if you would like to nominate a particular speech, please leave a comment below or send an email to team@theyworkforyou.com – just tell us the name of the MP speaking, and the URL of the page where this speech appears on theyworkforyou.com. We’ll put the best of them together and publish a list later this summer.