Score - 9.5: Across Loch Ericht to Sron a Chlaonaidh (by Geoff White on Geograph)

Harry Metcalfe and co have been working for us recently to build a new mini-site, ScenicOrNot.

The goal of ScenicOrNot is to be a gentle-ambling sort of quasi game that’s just compelling enough to keep clicking on, just in case the next picture is the dream valley in which you wish to be buried.

mySociety’s obviously not in the business of building games for their own sake, though. This is another crowdsourcing experiment to solve a specific problem – we need a scenicness map of the UK for a major upcoming mySociety project, and there ain’t one to be had any other way, for love or money.

So if you like mySociety, or just want to ogle the best and worst of this Island, please have a play.

From one extreme to another – the diversity among mySociety’s volunteers is most fascinating. You may, or may not, have read one of my previous blog posts on Tim Morley, a volunteer responsible for the day to day running of pledgebank.com. If you have, you will know that he is a primary school language teacher who is making a massive contribution towards keeping that site going.

It may be difficult to believe, and mySociety is ever so grateful for it, but there are actually other volunteers out there who are just as dedicated. One of them are Amandeep Rehlon – Treasurer as well as Trustee of UKCOD and Director of mySociety.

Having a background in accounting and finance Amandeep is responsible for, among other things, VAT returns, annual accounts and paying staff. We all know that money makes the world go around, so quite a lot of responsibility there.

As dealing with finances at mySociety is not enough, he also does it in his “real” job working for the Bank of England. As he puts it himself, being based in the Financial Stability area he has been ‘ahem, rather busy in the last year or so’. Yeah, I can imagine!

Amandeep originally got involved with mySociety in late 2005 as he was seen as a suitable replacement for Tom Loosemore, the previous Treasurer who left to due to family commitments. Three years or so later, he’s still around because he thinks ‘mySociety’s sites make the world a better place’ and (as with Tim Morley) he likes the people that he works with. Very encouraging to read that he also knows a lot of people who are using mySociety’s services in their day jobs – we would like to hear more of that!

When he has some spare time (not sure when that would be but there you go) he plays hockey, read, listen to music (including lots of gigs) and eat too much chocolate – ‘especially from James’s chocolate shop’. Sounds like a full on life, but fortunately with a bit of fun in between.

Heard the song ‘We Built this City (on rock and roll)’ by Starship? No? Not to worry, I am just trying to draw parallels with how mySociety.org is organised and managed. You may not have realised, but a good sized chunk of the work that we do is actually carried out by volunteers, that includes everything from translating a single page to full website development

In other words, volunteers make our organisation tick and thought it about time that we shed some light on who they are and what they do. First out being subject to closer scrutiny is Tim Morley who looks after the everyday running of PledgeBank.com.

Tim has been volunteering for mySociety.org since 2005. Having heard about us through an article in the Guardian, he started out by translating content to Esperanto, and has through the years progressed to his current role.

Being a trained primary school language teacher, Tim estimates that he spends anything from five minutes to 3-4 hours a day on his volunteering work depending on how much there is to do. Task varies from helping users with technical queries to help out organising events.

Challenge was an initial motivating factor in deciding to start volunteering. He could also see the benefit of PledgeBank.com as a tool for the Esperanto speaking community, to help and encourage people to organise happenings in what is a very widely- and thinly-spread group. Three years later what keeps him going are the people involved with mySociety.org, the fact that he’s impressed with other things that we are doing and is proud to be associated with our organisation. Making a contribution and taking PledgeBank.com forward still feels important.

If you are interested in volunteering for mySociety.org don’t hesitate to get in touch. As can be seen above, Tim is a highly involved volunteer. But all contributions to the running of our organisation are appreciated so don’t let him put you off. :-)) Further info on some of our other volunteers is in the pipeline if you are looking for inspiration on how you can help out.

Super WhatDoTheyKnow volunteer John Cross has made an interesting petition about Freedom of Information and publicly owned companies

“We the undersigned petition the Prime Minister to support a change to the law to make companies owned two thirds or more by public authorities subject to the Freedom of Information Act 2000.”

The petition goes on to explain (in more details at the bottom right of the petition page) that the situation is quite comical at the moment. If a company is owned by one local authority, then it is subject to FOI, but if it is jointly owned by two then it isn’t. This makes little sense, and it is also very important, as private companies owned by authorities often do important work.

Obviously it’s always great when any paper gives mySociety coverage – it helps get the word about our services out and helps more people get things done that help their lives.

However, today’s look at mySociety’s 5 years in the Guardian makes a few claims I think it’s important to challenge, so instead of writing to the readers editor I thought I’d just seize the power of Citizen Media(TM) to note them here.

First, has the No10 petitions site had “little notable impact” on government policy? Given that that project appears almost single handedly to have bounced Parliament into developing an online petitioning system and devoting debate time to major petitions, I’d say that it certainly has had some impact. But there is indeed a bigger problem of pointing at No10 petitions and going “That one changed policy.” It’s a problem of two halves: scale, and deniability.  Governments almost never acknowledge that they were forced into anything, ever. Policy announcements are almost always framed as if the right course of action was being followed all along. So apart from the fact that I don’t know how one could possibly assess the impacts of so many thousands of petitions without a huge research project, I would expect that even those that do have in impact will still usually be denied by the government, even when shifting policy. I would encourage No10 and the whole of Government to take a look at directly challenging this culture, and employ someone whose job it is to find out which petitions are having an impact, and shout about them in plain English.

Second, the majority of mySociety’s sites are programmed by staff and contractors, not volunteers. The volunteers are super-essential to mySociety running every day, but the sheer size of some of our projects makes it unlikely a volunteer could have built them without giving up their day job for many months. This needs mentioning to explain why it matters if our finances are precarious!

Next – do councils find FixMyStreet an irritation or an asset? Well, last time we did a count a few weeks ago, we had 4 complaining emails from councils, and 62 supportive ones, with several linking directly to us. As for the Customer Relationship Management at councils, we’d be delighted to send reports straight into their databases without going via email first, it’s just that only one council has set up such an interface so far. I hope that FixMyStreet can put pressure on councils and their suppliers to build a small number of standardised interfaces for the good of everyone. And yes, we are building FixMyStreet for iPhone and Android, and I’m happy to talk to anyone who wants to build UIs for any other phones.

There – hope that doesn’t come across as too ungrateful to Michael Cross et al. See you at the next birthday party, I hope!

Update: I also meant to mention that I’ve never been a ‘Downing Street Insider’. I was a junior civil servant in the Prime Minister’s Strategy Unit, which is not in Downing Street and more loosely affiliated than the name might suggest.

One of the nicest things about being involved with mySociety is seeing people in other countries starting similar organisations and building similar websites. After a Skype conversation with another eager hopeful last night, I thought I’d blog a bit about the things I think are most important to know if you’re just starting up. Here goes.

1. Absolutely the most totally essential thing is to be an organisation of amazing, politically minded coders, not an organization employing or contracting good coders*. Their skills are your lifeblood, their ideas your bread and butter, and finding the best civic hackers in your country and building your organization around them is the only path to success. And that means they should be making most of the day to day decisions, not you, you ignorant, arts-degree-clutching clot.

2.Ask the public what they think you should build. Not only will that give you access to more ideas than you have yourselves, it’ll engage people with you. Also, it’ll help you focus on the vital business of building sites that users want, not that YOU want.

3. Keep your cost base low, and put all the money you have into looking after your core staff and being nice to volunteers. Work on building a community of volunteers, even if most of them are really just friends rather than people putting in lots of time. Avoid renting offices, avoid non-essential non-coder staff, get people to donate serving infrastructure and bandwidth. Because building and running democratic websites is a fundamentally new area of human endeavor (not like blogging which at least has an analogy in journalism) there are basically no pre-existing funding streams for the type of work you’re about to do. You will have to create the buzz around yourselves that will lead to people wanting to fund you, and it will probably take years, if you get there at all (mySociety hasn’t quite done this yet, even at 5).

4. Ensure that the core of what you build can struggle on by even if your whole organization collapses. That means being open source, putting energy into sites that are as automated as possible, and making people excited about being volunteers.

5. If you aren’t pissing off at least some people all the time, you’ve probably been captured by the establishment.

6. Take whatever your first website plan is and remove 90% of the features you want. Then build it and launch it and your users will tell you which features they actually wanted instead. Build them and bask in the warm glow of appreciation.

*yes, yes I know I can’t code for toffee, and I’ve got an arts degree, but I’m still a geek, honest. (Proudest moment, working out that a batch of PCI network cards were unreliable because they’d come from the factory flashed with the EPROMs for the wrong hardware, and fixed it.)

The epic task of manually matching each of the 42,019 video clips of MPs was started way, way back, ooh, about 12 whole weeks ago. Two days ago the Number 1 rated volunteer timestamper in our league table, Abi Broom, completed the last clip in our database, bringing her personal tally to 8,543 clips.

Abi Broom, No1 timestamping league table champ

Last night we went out and met with Abi and Robert Whittakker, one of the other super-timestampers who had done over 2,000 himself.

As a result of their efforts, and those of hundreds of other volunteers, we have put all the video that we have of the House of Commons sitting over the last year online, next to the text of the debates. The many thousands of people per day who visit TheyWorkForYou can, as a direct consequence of this work, now see video of most of the debates for the last year. When people embed clips on their own sites, that’ll also be thanks in part to our volunteers.

Robert Whittaker, down in 5th place with a measly 2047 clips to his name

When Parliament starts again in the Autumn there’ll be another 300-400 clips a day to do, but we have a feeling the only problem doing them will be who gets to them first.

In the meantime, we’ll soon be working on another game-like toy to help create more data. Hint – it might have something to do with GroupsNearYou.

There’s been a lot written recently about the cognitive surplus, a phrase coined by Clay Shirky to describe the amount of human energy that can be deployed to create things if only barriers are lowered and incentives sharpened.

mySociety has recently been fortunate enough to see a little of this phenomenon through the explosion of volunteering activity which grew up around our TheyWorkForYou video timestamping ‘game’. For those of you not familiar, we needed video clips of politician’s speaking matched with the text of their speeches, and in just a couple of months a gang of volunteers new and old have done almost all of the video in the archive. Other, much larger examples include reCAPTCHA and the ESP game.

Reflecting on this, my friend Tom Lynn suggested that there was a gap in the market for a service that would draw together different crowdsourcing games, ensure that their usability standards and social benefit were high, and which then syndicate them out in little widgets, recaptcha style, to anyone who wanted to include one on a web page.

This is where Mozilla and Ubuntu come in. Anyone who uses Firefox knows what the home page is like, essentially the Google homepage with some Firefox branding. Ubuntu’s default browser homepage, post patch upgrade especially, is similarly minimalist and focused on telling you what’s changed.

Therein lies the opportunity – using pieces of these default home pages (maintained by organisations that claim to have a social purpose, remember) for more good than simply repeatedly reminding users about the the brand of the product. Traditionally that would mean asking people to donate or become volunteers, but the new universe of ultra-easy crowdsourcing games are challenging that assumption.

Here’s a scenario. One time in ten when I load Firefox, the homepage contains a widget right under the search box that contains an almost entirely self explanatory task that contributed to the public good in some way. This could be spotting an object on a fragment of satellite photo after a disaster, typing in a word that’s difficult to OCR, timestamping a video clip, or adding tags to an image or a paragraph of text. The widgets would be syndicated from the central repository of Cognitive Surplus Foundation ‘games’, and would help groups like Mozilla and Ubuntu to show themselves to millions of tech-disinterested users to be the true 21st century social enterprises that they want to be.

Debate pages that have at least one timestamped speech (such as the previously mentioned last week’s Prime Minister’s Questions) have a video fixed to the bottom right hand corner (if your browser is recent enough) showing that debate. While playing the video, the currently playing speech is highlighted with a yellow background, and you can start watching from any timestamped speech by clicking the “Watch this” link by any such speech. So how does all that work?

I’m very proud of this feature, I wasn’t sure it would be possible, and it’s very exciting. 🙂

Flash has an ExternalInterface API, where JavaScript can call functions in the Flash, and vice-versa. When the video player loads, it requests an XML list from the server of all speech GIDs and timestamps for the current debate (here’s the file for the above debate). So when someone clicks a “Watch this”, it calls a moveVideo function in main.mxml with the GID of the speech, which loops through all the speeches and moves to the correct point if possible.

The highlighting works the other way – as the video is playing, it checks to see which speech we’re currently in, and if there’s been a change, it calls the updateSpeech function in TheyWorkForYou’s JavaScript, which finds the right row in the HTML and changes the class in order to highlight it. Quite straightforward, really, but it does make following the debate very simple and highlights the linking between the video and the text, all done by our excellent volunteers (join in! 🙂 ).

Talking of our busy timestampers, I’ve also been busy making improvements (and fixing bugs) to the timestamping interface to make things easier for them. As well as warnings when it looks like two people are timestamping the same debate at the same time, various invisible things have been changed, such as using other people’s timestamps to make the start point for future timestamps on the same day more accurate. I also added a totaliser, using the Google Chart API, for which you simply have to provide image size and percentage complete.

Approaching 45% of our entire archive of video timestamped, with the totaliser approaching the chartreuse 🙂

Previous articles

1. The Flash player
2. Seeking
3. Highlighting the current speech

TheyWorkForYou video timestamping has been launched, over 40% of available speeches have already been timestamped, and (hopefully) all major bugs have been fixed, so I can now take a short breather and write this short series of more technical posts, looking at how the front end bits I wrote work and hang together.

Let’s start with the most obvious feature of video timestamping – the video player itself. 🙂 mySociety is an open-source shop, so it was great to discover that (nearly all of) Adobe Flex is available under the Mozilla Public Licence. This meant I could simply download the compiler and libraries, write some code and compile it into a working SWF Flash file without any worries (and you can do the same!).

Writing a Flex program is split into three main areas – MXML that lays out your application, defines any web services you’re using and so on; CSS to define the style of the various components; and ActionScript to deal with things like events, or talking to the JavaScript in the parent HTML. My code is probably quite shoddy in a number of places – it’s my first application in Flex 🙂 – but it’s all available to view if you want to take a peek, and it’s obviously running on the live TheyWorkForYou site.

To put a video component in the player is no harder than including an <mx:VideoDisplay> element – set the source of that, and you have yourself a video player, no worrying about stream type, bandwidth detection, or anything else. 🙂 You can then use a very useful feature called data binding to make lots of things trivial – for example, I simply set the value of a horizontal slider to be the current playing time of the video, and the slider is then automatically in the right place at all times. On the downside, VideoDisplay does appear to have a number of minor bugs (the most obvious one being where seeking can cause the video to become unresponsive and you have to refresh the page; it’s more than possible it’s a bug in my code, of course, but there are a couple of related bugs in Adobe’s bug tracker).

As well as the buttons, sliders and the video itself, the current MXML contains two fades (one to fade in the hover controls, one to fade them out), one time formatter (to format the display of the running time and duration), and three web services (to submit a timestamp result, delete a mistaken timestamp, and fetch an array of all existing timestamps for the current debate). These are all called from various places within the ActionScript when certain events happen (e.g. the Now button or the Oops button is clicked).

Compiling is a simple matter of running mxmlc on the mxml file, and out pops a SWF file. It’s all straightforward, although a bit awkward at first working again with a strongly-typed, compiled language after a long time with less strict ones 🙂 The documentation is good, but it can be hard to find – googling for [flex3 VideoDisplay] and the like has been quite common over the past few weeks.

Tomorrow I will talk about moving around within the videos and some bugs thrown up there, and then how the front end communicates with the video in order to highlight the currently playing speech – for example, have a look at last week’s Prime Minister’s Questions.

1. The Flash player
2. Seeking
3. Highlighting the current speech