1. Hacknight report

    hack night - Instagram image by Jen Bramley

    It was great to see everyone at the hacknight on Wednesday. This time, ideas ranged from using the FixMyStreet Platform for reporting of Trading Standards transgressions, to installing a Mzalendo for Greece. We hope that these ideas will flourish, and many others will arise at our future hacknights.

    Talking of which, don’t forget to reserve a place if you’d like to come along. Hacknights are weekly. All the details are here.

  2. mySociety Weekly Hack Nights – join in the conversation

    Mountain Rescue Team by BeingFocal
    Thanks to everyone who came to the inaugural mySociety Hack Night – and thanks too to our hosts, the Open Data Institute for such a great space to work in.

    Topics ranged from community-building in post-conflict societies, to mountain rescue in Wales, via an extended front-end for WriteToThem which would put campaigns in context. It really showed what a lot of exciting ideas there are, just waiting for someone to launch into them.

    We’ll be running these nights every Wednesday: we’re currently booking for the following dates, 6:00 -9:00 pm.

    Places are restricted, so drop us a line on hello@mysociety.org if you’d like to be sure of getting in. All you need is a little coding experience and a laptop.

    We’d also like to start a conversation in the comments below, so that like-minded folk can think about hacking together. If you’re looking for people to help you with an idea, or if you see something you like the look of, leave a note below and try to synchronise which nights you’ll be attending.

    Photo by Being Focal (CC)

  3. Google Summer of Code

    A Day Spent Lying in the Grass, Watching the Clouds, by Elaine Millan

    Ah, summer: walks in the park, lazing in the long grass, and the sound of chirping crickets – all overlaid with the clatter of a thousand keyboards.

    That may not be your idea of summer, but it’s certainly the ways ours is shaping up. We’re participating in Google’s Summer of Code, which aims to put bright young programmers in touch with Open Source organisations, for mutual benefit.

    What do the students get from it? Apart from a small stipend, they have a mentored project to get their teeth into over the long summer hols, and hopefully learn a lot in the process. We, of course, see our code being used, improved and adapted – and a whole new perspective on our own work.

    Candidates come from all over the world – they’re mentored remotely – so for an organisation like mySociety, this offers a great chance to get insight into the background, politics and technical landscape of another culture. Ideas for projects that may seem startlingly obvious in, say, Latin America or India would simply never have occurred to our UK-based team.

    This year, mySociety were one of the 180 organisations participating. We had almost 100 enquiries, from countries including Lithuania, India, Peru, Georgia, and many other places. It’s a shame that we were only able to take on a couple of the many excellent applicants.

    We made suggestions for several possible projects to whet the applicants’ appetite. Mobile apps were popular, in particular an app for FixMyTransport. Reworking WriteToThem, and creating components to complement MapIt and PopIt also ranked highly.

    It was exciting to see so many ideas, and of course, hard to narrow them down.

    In the end we chose two people who wanted to help improve our nascent PopIt service. PopIt will allow people to very quickly create a public database of politicians or other figures. No technical knowledge will be needed – where in the past our code has been “Just add developers”, this one is “Just add data”. We’ll host the sites for others to build on.

    Our two successful applicants both had ideas for new websites that would use PopIt for their datastore, exactly the sort of advanced usage we hope to encourage. As well as making sure that PopIt actually works by using it they’ll both be creating transparency sites that will continue after their placements ends. They’ll also have the knowledge of how to set up such a site, and in our opinion that is a very good thing.

    We hope to bring you more details as their projects progress, throughout the long, hot (or indeed short and wet) summer.

    PS: There is a separate micro-blog where we’re currently noting some of the nitty gritty thoughts and decisions that go into building something like PopIt. If you want to see how the project goes please do subscribe!

    Google Summer of Code

    Top image by Elaine Millan, used with thanks under the Creative Commons licence.

  4. 5 Years On: Why Understanding Chris Lightfoot Matters Now More Than Ever

    My friend and mySociety’s first developer Chris Lightfoot died five years ago today. He killed himself in his own flat for reasons that we will never really know, but which are doubtless linked to the depression which he’d been fighting for years. He was just 28, but had already achieved so much that The Times ran an obituary of him. He would have laughed mightily about the fact that this is now behind a paywall.

    To mark this occasion I wanted to write something for mySociety staff and volunteers who never knew Chris, and for a wider audience of people who work in places like GDSCode For America or indeed anyone with an interest in politics and governance. What Chris represented is too important to be lost in the grief at his passing.

    The basic fact to understand about Chris was that he was a very specific kind of polymath – one perfectly suited to the internet age. What I mean by this is that he did much more than simply master varying disciplines:  he saw and drew connections between fields. He wouldn’t just master cartographic principles, engage in politics and, as Francis Irving put it, ‘write Perl like other people write English’: he invariably saw the connections and mixed them up in meaningful and often pioneering ways.

    Moreover, this mixing of disciplines was conducted at a furious, restless pace, and knew absolutely no concept of ‘too hard’ – problems were either fundamentally impossible, or ‘trivially soluble’, to use one of his favourite and most gloriously under-stated phrases. Who else would build the technology to break a captcha, just to investigate what American truck rental costs tell us about internal migration in America, for fun?  The answer is trivial.

    That he was a genius is not what I want you to understand. Telling you that someone you never met was smarter than you is not helpful, and doesn’t fulfil my promise to tell you why understanding Chris matters.

    What is fundamentally valuable about Chris’ legacy (besides piles of code that power services still running today) is that his story signals how we all need to change our conception of what it means to be ‘wise enough to rule’. Let me explain.

    Unlike most of us, Chris had the luxury of being able to pick any field of study that interested him, dig up some books and papers, and teach himself a graduate-level understanding in what felt like a few days. It is hard to express quite how fast he could consume and internalise complex new information, and how relentlessly he went at it. To note that he got six A grades at A-level is too puerile a précis, but it is indicative.*

    Again, I am not telling you this to make you feel stupid: what matters is what he chose to do with this gift.  What he chose to do was built an ever-expanding palette of skills from which he could paint as he pleased. And what he chose to paint was a vision of a better, saner world.

    This painting ranged across a huge expanse of topics and disciplines: nuclear engineering, political ideologies, constitutional law, military history, statistics, psephology, economics, security engineering, behavioural psychology, propaganda, intellectual propery law and more. His favourite brushes were Perl and a blog composed of prose so sharp and funny that George Bernard Shaw would not have been displeased by the comparison. I still wish I could write half as well as him.

    What I want to communicate most is this: the disciplines he chose to study form a combined19th, 20th and 21st century curriculum of skills required by modern leaders, both leaders of political organisations and government bureaucracies.  Chris’s life was the invention of a massively expanded, far more up to date version of the traditional Politics, Philosophy and Economics course that this country still uses to educate its elites.

    Some of these disciplines are timeless, like the understanding of ideologies or economics. Some represent vital new issues that emerged in the 20th century, like nuclear energy and world-scale warfare. But mixed in there are wholly new, alien group of skills that the recent SOPA, Wikileaks and ID cards debacles show that modern leaders haven’t got anywhere near to internalising: they include knowledge about security engineering, intellectual property and how new technologies clash with old laws and ideologies. They are skills that nobody used to think were political, but which are now centre stage in a polity that can’t keep up.

    This doesn’t mean Chris would have made a perfect leader: I used to argue with him a lot about how he weighed up the costs and benefits of different issues. But what he fundamentally had right was the understanding that you could no longer run a country properly if the elites don’t understand technology in the same way they grasp economics or ideology or propaganda. His analysis and predictions about what would happens if elites couldn’t learn were savage and depressingly accurate.

    The canon of Chris’s writings and projects embody the idea that what good governance and the good society look like is now inextricably linked to an understanding of the digital. He truly saw how complex and interesting the world was when you understood power as well as networking principles in a way that few have since.

    There is, of course, much more to say about Chris’s life. His blog, built on software that foresaw Posterous, is wonderful, hilarious and utterly readable, so you can learn more yourself. Martin Keegan’s obituary is touching and a much better portrait of how much fun it was to be friends with Chris. I hope to memorialise what he represents to me, if I can. But for now, I’ll sign off with a quote from a blog commentor:

    “Chris was kind enough to take the time to reply to me, an internet nobody whom he didn’t know from a bar of soap, on a fairly complex statistical question once. He took a lot of time and effort in his response, and he made sure I understood it properly. It’s not often you find knowledgeable people willing to take their own time to educate an unknown person. We need more people like him, not less.”

    * For US readers, this is like having a GPA of 4.0, but achieved across twice as many subjects as you actually need to take.
  5. Upcoming business on TheyWorkForYou

    TheyWorkForYou has, until now, only covered things that have already happened, be that Commons main chamber debates since 1935, Public Bill committees back to 2000, or all debates in the modern Northern Ireland Assembly.

    From today, we are taking the UK Parliament’s upcoming business calendar and feeding it into our database and search engine, which means some notable new features. Firstly, and most simply, you can browse what’s on today (or the next day Parliament is sitting), or 16th May. Secondly, you can easily search this data, to e.g. see if there will be something happening regarding Twickenham. And best of all, if you’re signed up for an email alert – see below for instructions – you’ll get an email about any matching future business along with the matching new Hansard data we already send. We currently send about 25,000 alerts a day, with over 65,000 email addresses signed up to over 111,000 alerts.

    Mark originally wrote some code to scrape Parliament’s business papers, but this sadly proved too fragile, so we settled on Parliament’s calendar which covers most of the same information and more importantly has (mostly) machine-readable data. Duncan and I worked on this intermittently amidst our other activities, with Duncan concentrating on the importer and updating our search indexer (thanks as ever to Xapian) whilst I got on with adding and integrating the new data into the site.

    I’ve also taken the opportunity to rejig the home page (and fix the long-standing bug with popular searches that meant it was nearly always Linda Gilroy MP!) to remove the confusingly dense amount of recent links, bring it more in line with the recently refreshed Scottish Parliament and Northern Ireland Assembly home pages, and provide more information to users who might not have any idea what the site covers.

    Signing up for an email alert: If you want to receive an email alert on a particular person (MP, Lord, MLA or MSP), visit their page on TheyWorkForYou and follow the “Email me updates” link. If you would like alerts for a particular word or phrase, or anything else, simply do a search for what you’re after, then follow the email alert or RSS links to the right of the results page.

  6. Why I’d like mySociety to run a Masters in Public Technology

    WWII students looking at an Engine

    It is a cliché for any manager to say that they are proud of their team, and mildly nausea-inducing to listen to anyone who goes on about it too long. However, the purpose of this post is to argue that the world would benefit from a new kind of post-graduate Masters programme – something that is hard to do without  describing the virtues of the type of people who should come out of it. So please bear with me, and keep a sick bag to one hand.

    mySociety’s core development team is very, very good. But they’re not just good at turning out code. Louise Crow, for example, has a keen eye for things that will and won’t make a difference in the offline world, as well as the skills to build virtually whatever she can think of. And the exact same thing is true of the whole coding team:  Duncan, Matthew, Edmund and Dave in the current team, plus Francis, Chris and Angie before them.

    mySociety didn’t give these people their raw talent, nor the passion to be involved with projects that make a difference.   What it has given them, though, is the chance to spend a lot of time talking to each other, learning from their triumphs and their mistakes, and listening to users. This space and peer-contact made them into some of the world’s few genuine experts in the business of conceptualising and then delivering digital projects that deliver new kinds of civic and democratic benefits.

    So, why am I sitting here unashamedly blowing my colleagues trumpets like this? (I don’t have these skills, after all!) Well, in order to point out that there are quite simply far too few people like this out there.

    Too few experts

    “Too few for what?” you may well ask. Too few for any country that wants to be a really great place to live in the 21st century, is my answer.

    There is barely a not-for-profit, social enterprise or government body I can think of that wouldn’t benefit from a Duncan Parkes or a Matthew Somerville on the payroll, so long as they had the intelligence and self-discipline not to park them in the server room. Why? Because just one person with the skills, motivation and time spent learning can materially increase the amount of time that technology makes a positive contribution to almost any public or not-for-profit organisation.

    What they can do for an organistion

    Such people can tell the management which waves of technology are hype, and which bring real value, because they care more about results than this week’s craze, or a flashy presentation. They can build small or medium sized solutions to an organisation’s problems with their bare hands, because they’re software engineers. They can contract for larger IT solutions without getting ripped off or sold snake oil. And they can tell the top management of organisations how those organisations look to a digital native population, because they come from that world themselves.

    And why they don’t

    Except such experts can’t do any of these things for not-for-profit or public institutions: they can’t help because they’re not currently being employed by such bodies. There are two reasons why not, reasons which just may remind you of a chicken and an egg.

    First, such institutions don’t hire this kind of expert because they don’t know what they are missing – they’re completely outside of the known frame of reference. Before you get too snarky about dumb, insular institutions, can you honestly say you would try to phone a plumber if you had never heard that they existed? Or would you just treat the water pouring through the ceiling as normal?

    Second, these institutions don’t hire such experts because there just aren’t enough on the market: mySociety is basically the main fostering ground in UK for new ones, and we greedily keep hold of as many of our people as possible. Hands off my Dave!

    Which leads me to the proposal, a proposal to create more such experts for public and non-profit institutions, and to make me feel less guilty about mySociety hoarding the talent that does exist.

    Describing the Masters in Public Technology

    The proposal is this: there should be a new Masters level course at at least one university which would take people with the raw skill and the motivation and puts them on a path to becoming experts in the impactful use of digital technologies for social purposes. Here’s how I think it might work.

    In the first instance, the course would only be for people who could already code well (if all went well, we could develop a sister course for non-coders later on). Over the course of a single year it would teach its students a widely varied curriculum, covering the structure and activities of government, campaigns, NGOs and companies. It would involve dissecting more and less impactful digital services and campaigns, like biology students dissect frogs, looking for strengths and weaknesses. It would involve teaching the basics of social science methodologies, such as how to look for statistical significance, and good practice in privacy management. It would encourage good practice in User Experience design, and challenge people to think about how serious problems could be solved playfully. It would involve an entire module on explaining the dos and don’t of digital technology to less-literate decision makers. And most important, it would end with a ‘thesis’ that would entail  the construction of some meaningful tool, either alone or in collaboration with other students and external organisations.

    I would hope we could get great guest lecturers on a wide range of topics. My fantasy starter for 10 would include names as varied in their disciplines as Phil Gyford, David Halpern, Martha Lane Fox, Ben Goldacre, Roz Lemieux, William Perrin, Jane McGonigal, Denise Wilton, Ethan Zuckerman, as well as lots of people from in and around mySociety itself.

    What would it take?

    I don’t know the first thing about how universities go about creating new courses, so having someone who knew about that step up as a volunteer would be a brilliant start!

    Next, it would presumably take some money to make it worth the university’s time. I would like to think that there might be some big IT company that would see the good will to be gleaned from educating a new generation of socially minded, organisation-reforming technologists.

    Third, we’d actually need a university with a strong community of programmers attached, willing and ready to do something different. It wouldn’t have to be in the UK, either, necessarily.

    Then it would need a curriculum, and teaching, which I would hope mySociety could lead on, but which would doubtless best be created and taught in conjunction with real academics. We’d need some money to cover our time doing this, too.

    And finally it would need some students. But my hunch is that if we do this right, the problem will probably be fending people off with sticks.

    What next?

    I’m genuinely not sure – I hope this post sparks some debate, and I hope it provokes some people to go “Yeah, me too”. Maybe you could tell me what I should do next?

  7. Local e-Petitions – See if mySociety is providing your local system

    Councils all around England have been busy getting ready to comply with the new duty to provide e-Petitions which kicks in today, 15th December. This means that on council sites across England you should now be able to make petitions which will be formally considered by the councils, in accordance with their chosen policies.

    At mySociety we’ve spent a lot of time over the last twelve months helping councils to cope with this new duty by offering  them a commercial petitions service that is really good for users and easy to administer for councils. Some of the sites have been live for months, but many of the 35 council e-petitions sites we’re currently contracted to supply launch today.

    mySociety’s core developers Matthew Somerville and Dave Whiteland deserve huge credit for all the work they did re-purposing the No10 Petitions codebase and doing dozens of council customisations and rebrands. I’ve just seen one council officer email “Yippeee” at the prospect of launching, so I reckon they’ve done a pretty good job  – well done gents, everyone in mySociety owes you a debt of gratitude for a time consuming job well done.

    Here’s the current list of live local petitions sites. We’ll be adding more as they go up. Happy petitioning!

    Ashfield http://petitions.ashfield-dc.gov.uk/

    Barnet http://petitions.barnet.gov.uk

    Barrow http://petitions.barrowbc.gov.uk/

    Bassetlaw http://petitions.bassetlaw.gov.uk/

    Blackburn with Darwen http://petitions.blackburn.gov.uk/

    East Cambridgeshire http://petitions.eastcambs.gov.uk/

    East Northants http://petitions.east-northamptonshire.gov.uk

    Elmbridge http://petitions.elmbridge.gov.uk

    Forest Heath http://petitions.forest-heath.gov.uk

    Hounslow http://petitions.hounslow.gov.uk

    Ipswich http://petitions.ipswich.gov.uk

    Islington http://petitions.islington.gov.uk

    Lichfield http://petitions.lichfielddc.gov.uk

    Mansfield http://petitions.mansfield.gov.uk/

    Melton http://petitions.melton.gov.uk/

    New Forest http://petitions.newforest.gov.uk

    Nottinghamshire http://petitions.nottinghamshire.gov.uk

    Reigate & Banstead http://petitions.reigate-banstead.gov.uk

    Runnymede http://petitions.runnymede.gov.uk

    Rushcliffe http://petitions.rushcliffe.gov.uk/

    South Holland http://petitions.sholland.gov.uk

    Spelthorne http://petitions.spelthorne.gov.uk

    St Edmundsbury http://petitions.stedmundsbury.gov.uk

    Stevenage http://petitions.stevenage.gov.uk

    Suffolk Coastal http://petitions.suffolkcoastal.gov.uk/

    Surrey County Council http://petitions.surreycc.gov.uk

    Surrey Heath http://petitions.surreyheath.gov.uk

    Tandridge http://petitions.tandridge.gov.uk

    Waveney http://petitions.waveney.gov.uk

    Waverley http://petitions.waverley.gov.uk

    Wellingborough http://petitions.wellingborough.gov.uk

    Westminster http://petitions.westminster.gov.uk

    Royal Borough of Windsor and Maidenhead http://petitions.rbwm.gov.uk

    Woking http://petitions.woking.gov.uk

  8. Report submission edits

    A number of people report dog fouling through FixMyStreet, using slightly more… colloquial language. A number of councils have strict obscenity filters, blocking anything containing swearing. As I’m a pragmatist and not that interested in campaigning against councils blocking legitimate emails from their citizens (feel free!), FixMyStreet simply changes any “dog shit” reference to “dog poo”. This works well for everyone.

    Recently, the infamous Intellectual Property Manager from Portakabin™ Limited got in touch to complain about a couple of reports on FixMyStreet containing the words “portacabin” or “portaloo”. Again, as a pragmatist, I’m not really interested in whether users using trade marks or trade mark variants in a generic way on a problem report actually constitutes trade mark infringment (actually, I’d guess not), I just want legal people to go away and not waste our precious resources. So from now on, any report containing portakabin or similar will become [portable cabin], and portaloo will become [portable loo].

    For anyone who’s interested, this is accomplished through a simple regular expression, that looks for porta followed by 0 or more spaces, then cabin, kabin, or loo, and sticks “ble” in the middle.

  9. RIP Angie Martin 1974-2009

    It is with overwhelming sadness that I write to tell our community that Angie Martin, mySociety’s fourth core developer, has died. She was taken from us by the cancer that she had been fighting since soon after we hired her less than two years ago.

    Possessed of an almost unbelievably upbeat personality, Angie brought not only her formidable Perl skills, but her blazing warmth of character to our team. In remission during our yearly retreat in January this year, she combined laughter with a typically tough line of questioning on ideas she thought insufficiently robust. With typical disgregard for cool, her CV noted that she was “known to enjoy wrangling regular expressions on a Sunday Morning”. She didn’t see any contradiction between being a successful woman and a geek, throwing herself wholeheartedly into the Mac-toting, perlmonger ethos. She even brought her husband Tommy with her, who became a significant volunteer.

    Given her habit of plain speaking, it is pointless to pretend that Angie was able to make the contribution to mySociety’s users or codebase that she wanted to. What she achieved in terms of difficult coding during recovery from chemotherapy was incredible, breathtaking – but she wanted to change the world. It now falls to the rest of us, and our supporters, to live up to the expectations she embodied, to continue to push every day, using skills like those that she had to help people with everyday problems. We now have to ask ‘What would Angie do?’, as well as ‘What would Chris do?’. It is a lot to live up to.

    She was a mySociety core developer: I hope that meant as much to her as it meant for me to have her as one of my coders.  Remember and Respect.

    Updated: Angie changed her surname upon getting married, a couple of months ago. I have just read she wanted to be remembered as Angie Martin, and so I have made that change.

    Updated 21 7 2009: Tommy has just told me that those wishing to may memorial donations should send them to Hospice at Home.

  10. Freedom on Rails

    This week has been quite bitty. I’ve been doing more work on the Freedom of Information site, have been getting into the swing of Ruby on Rails. Once you’ve learnt its conventions, it is quite (but not super) nice.

    As far as languages are concerned, Ruby seems identical in all interesting respects to Python. It’s like learning Spanish and Italian. Both are super languages. Ruby has nice conventions like exclamation marks at the end of function names to indicate they alter the object, rather than return the value (e.g. .reverse!). But then Python has a cleaner syntax for function parameters. It is swings and roundabouts.

    Rails has lots of ways of doing things which we already have our own ways of doing for other sites. The advantage of relearning them, is that other people know them too. So Louise was able to easily download and run the FOI site, and make some patches to it. Which would have been much harder if it was done like our other sites. Making development easier is vital – for a long time I’ve wanted a web-based cleverly forking web application development wiki. But while I dream about that, Rails packaging everything you need to run the app in a standard way in one directory that quite a few people know how to use, helps.

    Other things… I’ve been helping Richard set up GroupsNearYou on our live servers, it should be ready for you to play with soon. It looks super nice, and is easy to use. I’ve had some work to do with recruitment. And catching up on general customer support email for TheyWorkForYou and PledgeBank. I’ve also been updating the systems administration documentation on our internal wiki, so others can work out how to run our servers.