1. PetitionYourCouncil.com: making local council petitioning easier

    PetitionYourCouncil.com from mySociety

    [Note, November 2014: Petition Your Council has now been retired]

    Local petitions can be highly effective, and we think that making them easier to create is in the public interest. Many councils have petitions facilities buried deep within their websites,  most often, very deeply. In fact it brings to mind Douglas Adams’ quote about important council documents being “on display on the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard'”.

    Our most recent mini-project is an attempt to make it as easy as possible to find your local council’s e-petitioning site, if they have one. PetitionYourCouncil.com (you’ll notice we stuck to our tried and tested format for site names, there) is a way of finding every council e-petitioning website we know about.

    Our original motivation for building the site was that we, along with other suppliers, have supplied online e-petitioning sites to numerous councils ourselves – it’s one of the ways in which we fund our charitable activities. Having delivered these sites, we later noticed that many of them are left under-used and in some cases, not used at all: only because people don’t know about them. We hate to think of councils spending money on a splendid resource that could be improving democratic processes for their citizens – and those citizens never knowing that they exist. In particular, we owe Dave Briggs thanks for pushing us into action with this blog post.

    And yes, in case you’re wondering, PetitionYourCouncil links to every council petitions site, not just the ones we made.

    The site was built by mySociety developer Edmund von der Burg using Django, jQuery, Google maps and Mapit, and like most mySociety projects, it’s open source. There’s a bit more detail on the About page. Please do try it out, and let us know what you think.

  2. FixMyStreet in Norway

    At mySociety we love our site FixMyStreet – it’s the epitome of a web tool that gives simple tangible benefits whilst generating a little accountability at the same time. Reports through the site were up 40% last year, so it’s clear that users quite like it too.

    FixMyStreet has been copied in many different countries, which makes everyone in mySociety very happy, too, even apparently appearing in a slide deck the White House uses to show innovative services. However, it turns out that the cheerfully minimalist, almost wantonly unfashionable user-interface has an unfortunate down side: most people who copy the site look at it, think “That looks easy!” and then cheerfully start coding their own clone.

    Deceptive simplicity

    Alas – the very simplicity that makes the site good hides the fact that making a site like FixMyStreet really work well is actually way harder than it looks. What will you do when a government email inbox fills up? What about when administrative boundaries change, due to an election or restructuring? How do you know you’re not scaring users away with careless wording? All the hard-won lessons from these questions have been baked into the FixMyStreet codebase, and we’re only too keen to talk to people about them.

    We were therefore particularly pleased when the Nowegian Unix User’s Group (NUUG) came to us to ask if we could help improve FixMyStreet to make it easier for them to install. Over the last month mySociety Senior Developer Matthew Somerville has been working hard with Petter Reinholdtsen and Christer Gundersen of NUUG, and here’s what they’ve managed in just a handful of weeks.

    • The launch of a Norwegian FixMyStreet called Fiksgatami, covering nearly every corner of Norway’s 300,000 square kilometers.
    • Problems reported anywhere within Norway will be correctly directed to any of the 400+ responsible municipalities, thanks to Petter and Christer’s amazing data sourcing skills.
    • As a necessary side-effect of developing this, Norway now has a free, public administrative web service gazeteer – http://mapit.nuug.no. If Norway is anything like the UK this will soon become an indispensable service for many other web sites and mobile tools.
    • The standard mapping is now OpenStreetMap, pulled together by the brilliant Norwegian OpenStreetMap community*. We couldn’t take a technological step backward, and so whilst the site uses OpenLayers if you have JavaScript, the map continues to work just fine without as well.
    • The open source FixMyStreet codebase has been upgraded to make it easier to translate into other languages, easier to use different mapping with, and easier to install. These efforts will continue, as we realise this has been one reason why others have made their own versions.
    • All this has been done without forking, so various major upgrades we have planned for the UK version will be exportable later in the year.

    NUUG’s Fiksgatami is the epitome of what makes civic open source at its best so unmatchably good. It was developed incredibly quickly: just a month to create what is effectively a fully fledged, best-of-breed nationwide e-government service – albeit an unofficial one. Thanks to the hard work of the public servants who fix problem reports, it will make small but meaningful improvements to the lives of a lot of people in Norway.  And it has made the free FixMyStreet codebase better and easier for other people to use to help them do the same thing in other countries.

    I know that at mySociety we are all looking forward to working with NUUG again. And I hope that this story inspires others to look at our code, and to work with mySociety to make FixMyStreet a service that can help everyone who would benefit from it.

    * We’ll be rolling out updated mapping (including OSM) and more in the UK, soon.

  3. Mapping points and postcodes to areas

    I’m very pleased to announce that mySociety’s upgraded point and postcode lookup service, MaPit, is public and available to all. It can tell you about administrative areas, such as councils, Welsh Assembly constituencies, or civil parishes, by various different lookups including name, point, or postcode. It has a number of features not available elsewhere as far as I know, including:

    • Full Northern Ireland coverage – we found a free and open dataset from the Office of National Statistics, called NSPD Open, available for a £200 data supply charge. We’ve paid that and uploaded it to our data mirror under the terms of the licence, so you don’t have to pay – if you feel like contributing to the charity that runs mySociety to cover our costs in this, please donate! 🙂
    • Actual boundaries – for any specific area, you can get the co-ordinates of the boundary in either KML, JSON, or WKT – be warned, some can be rather big!
    • Point lookup – given a point, in any geometry PostGIS knows about, it can tell you about all the areas containing that point, from parish and ward up to European electoral region.
    • History – large scale boundary changes will be stored as new areas; as of now, this means the site contains the Westminster constituency boundaries from both before and after the 2010 general election, queryable just like current areas.

    If you wish to use our service commercially or are considering high-volume usage, please get in touch to discuss options; the data and source code are available under their respective licences from the site. I hope this service may prove useful – we will slowly be migrating our own sites to use this service (FixMyStreet has already been done and already seems a bit nippier), so it should hopefully be quite reliable.

    Thanks must go to the bodies releasing this open data that we can build upon and provide these useful services, and everyone involved in working towards the release of the data. Thanks also to everyone behind GeoDjango and PostGIS, making working with polygons and shapefiles a much nicer experience than it was back in 2004.

  4. Outlook attachments now viewable in WhatDoTheyKnow

    When a bit of government forwards or attaches emails using Outlook, they get sent using a special, strange Microsoft email format. Up until now, WhatDoTheyKnow couldn’t decode it. You’d just see a weird attachment on the response to your Freedom of Information request, and probably not be able to do anything with it.

    Peter Collingbourne got fed up with this, and luckily for us, he can code too. He forked our source code repository, and made a nice patch in his own copy of it.

    He then told us about it, and I merged his changes into the main WhatDoTheyKnow code, tested them out on my laptop, then made them live. It all work perfectly first time. Peter even added the new dependency on vpim to WhatDoTheyKnow conf/packages.

    Now if you go to an Outlook attachment on WhatDoTheyKnow,such as this one you’ll just see the files, and be able to download them, and view them as HTML as normal. They’ll also get indexed by the search (although I need to do a rebuild for that for it to work with old requests).

    Thanks Peter!

    If you want to have a go making an improvement to a mySociety site, you can get the code for most of them from our github repositories. For some sites, there’s an INSTALL.txt file explaining how to get a development environment set up. Let us know if you do anything – even incremental improvements to installation instructions are really useful. And new, useful, features like Peter’s are even more so.

  5. ScenicOrNot raw data now available for re-use

    Scenic Or Not

    Matthew’s just updated ScenicOrNot, the little game that we built to provide a ‘Scenicness’ dataset for Mapumental, to include a data dump of the raw data. The dump will update automatically on a weekly basis, but currently it contains averaged scores for 181,188 1*1km grid squares, representing 83% of the Geograph dataset we were using, or 74% of all the grid squares in Great Britain. It is, in other words, really pretty good, and, I think, unprecedented in coverage as a piece of crowd sourced geodata about a whole country.

    It’s available under the Creative Commons Attribution Noncommercial 3 Licence, and we greatly look forward to seeing what people do with it.

  6. FixMyStreet RSS

    FixMyStreet has a lot of RSS feeds. There’s one for every one-tier council (170), one for every ward of every one-tier council (another 5,044), two for every two-tier (county and district) council (544), and two for every ward of every two-tier council (20,296) – two per two-tier council because you might want either problems reported to one council of a two-tier set-up in particular, or all reports within the council’s boundary.

    Then there’s an RSS feed every 162m across Great Britain in a big grid, returning all reports within a radius of that point, the radius by default being automatically determined by that point’s population density, but customisable to any distance if preferred. That’s, at a very rough approximation assuming Great Britain is a rectangle around its extremities, which it’s not, 19 million RSS feeds, lots of which will admittedly be very similar. 🙂

    Every single one of those feeds can be subscribed to by email instead if that’s preferable to you, and are all accessible through a simple interface at http://www.fixmystreet.com/alert.

    However, none of these RSS feeds was suitable for the person who emailed from a Neighbourhood Watch site and said that all they had was a postcode and they wanted to display a feed of reports from FixMyStreet. Given you could obviously look up a FixMyStreet map by postcode, it did seem odd that I hadn’t used the same code for the RSS feeds. Shortly thereafter, this anomaly was fixed, and if you now go to a URL of the form http://www.fixmystreet.com/rss/pc/postcode you will be redirected to the appropriate local reports feed for that postcode (I could say that adds another 1.7 million RSS feeds to our lot, but given they’re only redirects, that’s not strictly true). And after a couple more emails, I also added pubDate fields to the feeds which should make displaying in date order easier.

    It’s great to see our RSS feeds being used by other sites – other examples I’ve recently come across include Brent Council integrating FixMyStreet into their mapping portal (select Streets, then FixMyStreet), or the Albert Square and St Stephen’s Association listing the most recent Stockwell problems in their blog sidebar. If you’ve seen any notable examples, do leave them in the comments.

  7. So you want to start an organisation like mySociety? Some tips for aspirants.

    One of the nicest things about being involved with mySociety is seeing people in other countries starting similar organisations and building similar websites. After a Skype conversation with another eager hopeful last night, I thought I’d blog a bit about the things I think are most important to know if you’re just starting up. Here goes.

    1. Absolutely the most totally essential thing is to be an organisation of amazing, politically minded coders, not an organization employing or contracting good coders*. Their skills are your lifeblood, their ideas your bread and butter, and finding the best civic hackers in your country and building your organization around them is the only path to success. And that means they should be making most of the day to day decisions, not you, you ignorant, arts-degree-clutching clot.

    2.Ask the public what they think you should build. Not only will that give you access to more ideas than you have yourselves, it’ll engage people with you. Also, it’ll help you focus on the vital business of building sites that users want, not that YOU want.

    3. Keep your cost base low, and put all the money you have into looking after your core staff and being nice to volunteers. Work on building a community of volunteers, even if most of them are really just friends rather than people putting in lots of time. Avoid renting offices, avoid non-essential non-coder staff, get people to donate serving infrastructure and bandwidth. Because building and running democratic websites is a fundamentally new area of human endeavor (not like blogging which at least has an analogy in journalism) there are basically no pre-existing funding streams for the type of work you’re about to do. You will have to create the buzz around yourselves that will lead to people wanting to fund you, and it will probably take years, if you get there at all (mySociety hasn’t quite done this yet, even at 5).

    4. Ensure that the core of what you build can struggle on by even if your whole organization collapses. That means being open source, putting energy into sites that are as automated as possible, and making people excited about being volunteers.

    5. If you aren’t pissing off at least some people all the time, you’ve probably been captured by the establishment.

    6. Take whatever your first website plan is and remove 90% of the features you want. Then build it and launch it and your users will tell you which features they actually wanted instead. Build them and bask in the warm glow of appreciation.

    *yes, yes I know I can’t code for toffee, and I’ve got an arts degree, but I’m still a geek, honest. (Proudest moment, working out that a batch of PCI network cards were unreliable because they’d come from the factory flashed with the EPROMs for the wrong hardware, and fixed it.)

  8. Join us at OpenTech on 5th July

    OpenTech on 5th July is an informal one-day conference about technology, society and low-carbon living, featuring Open Source ways of working and technologies that anyone can have a go at.

    mySociety related sessions:

    • mySociety session – more on time travel maps, and the launch of WhatDoTheyKnow.com, and a few other surprise announcements
    • No2ID and Open Rights Group: State of the Nation
    • Here’s the UK EFF
    • Power to the People – One year one from the Power of Information Report
    • …and much more

    (more…)

  9. TheyWorkForYou video – the Flash player

    TheyWorkForYou video timestamping has been launched, over 40% of available speeches have already been timestamped, and (hopefully) all major bugs have been fixed, so I can now take a short breather and write this short series of more technical posts, looking at how the front end bits I wrote work and hang together.

    Let’s start with the most obvious feature of video timestamping – the video player itself. 🙂 mySociety is an open-source shop, so it was great to discover that (nearly all of) Adobe Flex is available under the Mozilla Public Licence. This meant I could simply download the compiler and libraries, write some code and compile it into a working SWF Flash file without any worries (and you can do the same!).

    Writing a Flex program is split into three main areas – MXML that lays out your application, defines any web services you’re using and so on; CSS to define the style of the various components; and ActionScript to deal with things like events, or talking to the JavaScript in the parent HTML. My code is probably quite shoddy in a number of places – it’s my first application in Flex 🙂 – but it’s all available to view if you want to take a peek, and it’s obviously running on the live TheyWorkForYou site.

    To put a video component in the player is no harder than including an <mx:VideoDisplay> element – set the source of that, and you have yourself a video player, no worrying about stream type, bandwidth detection, or anything else. 🙂 You can then use a very useful feature called data binding to make lots of things trivial – for example, I simply set the value of a horizontal slider to be the current playing time of the video, and the slider is then automatically in the right place at all times. On the downside, VideoDisplay does appear to have a number of minor bugs (the most obvious one being where seeking can cause the video to become unresponsive and you have to refresh the page; it’s more than possible it’s a bug in my code, of course, but there are a couple of related bugs in Adobe’s bug tracker).

    As well as the buttons, sliders and the video itself, the current MXML contains two fades (one to fade in the hover controls, one to fade them out), one time formatter (to format the display of the running time and duration), and three web services (to submit a timestamp result, delete a mistaken timestamp, and fetch an array of all existing timestamps for the current debate). These are all called from various places within the ActionScript when certain events happen (e.g. the Now button or the Oops button is clicked).

    Compiling is a simple matter of running mxmlc on the mxml file, and out pops a SWF file. It’s all straightforward, although a bit awkward at first working again with a strongly-typed, compiled language after a long time with less strict ones 🙂 The documentation is good, but it can be hard to find – googling for [flex3 VideoDisplay] and the like has been quite common over the past few weeks.

    Tomorrow I will talk about moving around within the videos and some bugs thrown up there, and then how the front end communicates with the video in order to highlight the currently playing speech – for example, have a look at last week’s Prime Minister’s Questions.

    1. The Flash player
    2. Seeking
    3. Highlighting the current speech
  10. Here’s to a shorter commute

    This project became Mapumental. Please visit that site for details of our travel-time maps services.
    The work was funded and supported by the Department for Transport.

    See also: the main travel-time maps report.

    ——————

    Our newly released travel time maps are currently shooting round the internet. It was great fun making them, and you might like to have a go too – there are plenty of public datasets you could overlay on the same base maps, using the same flash app (source code). There are a few notes about how we made them on the page itself, and the associated real time page. For a far more interesting view of the development process, read Tom Carden from Stamen’s account.

    The most interesting blog post I’ve seen to come from this is Whitehall staff have no life by Simon Dickson, who was inspired by the maps to think about the destruction of social capital caused by commuting. “Whitehall staff on all but the highest salaries can’t expect to live anywhere near their work, and hence can’t expect to have any kind of a social (capital) life.”