1. Which ward will you be in on May 5?

    Are you still in the same ward? Check whether your ward boundaries have changed here.

    May 5 is election day

    If you’re a UK citizen, you have an election in your near future. We can say that with confidence.

    May 5 sees elections not only for the Scottish Parliament, the National Assembly of Wales and the Northern Ireland Assembly, but also for many local councils. Londoners will be picking their London Assembly representatives and their Mayor. As if all that isn’t enough, there are also Police and Crime Commissioner Elections.

    Ward boundaries are changing

    You might think you already know where to vote, and who’s standing for election in your area.

    But both are dictated by which ward you live in — and that may not be the one you’re used to, thanks to ongoing changes in ward boundaries.

    There’s no need to worry, though. As before, we’ve provided a nifty tool that will tell you whether your ward has changed. Just enter your postcode here.

    Thanks to Democracy Club’s new project Who Can I Vote For? you’ll also be able to click through to see whatever information is available for candidates in your ward.

    Image: Matt Brown (CC)

  2. Technical look at the new FixMyStreet maps

    This post explains how various aspects of the new FixMyStreet maps work, including how we supply our own OS StreetView tile server and how the maps work without JavaScript.

    Progressive enhancement

    During our work on FiksGataMi (the Norwegian version of FixMyStreet) with NUUG, we factored out the map code (for the Perlmongers among you, it’s now using Module::Pluggable to pick the required map) as FiksGataMi was going to be using OpenStreetMap, and we had plans to improve our own mapping too. Moving to OpenLayers rather than continuing to use our own slippy map JavaScript dating from 2006 was an obvious decision for FiksGataMi (and then FixMyStreet), but FixMyStreet maps have always been usable without JavaScript, utilising the ancient HTML technology of image maps to provide the same functionality, and we wanted to maintain that level of universality with OpenLayers. Thankfully, this isn’t hard to do – simply outputting the relevant tiles and pins as part of the HTML, allowing latitude/longitude/zoom to be passed as query parameters, and a bit of maths to convert image map tile clicks to the actual latitude/longitude selected. So if you’re on a slow connection, or for whatever reason don’t get the OpenLayers JavaScript in some way, the maps on FixMyStreet should still work fine. I’m not really aware of many people who use OpenLayers that do this (or indeed any JavaScript mapping API), and I hope to encourage more to do so by this example.

    Zooming

    We investigated many different maps, and as I wrote in my previous blog post, we decided upon a combination of OS StreetView and Bing Maps’ OS layer as the best solution for the site. The specific OpenLayers code for this (which you can see in map-bing-ol.js is not complicated (as long as you don’t leave in superfluous commas breaking the site in IE6!) – overriding the getURL function and returning appropriate tile URLs based upon the zoom level. OpenLayers 2.11 (due out soon) will make using Bing tiles even easier, with its own seamless handling of them, as opposed to my slight bodge with regard to attribution (I’m displaying all the relevant copyright statements, rather than just the one for the appropriate location and zoom level which the new OpenLayers will do for you). I also had to tweak bits of the OpenLayers map initialisation so that I could restrict the zoom levels of the reporting map, something which again I believe is made easier in 2.11.

    OpenStreetMap

    Having pluggable maps makes it easy to change them if necessary – and it also means that for those who wish to use it, we can provide an OpenStreetMap version of FixMyStreet. This works by noticing the hostname and overriding the map class being asked for; everything necessary to the map handling is contained within the module, so the rest of the site can just carry on without realising anything is different.

    OS StreetView tile server

    Things started to get a bit tricky when it came to being ready for production. In development, I had been using http://os.openstreetmap.org/ (a service hosted on OpenStreetMap’s development server) as my StreetView tile server, but I did not feel that I could use it for the live site – OpenStreetMap rightly make no reliability claims for it, it has a few rendering issues, and we would probably be having quite a bit of traffic which was not really fair to pass on to the service. I wanted my own version that I had control over, but then had a sinking feeling that I’d have to wait a month for something to process all the OS TIFF files (each one a 5km square) into millions and millions of PNG tiles. But after many diversions and dead ends, and with thanks to a variety of helpful web pages and people (Andrew Larcombe’s guide [dead link removed] to his similar install was helpful), I came up with the following working on-demand set-up, with no pre-seeding necessary, which I’m documenting in case it might be useful to someone else.

    Requests come in to our tile server at tilma.mysociety.org, in standard OSM/Google tile URL format (e.g. http://tilma.mysociety.org/sv/16/32422/21504.png. Apache passes them on to TileCache, which is set up to cache as GoogleDisk (ie. in the same format as the URLs) and to pass on queries as WMS internally to MapServer using this layer:

    [sv]
    type=WMS
    url=path/to/mapserv.fcgi?map=os.map&
    layers=streetview
    tms_type=google
    spherical_mercator=true

    MapServer is set up with a Shapefile (generated by gdaltindex) pointing at the OS source TIFF and TFW files, meaning it can map tile requests to the relevant bits of the TIFF files quickly and return the correct tile (view MapServer’s configuration* – our tileserver is so old, this is still in CVS). The OUTPUTFORMAT section at the top is to make sure the tiles returned are anti-aliased (at one point, I thought I had a choice between waiting for tiles to be prerendered anti-aliased, or going live with working but jaggedy tiles – thankfully I persevered until it all worked 🙂 ).

    Other benefits of OpenLayers

    As you drag the map around, you want the pins to update – the original OpenLayers code I wrote used the Markers layer to display the pins, which has the benefit of being simple, but doesn’t fit in with the more advanced OpenLayers concepts. Once this was switched to a Vector layer, it now has access to the BBOX strategy, which just needs a URL that can take in a bounding box and return the relevant data. I created a subclass of OpenLayers.Format.JSON, so that the server can return data for the left hand text columns, as well as the relevant pins for the map itself.

    Lastly, using OpenLayers made adding KML overlays for wards trivial and made those pages of the site much nicer. The code for displaying an area from MaPit is as follows:

        if ( fixmystreet.area ) {
            var area = new OpenLayers.Layer.Vector("KML", {
                strategies: [ new OpenLayers.Strategy.Fixed() ],
                protocol: new OpenLayers.Protocol.HTTP({
                    url: "/mapit/area/" + fixmystreet.area + ".kml?simplify_tolerance=0.0001",
                    format: new OpenLayers.Format.KML()
                })
            });
            fixmystreet.map.addLayer(area);
            area.events.register('loadend', null, function(a,b,c) {
                var bounds = area.getDataExtent();
                if (bounds) { fixmystreet.map.zoomToExtent( bounds ); }
            });
        }

    Note that also shows a new feature of MaPit – being able to ask for a simplified KML file, which will be smaller and quicker (though of course less accurate) than the full boundary.

     

    *Broken link removed, Nov 2014

  3. Zoomable maps and user accounts on FixMyStreet

    Last week, FixMyStreet gained a number of new features that we hope you will find useful.

    Birmingham on FixMyStreetStroud on FixMyStreetFirstly, we’ve thrown away our old maps and replaced them with new, shiny, zoomable maps. This should make it easier for people to find and report problems, especially in sparser locations. We’re using the OS StreetView layer (hosted internally) when zoomed in, reverting to Bing Maps’ Ordnance Survey layer when zoomed out, as we felt this provided the best combination for reporting problems. In urban areas, you can still see individual houses, whilst in more rural areas the map with footpaths and other such features is probably of more use. FixMyStreet tries to guess initially which map would be most appropriate based upon population density, meaning a search for Stroud looks a bit different from that for Birmingham.

    FixMyStreet with OpenStreetMap

    OpenStreetMap fans, don’t worry – as part of our mapping technology upgrade, you can now use osm.fixmystreet.com to access your favourite mapping instead.

    Secondly, we now have user accounts. We’ve rolled these out alongside our current system of email confirmation, and it’s up to you which you use when reporting a problem or leaving an update. This means that those who come to the site one time only to report a pothole can continue to do so quickly, but have the option of an account if they want. Having an account means you no longer have to confirm reports and updates by email, and you have access to a page listing all the reports you’ve made through FixMyStreet, and showing these reports on a (obviously new and shiny) summary map.

    Adur District Council reports

    Other improvements include a much nicer All Reports section, so you can see all reports to Adur District Council on a map, paginated and with the boundary of the council marked – and individual wards of councils now each have their own pages too.

    I’ll follow up this post with another, more technical, look at the maps and how they work, for anyone who’s interested 🙂

  4. Geovation funding for FixMyTransport

    FixMyTransport...anywhere

    On Wednesday this week, mySociety’s Tom and Paul were in Southampton, competing in the Geovation finals.

    Geovation is an initiative coordinated by Ordnance Survey which gives out funding to projects that help “communities address their unmet needs through the application of geographic data, skills and expertise”. When we discovered that the theme this time was “How can we improve transport in Britain?” we knew we had to enter.

    As many of you will know, mySociety has been working for some time on FixMyTransport, a project for reporting problems with public transport. Taking much of what we’ve learned from FixMyStreet, we are, in the trademark mySociety way, building a website that will make the process easy, whilst hiding all the complexities out of sight.

    FixMyTransport is well under way, and we’re hoping to launch shortly. But with Geovation funding, we hoped to be able to roll out an accompanying mobile application.

    This is incredibly important because, after all, the best time to make a transport report is immediately you experience the problem.

    mySociety has, of course, always been into maps and geodata – we use them in what we hope are fun and innovative ways across many of our sites, including (obviously) Mapumental, and (less obviously) TheyWorkForYou and WriteToThem. We’re also rather fond of public transport.

    We also really enjoyed meeting the other contestants, particularly Cyclestreets whose project looks like it will be one to watch.

    At the end of the day, we were delighted to learn that we had been awarded £27,000 to develop a simple, intuitive, cross-platform mobile application for FixMyTransport. We can’t wait to get started. We really believe it’s going to be of real benefit to public transport users across the UK (and possibly further, given the open-source nature of all our work).

    If you’d like to stay up to date with FixMyTransport as we build and launch it, you might want to be one of the very first to “like” our Facebook page or follow us on Twitter.

  5. New features on MaPit

    We’ve added a variety of new features to our postcode and point administrative area database, MaPit, in the past month – new data (Super Output Areas and Crown dependency postcodes), new functionality (more geographic functions, council shortcuts, and JSONP callback), and most interestingly for most people, a way of browsing all the data on the site.

    • Firstly, we have some new geographic functions to join touches – overlaps, covered, covers, and coverlaps. These do as you would expect, enabling you to see the areas that overlap, cover, or are covered by a particular area, optionally restricted to particular types of area. ‘coverlaps’ returns the areas either overlapped or covered by a chosen area – this might be useful for questions such as “Tell me all the Parliamentary constituencies fully or partly within the boundary of Manchester City Council” (three of those are entirely covered by the council, and two overlap another council, Salford or Trafford).
    • As you can see from that link, nearly everything on MaPit now has an HTML representation – just stick “.html” on the end of a JSON URI to see it. This makes it very easy to explore the data contained within MaPit, linking areas together and letting you view any area on Google Maps (e.g. Rutland Council on a map). It also means every postcode has a page.
    • From a discussion on our mailing list started by Paul Waring, we discovered that the NSPD – already used by us for Northern Ireland postcodes – also contains Crown dependency postcodes (the Channel Islands and the Isle of Man) – no location information is included, but it does mean that given something that looks like a Crown dependency postcode, we can now at least tell you if it’s a valid postcode or not for those areas.
    • Next, we now have all Lower and Middle Super Output Areas in the system; thanks go to our volunteer Anna for getting the CD and writing the import script. These are provided by ONS for small area statistics after the 2001 census, and it’s great that you can now trivially look up the SOA for a postcode, or see what SOAs are within a particular ward. Two areas are in MaPit for each LSOA and MSOA – one has a less accurate boundary than the other for quicker plotting, and we thought we might as well just load it all in. The licences on the CD (Conditions of supply of SOA boundaries and Ordnance Survey Output Area Licence) talk about a click-use licence, and a not very sraightforward OS licence covering only those SOAs that might share part of a boundary with Boundary-Line (whichever ones those are), but ONS now use the Open Government Licence, Boundary-Line is included in OS OpenData, various councils have published their SOAs as open data (e.g. Warwickshire), and these areas should be publicly available under the same licences.
    • As the UK has a variety of different types of council, depending on where exactly you are, the postcode lookup now includes a shortcuts dictionary in its result, with two keys, “council” and “ward”. In one-tier areas, the values will simply by the IDs of that postcode’s council and ward (whether it’s a Metropolitan district, Unitary authority, London borough, or whatever); in two-tier areas, the values will again be dictionaries with keys “district” and “council”, pointing at the respective IDs. This should hopefully make lookups of councils easier.
    • Lastly, to enable use directly on other sites with JavaScript, MaPit now sends out an “Access-Control-Allow-Origin: *” header, and allows you to specify a JSON callback with a callback parameter (e.g. put “?callback=foo” at the end of your query to have the JSON results wrapped in a call to the foo() function). JSONP calls will always return a 200 response, to enable the JavaScript to access the contents – look for the “error” key to see if something went wrong.

    Phew! I hope you find this a useful resource for getting at administrative geographic data; please do let us know of any uses you make of the site.

  6. He was a man, take him for all in all

    Chris Lightoot died a year ago today (or yesterday, by a few minutes).

    I’m just sitting here reading the very first emails I ever got from him, back in 2003. Within the first few mails he’s invented and hacked up the idea that is now Richard Pope’s PlanningAlerts.com, coded and developed the idea that persuaded YouGov to donate vast amounts of free polling data to form PoliticalSurvey2005.com (a wider understanding of which would greatly help in the US election if the methodology was only applied there) and in this post he’s foreseen the Google maps mashup craze and offered it on a plate to the Ordnance Survey to pioneer, two years before Google started.

    The invention and brilliance comes so thick and fast reading these mails that I now realise that I’d persuaded myself over the year that I’d mis-remembered quite how insanely creative he was, trying to correct for rose-tinted lenses. But he was a proper, bona fide, no-holds-barred cantankerous genius. Most days I think about Chris at least once: I try to make sure we live up to his standards (he wouldn’t have tolerated my use of ‘But’ at the start of the last sentence, for example). Reading these mails tonight drives home the scale of what we all lost, amongst our friends, on the Internet and in society at large. It aches to contemplate.

  7. Game, Set, and Matched

    The last batch of councillor data arrived this morning, thanks very much to GovEval, so pratically every council (bar all Scottish councils and the 17 English councils that had boundary changes, for which we’re just awaiting a new version of Boundary-Line, Ordnance Survey’s product that says where constituency boundaries are) should now be contactable again through WriteToThem.

    I’ve been doing some work on helping people promote our sites and the things on them – spurred by a request from a user who was holding a street party, we’ve made some posters and flyers for FixMyStreet (thanks to volunteer Ayesha Garrett for designing them), and we’ve started providing online tools to promote pledges on PledgeBank, including an up-to-date status image or text of a particular pledge, alongside the established, more offline, flyers.

  8. Neighbourhood Fix-It passes 1000th report

    Just a week after doing the proper press launch Neighbourhood Fix-It has passed its 1000th confirmed problem report.

    The next two features to add, in response to demands from users and councils are:

    1. The integration of the Ordnance Survey’s Address-Point dataset, so we can give councils a nearest street address, not just a map.
    2. A facility so that councils can enter and update different contact email addresses for different sorts of problem, if the central email address we use at the moment isn’t what they want. We’ll base the categories on the government standards for these things.

    Matthew is doing these, so they’ll probably be finished before I finish this post 🙂

  9. Ordnance Survey mashups day

    Last Friday Matthew and I went to the Ordnance Survey’s UK Geospatial Mash-up day. And a splendid time was had by all. Really this post is just a placeholder for a link to a copy of my presentation slides (not quite what I delivered, I’m afraid), but if anyone was there and has any questions they weren’t able to put to us in person, or wasn’t there and wished they had been, then the comments section awaits….

  10. How much is that map in the window?

    A charity wrote to us shortly after we published our time travel maps asking whether we could make such a map for them. We were happy to oblige – it is nice to know that there are real rather than theoretical users for work like this.

    Obviously, we knew we couldn’t do this work without getting the data licenced for commercial use. So I called the Ordnance Survey, left a query, and after a couple of clarifications here’s what we got back:

    ———- Forwarded message ———-
    From: PGA Helpdesk
    Date: 08-Jun-2006 17:15
    Subject: RE: mySociety mapping
    To: Tom Steinberg

    Tom,
    Sorry for the delay. Prices would be…

    25k – 16 tiles = £301.12
    Code Point – £731.59
    250k Raster – £106.22 for England cover or £183.22 for UK cover
    (please ignore what I put below about 250k only being available in uk
    cover)

    All prices for 1 user, 1 year licence and exclude VAT. Terms of data
    use are internal business use, display and promotion as long as there
    is no financial gain.

    Hope this is okay.
    ——–

    So fact fans, that’s between £837.81 and £1032.71 to produce a single map (depending on scale) as long as the user doesn’t publish it or show it to anyone else.