Everyone at mySociety is quite bubbling with excitement at the news that we’re today officially launching FixMyTransport.com , mySociety’s first new core charitable website since WhatDoTheyKnow launched in 2008. We’ve never before launched a site that took so much work to build, or that contained so much data.
What is it for?
FixMyTransport has two goals – one in your face, and the other more subtle.
The first goal, as the site’s name suggests, is to help people get common public transport problems resolved. We’re talking broken ticket machines, gates that should be open and stations without stair-free access. We’ll help by dramatically lowering the barrier to working out who’s responsible, and getting a problem report sent to them – a task that would have been impossible without the help of volunteers who gathered a huge number of operator email addresses for us. Consequently the service works everywhere in Great Britain, our database has over 300,000 stops and routes for train, tube, tram, bus, coach and ferry.
The second goal – the subtle one – is to see if it is possible to use the internet to coax non-activist, non-political people into their first taste of micro-activism. Whilst the site intentionally doesn’t contain any language about campaigning or democracy, we encourage and provide tools to facilitate the gathering of supporters, the emailing of local media, the posting of photos of problems, and the general application of pressure where it is needed. We also make problem reports and correspondence between operators and users public, which we have frequently seen create positive pressure when used on sister sites FixMyStreet and WhatDoTheyKnow.
Who made it?
FixMyTransport was largely built by one remarkable coder – Louise Crow, who started as a volunteer and who is now one of our longest serving core developers. She spent 18 months coding the site almost entirely by herself, wrestling with truly tortuous data problems and collaborating with Birmingham’s fantastic SuperCool design to make it look lovely (you should hire them, they’re great). She also tolerated my ‘aspirational scattergun’ school of project management with remarkable good humour. She really is the king of transport coding.
Credit must also go to mySociety core dev Dave Whiteland, who made the Facebook integration work, despite not having an account himself!
Why is it dedicated to Angie Martin?
Angie Martin was a mySociety coder for an all-too-brief period before she succumbed to cancer at a devastatingly early age. We’re dedicating this site to her in remembrance of a great, self taught perl monger who should still be here.
We’ll be posting further blog posts about the development process, the data challenges, and the overall project philosophy. In the mean time, please keep arms and legs inside the carriage – FixMyTransport is just about to depart.
We had a complaint that FixMyStreet maps weren’t displaying on someone’s computer. We hadn’t had any other complaints, and we quickly narrowed it down to the fact that the person was on the internet using a tethered T-Mobile phone.
T-Mobile (and Orange, and quite possibly others) are injecting JavaScript and altering content served over their networks. Their reason for doing this, according to their websites (T-Mobile, Orange), is to compress images and video sent to your browser, so as to speed up your browsing. Seeing it in action, they also inline some CSS and JavaScript, though not all, and remove comments from external files.
However, their implementation breaks things. In this particular instance, the T-Mobile JavaScript comment stripper appears to be searching for “/*” and “*/” and removing everything inbetween. This might work in most cases; however in the jQuery library, we find a string containing “*/*”, and later down the file, another string containing “*/*”. T-Mobile remove everything between the things it thinks are comment markers, even though they’re actually contained within strings, causing the jQuery library to be invalid JavaScript and stopping anything using jQuery from running.
Their decision to inline lots of the CSS also seems a bit odd – sure, on a mobile this might be quicker, but even ignoring tethering nowadays plenty of mobiles have caches too and having the CSS download once and be cached would seem better than adding weight to every page download. But I’m sure they’ve studied their decision there, and it doesn’t make any difference to the actual browsing, as opposed to the comment removal.
To turn off this feature on your mobile phone or broadband, visit accelerator.t-mobile.co.uk or accelerator.orange.co.uk on your connection and pick the relevant option – if anyone knows of similar on other networks, do leave updates in the comments.
From a FixMyStreet point of view – whilst FixMyStreet functions just fine without JavaScript, I had made the (perhaps incorrect) decision to put the map inside a <noscript> element, to prevent a flash of map-oddity as the JavaScript map overlaid the non-JS one. However, this meant in this circumstance the map did not work, as JavaScript was enabled, but jQuery was unable to be loaded. I haven’t decided whether to change this behaviour yet; obviously it would help people in this situation as the map would still display and function as it does for all those without JavaScript, but for those with JavaScript it does look a bit jarring as the page loads. Any suggestions on a better approach welcome 
Thanks everyone who’s turned up so far for a drink and a chat at our monthly pub meets. The next one is on Wednesday the 17th, again at the Counting House pub. You are invited!
If you’d like to tweet about the night, or put photos on Instagram or Flickr, you can use the hashtag #mysocial.
7.30, Wednesday 17th August at the Counting House
50 Cornhill
London
EC3V 3PD
Map here. The nearest Underground stations are Monument and Bank.
Back in November 2006 we launched Number 10’s petitions website. We were pretty proud of the usability-centred site we built – we can still lay a pretty good claim to it being one of the biggest democracy sites (measured in terms of people transacting) that the world’s ever seen.
Over 12 million signatures had been added to petitions by the time the site was switched off after the 2010 general election. We were particularly proud of developing a system that was highly load-tolerant: we once survived over 20,000 people signing within a single hour, all whilst running on a pair of cheap little servers. That performance on so little hardware was down to the raw brilliance represented by a coding team made up of Francis Irving, Matthew Somerville, and the late, great Chris Lightfoot.
We’re also pleased that the popularity of the site led to the irresistible rise of the belief that the public should be able to petition the government via the internet. So even though our site was mothballed, Parliament and DirectGov have taken over the idea, and the commitment has been upped a notch, from ‘we’ll send a reply’ to ‘we’ll talk about it’. To be clear, we are not, nor have ever been a community interested in replacing representative democracy with direct democracy, but anything that can squeeze any drop of change from Parliament is worth a small celebration.
What’s most pleasing, though, is that we’ve been able to take the open source code built for Number 10, improve and expand upon it to develop a hosted petitions service for local councils around the country, or the rest of the world. And this is where we found the most important lesson for us: local petitions can be awesome, and despite the much smaller numbers of signatories involved, we’ve been more widely and frequently impressed by local petitions and responses than at the more glamorous national level. We’re particular fans of Hounslow Borough Council who have given positive and detailed feedback on all sorts of genuine local issues, as well as working hard to let local residents know that the service exists.
Just recently we launched a site to make it really easy to find local council petition websites, because there are hundreds hidden away (we built some; most are supplied by other vendors). If we could see anything result from today’s huge explosion of interest in online petitions, it would be that people might start to look local, and explore what petitions in their community could mean.
If you’ve visited our parliamentary site TheyWorkForYou.com, you’ll have noticed that on each MP’s page there is a short summary of his or her voting record on various key issues.
These issues have always been carefully chosen to give a simple but neutral top-line view of each MP’s voting activity. Judging by Twitter, they’re a fairly popular part of the site, too.
There’s way, way more tedious complexity behind producing these little summaries than you might think, and due to a lack of appropriately skilled people in our team over the last year we had let our vote analyses get a bit behind the times. If you’re really interested you can read about why authoring these things in such a scrupulously balanced way is so time consuming here.
We’re posting today to tell you that we have recruited a pair of excellent new part-time voting analysts, David and Ambreen, and they have recently produced the first of a new generation of voting summaries.
The first shows how each MP has voted on increasing the rate of VAT, and second on the recent changes to university tuition fees. We have also increased the number of votes which feed into the EU integration policy to bring it more up to date.
To see this new data, just pop along to TheyWorkForYou’s home page, stick in your postcode, and check out your own MPs’ page. Then, if you want to be made aware as soon as we’ve published the next analyses, please follow our new TheyWorkForYou Twitter account.
Lastly, I just want to say thank you to the vote analysts Ambreen and David, to senior developer Matthew and to uber-volunteer Richard Taylor for kicking this vital part of TheyWorkForYou back into top gear.
Image by European Parliament.
Last month’s pubmeet was fun, but it was rather crowded, so we’re moving venues. This month, our chosen pub is the Counting House, which is cavernous as well as rather beautiful. Please do join us for a drink, to ask questions or suggest new ideas – or just for a chat.
If you’d like to tweet about the night, or put photos on Instagram or Flickr, you can use the hashtag #mysocial to make sure that others find ‘em.
7.30, Wednesday 20th July at the Counting House
50 Cornhill
London
EC3V 3PD
Map here. The nearest Underground stations are Monument and Bank.
This post explains how various aspects of the new FixMyStreet maps work, including how we supply our own OS StreetView tile server and how the maps work without JavaScript.
Progressive enhancement
During our work on FiksGataMi (the Norwegian version of FixMyStreet) with NUUG, we factored out the map code (for the Perlmongers among you, it’s now using Module::Pluggable to pick the required map) as FiksGataMi was going to be using OpenStreetMap, and we had plans to improve our own mapping too. Moving to OpenLayers rather than continuing to use our own slippy map JavaScript dating from 2006 was an obvious decision for FiksGataMi (and then FixMyStreet), but FixMyStreet maps have always been usable without JavaScript, utilising the ancient HTML technology of image maps to provide the same functionality, and we wanted to maintain that level of universality with OpenLayers. Thankfully, this isn’t hard to do – simply outputting the relevant tiles and pins as part of the HTML, allowing latitude/longitude/zoom to be passed as query parameters, and a bit of maths to convert image map tile clicks to the actual latitude/longitude selected. So if you’re on a slow connection, or for whatever reason don’t get the OpenLayers JavaScript in some way, the maps on FixMyStreet should still work fine. I’m not really aware of many people who use OpenLayers that do this (or indeed any JavaScript mapping API), and I hope to encourage more to do so by this example.
Zooming
We investigated many different maps, and as I wrote in my previous blog post, we decided upon a combination of OS StreetView and Bing Maps’ OS layer as the best solution for the site. The specific OpenLayers code for this (which you can see in map-bing-ol.js is not complicated (as long as you don’t leave in superfluous commas breaking the site in IE6!) – overriding the getURL function and returning appropriate tile URLs based upon the zoom level. OpenLayers 2.11 (due out soon) will make using Bing tiles even easier, with its own seamless handling of them, as opposed to my slight bodge with regard to attribution (I’m displaying all the relevant copyright statements, rather than just the one for the appropriate location and zoom level which the new OpenLayers will do for you). I also had to tweak bits of the OpenLayers map initialisation so that I could restrict the zoom levels of the reporting map, something which again I believe is made easier in 2.11.
OpenStreetMap
Having pluggable maps makes it easy to change them if necessary – and it also means that for those who wish to use it, we can provide an OpenStreetMap version of FixMyStreet. This works by noticing the hostname and overriding the map class being asked for; everything necessary to the map handling is contained within the module, so the rest of the site can just carry on without realising anything is different.
OS StreetView tile server
Things started to get a bit tricky when it came to being ready for production. In development, I had been using http://os.openstreetmap.org/ (a service hosted on OpenStreetMap’s development server) as my StreetView tile server, but I did not feel that I could use it for the live site – OpenStreetMap rightly make no reliability claims for it, it has a few rendering issues, and we would probably be having quite a bit of traffic which was not really fair to pass on to the service. I wanted my own version that I had control over, but then had a sinking feeling that I’d have to wait a month for something to process all the OS TIFF files (each one a 5km square) into millions and millions of PNG tiles. But after many diversions and dead ends, and with thanks to a variety of helpful web pages and people (Andrew Larcombe’s guide to his similar install was helpful), I came up with the following working on-demand set-up, with no pre-seeding necessary, which I’m documenting in case it might be useful to someone else.
Requests come in to our tile server at tilma.mysociety.org, in standard OSM/Google tile URL format (e.g. http://tilma.mysociety.org/sv/16/32422/21504.png. Apache passes them on to TileCache, which is set up to cache as GoogleDisk (ie. in the same format as the URLs) and to pass on queries as WMS internally to MapServer using this layer:
[sv]
type=WMS
url=path/to/mapserv.fcgi?map=os.map&
layers=streetview
tms_type=google
spherical_mercator=true
MapServer is set up with a Shapefile (generated by gdaltindex) pointing at the OS source TIFF and TFW files, meaning it can map tile requests to the relevant bits of the TIFF files quickly and return the correct tile (view MapServer’s configuration – our tileserver is so old, this is still in CVS). The OUTPUTFORMAT section at the top is to make sure the tiles returned are anti-aliased (at one point, I thought I had a choice between waiting for tiles to be prerendered anti-aliased, or going live with working but jaggedy tiles – thankfully I persevered until it all worked ).
Other benefits of OpenLayers
As you drag the map around, you want the pins to update – the original OpenLayers code I wrote used the Markers layer to display the pins, which has the benefit of being simple, but doesn’t fit in with the more advanced OpenLayers concepts. Once this was switched to a Vector layer, it now has access to the BBOX strategy, which just needs a URL that can take in a bounding box and return the relevant data. I created a subclass of OpenLayers.Format.JSON, so that the server can return data for the left hand text columns, as well as the relevant pins for the map itself.
Lastly, using OpenLayers made adding KML overlays for wards trivial and made those pages of the site much nicer. The code for displaying an area from MaPit is as follows:
if ( fixmystreet.area ) {
var area = new OpenLayers.Layer.Vector("KML", {
strategies: [ new OpenLayers.Strategy.Fixed() ],
protocol: new OpenLayers.Protocol.HTTP({
url: "/mapit/area/" + fixmystreet.area + ".kml?simplify_tolerance=0.0001",
format: new OpenLayers.Format.KML()
})
});
fixmystreet.map.addLayer(area);
area.events.register('loadend', null, function(a,b,c) {
var bounds = area.getDataExtent();
if (bounds) { fixmystreet.map.zoomToExtent( bounds ); }
});
}
Note that also shows a new feature of MaPit – being able to ask for a simplified KML file, which will be smaller and quicker (though of course less accurate) than the full boundary.
Last week, FixMyStreet gained a number of new features that we hope you will find useful.
Firstly, we’ve thrown away our old maps and replaced them with new, shiny, zoomable maps. This should make it easier for people to find and report problems, especially in sparser locations. We’re using the OS StreetView layer (hosted internally) when zoomed in, reverting to Bing Maps’ Ordnance Survey layer when zoomed out, as we felt this provided the best combination for reporting problems. In urban areas, you can still see individual houses, whilst in more rural areas the map with footpaths and other such features is probably of more use. FixMyStreet tries to guess initially which map would be most appropriate based upon population density, meaning a search for Stroud looks a bit different from that for Birmingham.
OpenStreetMap fans, don’t worry – as part of our mapping technology upgrade, you can now use osm.fixmystreet.com to access your favourite mapping instead.
Secondly, we now have user accounts. We’ve rolled these out alongside our current system of email confirmation, and it’s up to you which you use when reporting a problem or leaving an update. This means that those who come to the site one time only to report a pothole can continue to do so quickly, but have the option of an account if they want. Having an account means you no longer have to confirm reports and updates by email, and you have access to a page listing all the reports you’ve made through FixMyStreet, and showing these reports on a (obviously new and shiny) summary map.
Other improvements include a much nicer All Reports section, so you can see all reports to Adur District Council on a map, paginated and with the boundary of the council marked – and individual wards of councils now each have their own pages too.
I’ll follow up this post with another, more technical, look at the maps and how they work, for anyone who’s interested
Local petitions can be highly effective, and we think that making them easier to create is in the public interest. Many councils have petitions facilities buried deep within their websites, most often, very deeply. In fact it brings to mind Douglas Adams’ quote about important council documents being “on display on the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’”.
Our most recent mini-project is an attempt to make it as easy as possible to find your local council’s e-petitioning site, if they have one. PetitionYourCouncil.com (you’ll notice we stuck to our tried and tested format for site names, there) is a way of finding every council e-petitioning website we know about.
Our original motivation for building the site was that we, along with other suppliers, have supplied online e-petitioning sites to numerous councils ourselves – it’s one of the ways in which we fund our charitable activities. Having delivered these sites, we later noticed that many of them are left under-used and in some cases, not used at all: only because people don’t know about them. We hate to think of councils spending money on a splendid resource that could be improving democratic processes for their citizens – and those citizens never knowing that they exist. In particular, we owe Dave Briggs thanks for pushing us into action with this blog post.
And yes, in case you’re wondering, PetitionYourCouncil links to every council petitions site, not just the ones we made.
The site was built by mySociety developer Edmund von der Burg using Django, jQuery, Google maps and Mapit, and like most mySociety projects, it’s open source. There’s a bit more detail on the About page. Please do try it out, and let us know what you think.
The Ministry of Justice have just published their latest quarterly statistics on the handling of Freedom of Information requests by central government bodies. We’ve crunched the numbers to compare them to the requests made using WhatDoTheyKnow.com
The graph shows our share of FOI requests sent to central Departments of State jumped to 14.6% in the 1st quarter of 2011.
This time round, the top 3 departments were:
- Home Office (which includes the UK Border Agency, CRB & Identity & Passport Service) – 254 requests out of 866 – 29%
- Department for Education – 81 requests out of 328 – 25%
- Department for Communities and Local Government – 59 requests out of 250 – 24%
Many of the WhatDoTheyKnow users contacting the Home Office & UK Border Agency are trying to find out information about their own immigration case. We regularly receive emails from applicants asking for help, as they have often been waiting months (or even years in some cases) for an official update to their case, often with the UKBA holding on to identity documents or passport. Applicants then feel they have to resort to making FOI requests. Many of these are auto-replied by this standard FAQ, and applicants don’t receive a personal answer. The large 29% share of all Home Office requests suggests that the normal contact methods to keep people updated aren’t working or even that their service is simply struggling with demand. It’s also likely that they don’t consider these types of requests as formal FOI requests, so it is worth noting that we are likely to be slightly overstating the percentage share figures.
Free schools were a popular topic for the Department of Education – 9 out of 81 requests were on this subject, and nearly all were refused on the basis that information would be published at some unspecified date in the future.
To understand the limitations of the data analysis, please see here.
One interesting trend that has been consistently seen is that FOI requests are more frequent in odd-numbered quarters compared to even ones – if you have any ideas why this may be the case, please add them to the comments!
To
| - Communities and Local Government |
