A charity wrote to us shortly after we published our time travel maps asking whether we could make such a map for them. We were happy to oblige – it is nice to know that there are real rather than theoretical users for work like this.

Obviously, we knew we couldn’t do this work without getting the data licenced for commercial use. So I called the Ordnance Survey, left a query, and after a couple of clarifications here’s what we got back:

———- Forwarded message ———-
From: PGA Helpdesk
Date: 08-Jun-2006 17:15
Subject: RE: mySociety mapping
To: Tom Steinberg

Tom,
Sorry for the delay. Prices would be…

25k – 16 tiles = £301.12
Code Point – £731.59
250k Raster – £106.22 for England cover or £183.22 for UK cover
(please ignore what I put below about 250k only being available in uk
cover)

All prices for 1 user, 1 year licence and exclude VAT. Terms of data
use are internal business use, display and promotion as long as there
is no financial gain.

Hope this is okay.
——–

So fact fans, that’s between £837.81 and £1032.71 to produce a single map (depending on scale) as long as the user doesn’t publish it or show it to anyone else.

mySociety developer Chris has made some maps. I think you will like them.

So, a silly post for today: Postcodeine. This is a British version of Ben Fry’s zipdecode, a “tool” for visualising the distribution of zipcodes in the United States. This is, as has been pointed out to me, wholly pointless, but it’s quite fun and writing it was an interesting exercise (it also taught me a little bit about AJAX, the web’s technology trend du jour). If you want the source code, it’s at the foot here; licence is the Affero GPL, as for all the other mySociety code.

How it works: this is pretty obvious, but I might as well spell it out. The web page has four images on it: the big and small base maps, and two overlays. The back-end code is responsible for drawing sets of postcode locations into transparent PNGs, and when you type things in the text field, the src for each of the overlay images is changed. Panning the large map is done by issuing another request from Javascript to grab the mean location of all postcodes matching the given prefix (slightly hobbled, so that this isn’t a generalised postcode-to-coordinates oracle — sorry!); the rightmost pane, with a list of postcodes and their areas, is populated from another HTTP request. It could be done with an iframe but, as Paul Graham puts it, “Javascript works now”, so we might as well use that.

(I should say, by the way, that I wrote this in my copious spare time. It’s copyright mySociety because I don’t have the right to use the postcode database myself.)

A very quick post to announce the launch of a public interface to our Gaze web gazetteer service. The motivation behind Gaze is collecting location information from users without using maps (a clunky approach with poor accessibility and licensing problems) or postcodes (which do not have universal coverage and have privacy issues as well as licensing problems). Instead the idea is to use place names to identify locations, even in the presence of ambiguity, alternate names, etc. We do this by providing a search service over a large gazetteer (2.2 million places and 3 million names), and supplying additional contextual information to disambiguate common place names. The API is very simple, with one major function and two other supporting ones.

Anyway, without further ado, here is the API. Internally we use one based on RABX, but we’ve done a special “RESTful” API for everyone else. All requests should be HTTP GETs; all parameters must be in UTF-8; and all responses are in UTF-8 plain text or comma-separated values. All calls should be passed to the URL,

http://gaze.mysociety.org/gaze-rest

selecting a particular function by specifying the HTTP parameter f, for instance

http://gaze.mysociety.org/gaze-rest?f=get_find_places_countries

Available functions are:

get_country_from_ip

Parameters:

ip

IPv4 address of a host, in dotted-quad format

Guess the country of location of a host from its IP address. The result of this call will be an ISO country code, followed by a line feed; or, if it was not possible to determine a country, a line feed on its own.

get_find_places_countries

No parameters.Return the list of countries for which the find_places call has a gazetteer available. The list is returned as a list of ISO country codes followed by line feeds.

find_places

Parameters:

country

ISO country code of country in which to search for places

state

state in which to search for places; presently this is only meaningful for country=US (United States), in which case it should be a conventional two-letter state code (AZ, CA, NY etc.); optional

query

query term input by the user; must be at least two characters long

maxresults

largest number of results to return, from 1 to 100 inclusive; optional; default 10

minscore

minimum match score of returned results, from 1 to 100 inclusive; optional; default 0

Returns in CSV format (as defined by this internet draft) with a one-line header a list of the following fields:

name

name of the place described by this row

in-qualifier

blank, or the name of an administrative region in which this place lies (for instance, a county)

near-qualifier

blank, or a list of nearby places, separated by commas

latitude

WGS-84 latitude of place in decimal degrees, north-positive

longitude

WGS-84 longitude of place in decimal degrees, east-positive

state

blank, or containing state code for US

score

match score for this place, from 0 to 100 inclusive

Enjoy! Questions and comments to hello@mysociety.org, please.

Update: we’ve now added the facilities for discovering population densities and “customary proximity” (as discussed in this post) to Gaze. The additional APIs are documented here.

mySociety yesterday launched a pair of Back o’ The Envelope projects based on Google Maps.

Placeopedia.com — Connect Wikipedia articles with the places they represent

YourHistoryHere.com — Share local and geographic history and trivia.

There are a few things to say about both projects:

1 – As is normal with mySociety projects the code for these projects (excepting Google maps) is open source. We hope that by providing a ready-made annotation system, people will find it easier to make their own publicly-authored layers of information.

2 – Both sites syndicate their data under open source licenses, and in a location-queryable fashion. This is really important, as it allows for all types of nice local history to be syndicated to tourism sites, local community discussion boards, blogs and so on.

3 – We’re calling them ‘Back o’ the Envelope’ to contrast them to the big, polished and time consuming projects we run like PledgeBank.com and WriteToThem.com.

Lots of countries gradually loading into one of our servers. There’s 220Mb of data comprising 227 countries, with about 5,000,000 places altogether. With a global population of about 6 billion, that means the average “place” has 1,200 people living in it. For each place we have the latitude and longitude. (All this data comes from the US military)

Try it out by signing up for a local alert in any country. Let us know if you find any bugs, or have any problems or suggestions to make. Also, if you want access to this gazetteer as a web service, send us a mail.

Currently it’s up to Uruguay, it’ll be a bit longer before we’ve finished the alphabet. It takes quite a while partly because of the volume of data and indices being built, partly because for places with the same name as each other it hunts for nearby towns to disambiguate, and partly because we didn’t optimise the perl script. It won’t run very often.

Well, I’m back from my holiday, suitably sunburned and (relatively) relaxed. As Francis mentions, I was off in the Mediterranean somewhere (Majorca, specifically) suffering from miserable internet withdrawal symptoms. I did manage to get IRC up-and-running over dialup for election night, though this turned out to be surprisingly expensive. For once I was grateful to my iBook, which did actually Just Work when plugged into the wall.

Anyway, today’s job is sorting out the new Scottish constituency boundaries. Scotland’s Parliament was dissolved in 1707 on the passing of the Act of Union, to be reconstituted in 1999. The quid pro quo for the Scots was enhanced representation in the House of Commons; Scottish constituencies had, in 1998, an average of 55,000 electors, compared to 69,000 in England. This anomaly has now been corrected, reducing the number of constituencies in Scotland from 72 to 59; all but three of the latter have different boundaries.

This means updating MaPit, the component we built to map postcodes into electoral geography, to deal with the new boundaries. Ideally the way that we’d do this is to wait for Ordnance Survey to ship us, via our friends in ODPM, the new revision of their Boundary-Line (TM, apparently) product, with the outlines of the new constituencies encoded in attractive machine-readable form, and feed it to our existing import scripts. (As so often in life, it’s not quite that simple, but you get the general idea.) In an ideal world, this would also contain all the changed boundaries of the English counties and their constituent county electoral divisions.

However, this is not an ideal world, and though there is a new revision of Boundary-Line in the works, it hasn’t come out yet, so we have to construct the point-to-constituency mapping in some other way. Happily, at this stage of the boundary revision process, the constituency boundaries are coterminous with ward boundaries, so it’s possible to just lift the definitions of the new constituencies from the relevant Statutory Instrument and fix up the constituencies from the ward boundaries, which haven’t changed. This, sadly, has occasioned a bit of a hack to our code, because we generally don’t assume that electoral geography is hierarchically defined — because it isn’t.

(I don’t feel too bad about committing this hack, actually, because we’re likely to chuck the whole MaPit database and reconstruct it later in the year from OS data. When we built it originally, we did so from data in ESRI shapefile format; unfortunately, OS stuffed up the process of generating this from their own, internal and quite bonkers, NTF format, so the various area ID numbers in the database are not unique and not expected to be stable. We’d rather like stable ID numbers, so that we can cope gracefully with revisions to geography while maintaining continuity of, for instance, statistical data about MPs, so next time round we’re going to work from the NTF instead.)

Sadly this Scottish hack doesn’t get us anywhere with the new county boundaries, and OS have told us that not all of the updated counties will be included in the forthcoming Boundary-Line revision. So it’ll be back to the tedious conversion of statutory instruments into SQL at some point in the near future, except that we’ll probably have to start building things up from parishes, rather than wards. Expect more anguished posts on this in the future.

Meanwhile, Francis and Tom are collecting names and contact details for the new MPs. Tom tells me that this intake looks much more tech-savvy than the last, which could be good news from our (and everyone else’s) point of view. Hopefully WriteToThem will be cranking back into action — as far as MPs go, at least — fairly soon.