1. Mapumental Property Launches

    If you’ve been following mySociety for a while, you’ll know that we have been interested in making maps that show commuting times for several years.

    However, we’ve never made public a simple, free, useful version of our slidy-swooshy Mapumental journey times technology. Until today.

    Today we pull the wraps off Mapumental Property , a house-hunting service covering England, Scotland and Wales, designed to help you work out where you might live if you want a public transport commute of a particular maximum duration. Have a go, and we guarantee you’ll find it an oddly compelling experience.

    We think it’s a genuinely useful tool – especially since unlike some of the other players in this space, we’ve got all the different kinds of public transport, right across the whole of Great Britain. We hope that some of you will find it helpful when deciding where to live.

    However, this launch doesn’t mean mySociety is bent on taking over the property websites sector. Mapumental Property isn’t a challenger to the likes of Rightmove, it’s a calling card – an advertisement for our skills – which we hope will help mySociety to attract people and organisations who want beautiful, useful web tools built for them.

    In particular we’d like people interested in Mapumental to note that:

    • We like to build attractive, usable web tools for clients of all kinds.
    • We know how to use complex data to make simple, lovely things.
    • We can do some mapping technology that others haven’t worked out yet.

    If any of that is of interest, please get in touch, or read about our software development and consulting services.

    I’d like to thank quite a few people for helping with this launch. Duncan Parkes was the lead developer, Matthew Somerville ably assisted. Jedidiah Broadbent did the design. The idea originally came from the late Chris Lightfoot, and me, Tom Steinberg. Francis Irving built the first version, and Stamen came up with the awesome idea of using sliders in the first place (and built some early tech). Kristina Glushkova worked on business development, and Zoopla’s API provides the property data. I’m also grateful to Ed Parsons of Google for very kindly giving us a hat tip when they built some technology that was inspired by Mapumental.  Thanks to everyone – this has been a long time coming.

    We’ll follow up soon with a post about the technology – and in particular how we got away from using Flash. It has been an interesting journey.

  2. Mapumental Property – To Launch Thursday 8th November 2012

    After a great deal of hard work we are pleased to announce that Mapumental Property will be launching on the 8th of November 2012.

    Mapumental Property uses public transport open data from across the country to show you areas you can live that are an acceptable commute from your office, school or other destination. We have data on buses, trains, trams, tubes – so we look into all the combinations that might help you get to work quickly.

    We’ve built the site to solve a problem that these other big sites don’t quite get right – commuting. Nobody likes to commute a minute further than strictly necessary. But in a world of complex public transport networks, especially in our big cities, it can be highly unclear where you might be able to live and still get to work in 30 minutes. Mapumental Property will help, and it works anywhere in Britain. It works in Aberdeen just as well as Shoreditch.

    From next week people in Britain can easily see areas that are less than a specified amount of time away from a place of work, study or other importance, by public transport. So if you’ve ever thought “I wish I could see a map of everywhere less than half an hour’s commute from this office”, this is your answer.

    Look for more updates next week!

  3. Mapumental: mapping a new path

    Mapumental map showing travel times to the Royal Festival Hall

    It’s high time we updated you on Mapumental, our journey-time mapping project. For those who may not remember, Mapumental is based on a simple idea: to visualise transit times, by public transport, from or to any postcode in Great Britain.

    It all began in 2006, when the Department for Transport approached us to see what we might do with public transport data; in 2009 we won an investment loan from Channel 4 and Screen West Midlands which enabled us to build a beta tool – you might have played with it. If not, go on, have a go. It’s fun!

    It’s been quite a long journey to where we are today. Unlike many mySociety projects, funding for Mapumental’s development came from a commercial investment loan, with a condition that we set it up as a business. For that reason, it’s not enough that it’s beautiful and useful – we need to find ways for it to be profitable, too. All revenues are set to come back to fund our not-for-profit activities.

    We could tell from very early on in the project that Mapumental would be a sought-after tool for all sorts of purposes, from business to personal use. For example, you can see commute times at a glance, so it’s great for house-hunters and job-seekers. Consequently, it’s also great for the property and recruitment industries.

    “Your maps look amazing, such a great way of representing what could be really boring data, but isn’t.” – A jobseeker

    We can see loads of other possibilities too – like urban planning. This sort of analysis would have been far more expensive in the past; with Mapumental, planners can see at a glance how accessible a new development would be by public transport. Its potential uses are wide-ranging, answering questions for businesses, organisations, charities, and public facilities – especially those wanting to maximise accessibility or encourage use of greener transport options.

    “The maps are a fantastic, a great tool and should be used for every planning application. I will be using Mapumental for all of our projects!” – Lee Taylor, Veridis Design

    We’ve recently refined a product that’s pared down from the dynamic maps you may remember from that beta tool: static maps. These are simple, non-interactive maps which show transit time in bands. They’re flexible in that they can be generated for any postcode, with any maximum travel time, and depict travel at any given time of day.

    We can provide a one-off map for personal use, or batches of many thousands of maps – as we have done for estate agents Foxtons, who now have a Mapumental map on every property listing.

    As we generate more and more maps for different uses, showing different parts of the country, we’re really enjoying digging out all sorts of surprising facts – like how it’s quicker to travel from Watford to Westminster than it is from some parts of Harringay. Or how Cardiff University students might sensibly live at all points east as far as Newport, but will be stymied for transport in the west if they live anywhere other than Barry or Bridgend.

    In fact, our very favourite use so far has come from an individual who centred his map around his home postcode. He tells us he has printed it off and put it up by the front door, so that on his way out of the house, he can find a new and surprising destination for day-trips.

    Find out more on the Mapumental website – and please do spread the word among friends and colleagues who might benefit from a Mapumental map.

  4. Why we made the travel time maps service

    This project became Mapumental. Please visit that site for details of our travel-time maps services.
    The work was funded and supported by the Department for Transport.

    ——————
    We released our new service yesterday, which allows anyone to order personalised travel commuter maps for any location in Great Britain. Those of you who’ve followed this project for a while might be interested to know how we came to take this route.

    Having finished working on the backend and hosting infrastructure of the Mapumental technology last year, we started thinking about the products that should be built with it. To help us work this out, we talked to lots of people in sectors where journey times matter a lot: residential and commercial property, job search, tourism and public services. What we found is that while everyone loved the dynamic location search technology, there were many situations when people wanted to have a simple static map of commuting times.

    We heard that these maps would be useful to individuals looking for jobs or property – but also organisations, from property sites to providers of public services, businesses and entertainment venues who’d like a map to put on their website and brochures, or to use in internal analysis.

    At first we were surprised, but the more we thought about it, the more sense it made. Our search tool, which we are currently working on updating, serves a different purpose: it shows a combination of search criteria, including travel times, and lets the user play with different parameters interactively. But it did not provide a simple snapshot of travel times for a location, divided in bands which are very helpful in assessing commuting times. So we set out to make the map image service, which is what we launched yesterday.

    This was not particularly straightforward to make, and there were many things to consider: how exactly should the shop work, and what should it offer people? We have settled on four core options for the standard maps: total time mapped, direction of travel (whether the location is where one arrives at, or departs from), arrival or departure time, and custom map title. These maps are really easy to order from the website, and we can make them very quickly.

    Online ordering works really well for small quantities, but is not ideal for high-volume clients. So we also created a new API – a URL fetcher which allows to create maps in high quantities, as and when needed. These maps can be fully customised, from the choice of colours to number of bands and zoom levels.

    The the very first user of our API is Foxtons, the estate agent, who added commuter maps to their property listings last week. It is suitable for any property, jobs or hotels site who hold location information (postcodes, or latitude and longitude) for their listings. The API can equally be used by those needing maps for internal purposes, such as city planners, public services and businesses with multiple branches.

    We are really excited that the service has gone live, and we hope that it helps people and organisations in all sorts of ways. A big thank you to Channel 4 and Screen West Midlands, who have provided the commercial investment to enable the development of Mapumental technology and the new service.

    If you have any feedback or comments, we’d love to hear them.

    Sample map: travel times to Wembley Stadium

  5. First Mapumental-powered property travel time maps go live

    We’re delighted to announce that leading London estate agent Foxtons has become the first property player to use Mapumental maps on its website. Visitors to Foxtons.co.uk will now see that every property listed includes a travel time map, highlighted in Foxtons’ brand colours.

    Foxtons, whose website just won an award for Best Interface Design at the 2011 International Business Awards, were quick to see the value of travel time maps for house-hunters. Thousands of listings now display a simple, beautiful, map showing how long a commute to work or visit to friends will take on public transport – vital pieces of information to consider when looking for a new home.

    The property sector is not the only area of business that stands to benefit from Mapumental’s ground-breaking mapping technology. Mapumental is already talking to major players in the travel industry and recruitment sectors.  Virtually any business that needs to show users how much time it takes to travel to or from a given spot will find these maps very valuable.

    One of Mapumental’s core strengths is its  flexibility when it comes to volume – it can provide anything from a single map at a great price to tens of thousands at a significant volume discount.

    The service utilises travel-time mapping technology developed by mySociety, drawing journey data from the NPTDR dataset. The same data also drives mySociety’s newest project FixMyTransport.com, which launched just last week, and covers all modes of public transport within GB.

    For the maps service, our algorithm calculates journey times from any given point (postcode or latitude and longitude) to every other point in Great Britain. These journey times are displayed as a heatmap, on a background from OpenStreetMap.

    Foxtons has made use of the new Mapumental API which enables clients to define the maps’ appearance precisely according to their company preferences. Parameters for choice include:

    • maximum travel time
    • number of time bands to show
    • colour scheme
    • the direction of travel (to or from the chosen location)
    • target arrival or departure time
    • other information (such as title and legend) that goes on the map.

    The image is then automatically created and can be published on a website and/or included in printed materials. Website owners can publish the maps themselves, or we can create bespoke integration solutions for them..

    To find out more about how Mapumental might work for you, please drop us a line.

    Here are some samples of our maps:

    Travel times from a residential development in Sevenoaks, departing at 7am

    Travel times from St Pancras Reneissance Hotel, departing at 8am

    Travel times to reach Cardiff University by 10am

  6. Job Advert: Commercial Product Manager for Mapumental

    mySociety is looking for someone who loves building relationships, and who enjoys thinking about travel, property and what makes for a great day out.

    We are a non-profit group of staff and volunteers that builds websites that help people do things like find out how their politicians vote, or get broken street lights and potholes fixed in their road. We make most of our charitable money through commercial means, and to that end we’re setting up a spin-off called Mapumental Ltd that is building web mapping tools the likes of which have never been seen before. Watch this video to learn more.

    Mapumental’s underlying technology is now complete, and ready for user-facing products to be built on top. What we need is someone who can do the market research to work out what products we should build, and who can go out and sell them to clients of all shapes and sizes – from holidaymakers to property firms.

    Skills

    • Market analysis – to research what products we should and shouldn’t be building with Mapumental
    • Sales and marketing – to tell people authentic, convincing stories about how our products can help them, and sell the finished products
    • Good copywriting skills – and ability to work with designers to create marketing materials

    Experience

    • At least 12 months experience selling products or services, ideally to corporate clients.
    • Ideally, more than two years’ experience in product management, market research, or marketing
    • First degree in any subject
    • You’ll be able to tell us how you made tricky but successful marketing decisions

    Personal Qualities

    • Most important – you’ll need to be able to meet and collaborate with a diverse range of people, both colleagues and customers, in an amiable, confidence-inspiring manner
    • Preferably – an interest in maps and the internet.
    • You’ll have to be a self-starter who can work independently, and often in a location of your choosing
    • Enthusiasm for new things, new ideas, new businesses

    Location, Hours and Salary

    We can be flexible between a minimum of 3.5 days per week, up to full time. You can live and work anywhere in the UK, but your life will be easier if it is within a couple of hours of both London and Birmingham.

    We offer a salary of £33k+, and we are also offering a decent bonus tied to success at sales.

    Applications should be sent to hello@mysociety.org by noon of 13th September, with the tag msjob5  in the subject line. This is a re-advertisement of a previously advertised position, and previous applicants need not apply.

  7. Job Advert: Commercial Product Manager for Mapumental

    mySociety, the non-profit project that built TheyWorkForYou and FixMyStreet is seeking a West Midlands based Commercial Product Manager to drive the success of its new commercial subsidiary Mapumental Ltd

    The job will entail working with the developers to design commercial products based on our  Mapumental mapping technology, and will involve carrying out all the sales and marketing required to secure contracts based on them.

    Skills

    Software as a service product design

    Sales and Marketing

    Strong copywriting skills, and ability to work with designers to produce marketing materials

    Good knowledge of online mapping and GIS

    Experience

    Must have experience of selling software as a service

    Must show proof of having made difficult and successful product design decisions.

    Must have experience carrying out marketing campaigns

    Must be Internet savvy

    Ideally familiar with the GIS sector

    Personal Qualities

    Self starter who can work independently, and at home.

    Driven to make a new business succeed

    Attention to detail

    Friendly and able to collaborate with diverse colleagues

    Location and Hours

    The job is full time and is to be based in the West Midlands (exact location TBD)

    Applications should be sent to hello@mysociety.org by noon of 21st June, with the tag msjob3 in the subject line.

  8. Duncan Parkes is our new Core Developer

    We are very happy to announce that Duncan Parkes has joined mySociety, bringing our team of full time core developers up to four.

    Duncan is the incredibly prolific author of screen scrapers for the lovely PlanningAlerts.com which he runs with Richard Pope.

    He also has a PhD in Mathematics, which I expect you’ll want to read all of here, and is an editor of Open Source programming books with APress. During the vetting process he listed one of the passions of his life as being ‘Unit Testing‘, which, combined with his love of postbox crowdsourcing, made picking him more or less a no brainer.

    In the short run we’ve let him loose, under the tutelage of Francis Irving, on the scaling challenges presented by Mapumental – I can’t wait to see what comes out of it.

  9. What are the two sorts of Cloud infrastructure called?

    I’ve been doing lots of research around “cloud computing” recently, so we can change how Mapumental works and take it out of private beta.

    One thing that’s struck me is that there doesn’t seem to be a proper, industry standard name to distinguish what to me are two fundamentally different sorts of “cloud computing”. I’m focusing here entirely on cloud services for programmers (let’s leave what it means to end users or businesses for another day).

    Here are my own names and descriptions of them:

    1) Cloud hardware server provision (Cloud HSP)
    Low level APIs for making and destroying (virtual) servers, and loading machine images onto them. e.g. Amazon Elastic Compute Cloud, Rackspace Cloud Servers, Eucalyptus’s EC2 bits. Basically, what Eucalyptus v 1.5 can do and what libcloud should do. (By analogy, this is the assembly language of cloud computing)

    2) Cloud developer service provision (Cloud DSP) A service that a developer accesses with one name and a simple API, and behind the scenes it scales for him, automatically. e.g. Amazon Queue Service, Rackspace Cloud Files. (By analogy, this layer is the C programming language of cloud computing)

    [as an aside, Google AppEngine is an interesting one. It is definitely in the Cloud DSP category, but I think it is larger than that – it is a whole set of APIs all in that category. Something like Google DataStore is a single Cloud DSP, albeit one apparently only accessible within AppEngine apps]

    It’s possible to use a Cloud HSP (assembly language), along with a bunch of your own software or open source software, to build new Cloud DSPs (C code). Right now this is pretty hard – even quite well known open source distributed datasbases like CouchDB still need scripting to even make them replicate. The code that makes and destroys servers and gives the service one name, needs manually stringing with quite new bits of wire (things like scalr and Wackamole).

    For this reason, I’m reluctant for mySociety to get into the “making our own Cloud DSP out of Cloud HSP” game. It feels to me like a suck of time, and like we wouldn’t be able to guarantee without lots of careful and expensive testing that it would scale. I’m more tempted to use the commercial Cloud DSP services where possible, even though they are proprietary. But use them via our own abstraction layer, so we can change as we need to. Of course, we have some C++ code (the public transport route finder), so will have to use the Cloud HSP API to get that going, perhaps with Amazon’s Auto Scaling. But it can jolly well use AQS and S3 to talk to other services.

    So, what do you think about the names Cloud HSP/DSP? Are there already existing names for the distinction that I’m making? Is it a useful distinction for you? Can you think of better names?

  10. How Mapumental works

    Here is a diagram of how the backend of Mapumental works. Take it in the spirit that Chris Lightfoot set when he made a similar diagram for the No. 10 petitions site – although many such diagrams are useless, hopefully this one contains useful information.

    (Click on the diagram for a large version)

    Below, I’ve explained what the main components are, and some interesting things about them.

    Everything can, at least in theory, run on lots of servers. Currently we are only actually using one server for web requests, because of problems with HAProxy. We’re runnning isodaemons on two different servers.

    Basic web application – it started out as raw Python, but the more Matthew hacks on it the more Django libraries he pulls in. Soon it’ll be indistinguishable from a Django app. When someone enters a new postcode, it adds it to the work queue in the PostgreSQL database, then refreshes waiting for the job to be finished. Then it displays the flash application (made by Stamen), set up to load the appropriate tile layers.

    Tile server and cache – This uses the Python-based TileCache, calling Geospatial Data Abstraction Library (GDAL) to help render the tiles from points. It was originally written by Stamen, and expanded by mySociety. GDAL isn’t perfect, it doesn’t have fancy enough algorithms for my liking. e.g. Using a median rather than a weighted mean.

    Isodaemons – These are controlled by a Python script, but the bulk of the code is custom written in C++. Slightly crazily, this can find the quickest route by public transport for each of 300,000 journeys from every station in the UK to a particular station, arriving at a particular time, in 10 to 30 seconds.

    I had no idea how to do this, but luckily I live in Cambridge, UK. It’s a city fit to bursting with computer scientists. Many of the jobs are dull, and need little computing, never mind science – like writing interface layers for SQL server. So if you have a real interesting problem it’s easy to get help!

    The universal advice was to use Dijkstra’s algorithm, which needed a bit of adaptation to work efficiently over space-time, rather than just space. Normally it is used for planning routes round a map, but public transport isn’t like that, you have to arrive in time for each particular train, so time affects what journeys you can take.

    I originally wrote it in Python, which was not only too slow, but used up far far too much RAM. It could never have loaded the whole dataset in. However, the old Python code is still run by the test script, to double check the C++ code against. It is also still used to make the binary timetable files, see below.

    Travel times, 1 binary file / postcode – I briefly attempted to insert 300,000 rows into PostgreSQL for each postcode looked up, but it was obvious it wasn’t going to scale. Going back to basics, it now just saves the time taken to travel to each station in a simple binary file – two bytes for each station, 600k in total. The tile server then does random access lookups into that file, as it renders each tile. It only needs to look up the values for the stations it knows are on/near the tile.

    There’s various other bits:

    • cron jobs for sending out invites
    • converting timetable data from ATCO-CIF to the binary format
    • loading static layer data into the database
    • precaching every tile for static datasets
    • Squid and Apache and FastCGI both sit in front of the web applications
    • for speed, we cache the mapping background tiles from Cloudmade
    • when zoomed out, there is code to cull which stations are used to draw tiles
    • of course, a bunch of test code

    Thanks to everyone who helped make Mapumental, we couldn’t have done it without lots of clever people.

    I realise the above is a sketchy overview, so please ask questions in the comments, and I’ll do my best to answer them.