1. New kinds of geography in MapIt

    MapIt is a mySociety service that can take UK postcodes and return which administrative boundaries those postcodes are inside. This can be used to find out what council or constituency an area is in —  you can test it at https://mapit.mysociety.org/.

    Over the last few months we’ve been updating some existing boundaries in MapIt, and adding new kinds of geographies.

    LSOA boundary on openstreetmap

    What has changed

    Local authorities and Clinical Commissioning Groups have been updated to their new April 2020 boundaries. Small census areas have been updated to their latest version across all UK nations.

    There is also a new kind of statistical geography: Travel To Work areas. These are areas that include the home and work location of 75% of people inside them. They are a way of visualising the commuting boundaries of an area, which may be significantly different from the administrative ones (See maps of all Travel To Work areas in England).

    Small census areas are small statistical areas that cover a neighbourhood sized areas (although what locals consider the neighbourhood to be may vary). Many sets of official statistics are mapped against these small areas, making them an important intermediary between postcode or coordinate data and measures such as the indices of multiple deprivation.

    As many statistics are produced separately for different UK nations, there are different kinds of small areas in different nations.

    Smaller:

    • Lower Super Output Areas  (LSOA) – England and Wales
    • Datazones (DZ) – Scotland
    • Super Output Areas (SOA) – Northern Ireland

    Larger:

    • Middle Layer Super Output Area (MSOA) – England and Wales
    • Intermediate Zones (IZ) – Scotland

    In MapIt, all of these boundaries are present, available under the English geography names (LSOA/ MSOA) to avoid needing more complicated lookups when working with postcode data from across different nations.

    What can I use this for?

    Mapping from user postcode data to LSOA helps build a picture of the environment of users. As we’ve done with FixMyStreet, this can be used to understand patterns of use. It can also help researchers with existing postcode datasets to find the equivalent statistical areas to expand the dataset.

    You can see some of these areas in practice powering the postcode lookup on this minisite looking at the new 2019 maps of multiple deprivation in England.

    You can access MapIt from an application through an API, or use the bulk upload tool to convert an existing dataset.

  2. Friends of the Earth’s Climate tool and MapIt

    What is your local authority doing about the climate emergency?

    Of course, we all want to see action, and fast. Several authorities across the UK have declared a climate emergency, while others are bringing climate-friendly propositions to the table. But how do you know the actual concrete outcomes of these?

    Fortunately, Friends of the Earth have put together a tool which helps you see just that — and we’re glad to say it makes use of our MapIt API.

    We spoke to Joachim Farncombe, FoE’s Digital Delivery manager, to find out more about what they built, how it works, and how exactly MapIt fits in.

    How climate-friendly is your area?

    “The Climate Tool invites people to tap in their postcode, and then discover how their local authority is performing on a number of measures, including renewable energy, transport, housing, waste and tree cover.”

    Joachim explains that in fact, they’ve produced two tools: “There’s one highly detailed version which we think our existing supporters will use, and another which provides a summary of the data for those newer to Friends of the Earth and the whole area of councils’ climate responsibility. 

    “Both tools reveal data from local authority areas, around key issues that are impacting our climate. The ultimate aim was to create an engagement opportunity that would drive new and existing supporters to take climate action locally.

    “The whole project is designed to highlight that there are different ways of addressing the climate emergency. One of the key drivers of change is for communities to put pressure on their local authorities to make urgent changes to reduce emissions”.

    Taking action

    So — once you’re all clued up on how your local area is doing, what then?

    “Once you’ve absorbed the data, there’s the option to click on ‘What can I do to help?’.

    “We’re asking people to add their name to support a climate action plan in their area. We’ll also be introducing those who sign up to our climate action groups, a network of community groups working to make our communities more climate friendly.”

    Where does MapIt fit in?

    The MapIt API allows developers to include a postcode input box anywhere on a web page. When a user enters their postcode, MapIt checks which administrative boundaries it sits within. The developer can choose what type of area they need — for example, if the site wants to encourage people to write to their MP, MapIt will return the constituency; or, in this case, as users will be contacting their local authorities, it returns the relevant council.

    Joachim says that FoE already knew of MapIt as they’d used it in their campaign for more trees. “It was very straightforward. The JSON response was easy to parse and the API speed was impressive.”

    Once the user has been matched to the right council, the climate tool dips into its store of data to show them the current climate performance in their area, across key topics.

    “We developed an internal API called FactStore which indexes whatever sets of data you need. In this case, this was data collated from approximately fifty different external datasets. This data was all pulled from open data sources, mostly released by the authorities themselves.”

    The tool was well received, and was shared across social media by supporters and new users alike. “Actually”, says Joachim, “it was a bit more popular than we’d anticipated, and we hit our initial quota on MapIt very early after launch, but there was a quick fix (we just upgraded our quota!)”.

    In short? “MapIt has been invaluable. Without it, we’d be unable to connect the users location with the datasets we’d collated”.

    We’re looking forward to working with Friends of the Earth more in the coming months — watch this space.

    Image: Alicja-ab

  3. Got a list of postcodes and need to match them to administrative areas?

    It’s a more common problem than you might think: given a list of postcodes, how can you match them to the administrative and electoral areas, such as wards or constituencies, that they sit within?

    MapIt’s data mapping tool gives a quick, easy and cheap solution: just upload your spreadsheet of postcodes, tell it which type of area you want them matched to, and the data is returned to you  — complete with a new column containing the information you need.

    The tool can match your postcodes to every type of data that MapIt offers in its API, including council areas, Westminster constituencies, parish wards and even NHS Clinical Commissioning Groups (CCGs).

    If that doesn’t sound like something you can imagine being useful, let’s look at a few hypothetical use cases (and if you have an actual case that you’d like to tell us about, please do let us know  — we’re always keen to hear how our tools are being used).

    Organisations, charities and campaigns sometimes need to match postcodes to administrative areas

    Membership organisations, charities and campaigns usually collect the addresses of supporters, but don’t commonly ask them who their MP is (even if they did ask, most people in the UK don’t actually know the name of their MP).

    But when a campaign asks followers to contact their MPs, it’s helpful to be able to suggest an angle based on whether the MP is known to be sympathetic to their cause, or not — indeed, there’s arguably no point in contacting MPs who are already known to be firmly on board.

    So: input a spreadsheet of supporters’ postcodes, and get them matched to the associated Westminster constituencies.

    For more advanced usages, organisations might match the MapIt tool’s output of postcodes with other datasets to discover the answers to questions like:

    • Which members in a disability group have fewest GPs in their area, and might be finding it difficult to get help for their condition?
    • Which supporters of a transport charity live in regions less served by public transport, and would be likely to take action to campaign for improved bus and train services?
    • Which people affiliated to an ecological organisation live in predominantly rural areas and could help with a wildlife count?

    Researchers sometimes need to match postcodes to administrative areas

    Researchers often need to correlate people, institutions or locations with the boundaries they fall within.

    They might have a list of postcodes for, say, underperforming schools, and want to find out whether they are clustered within authorities that have similar characteristics, like cuts to their funding or an administration that has a political majority one way or the other.

    Teamed with other datasets, MapIt can help towards answering important questions like the number of people each CCG serves, how unemployment rates vary in different European regions, or average house prices within parliamentary constituencies.

    Journalists sometimes need to match postcodes to administrative areas

    Investigative or data journalists may obtain long spreadsheets full of postcodes in the course of their work, perhaps as a result of having submitted Freedom of Information requests to one or more authorities.

    Perhaps they have the address of every university in the country, and there’s an election coming up — during the summer holidays. Knowing that students will mostly be in their home constituencies, they might be able to make informed predictions about how votes in the university towns will be affected.

    Or let’s say that a journalist has gathered, from local councils, an address for every library scheduled to close. This could be compared with another dataset — perhaps literacy or crime rates — to draw conclusions over what impact the closures would have.

    Part of a wider service

    The MapIt machine

     

    The one-off data mapping tool is just one service from mySociety’s MapIt, which is best known for its API.

    This provides an ongoing service, typically for those running websites that ask users to input geographical points such as postcodes or lat/longs, and return tailored results depending on the boundaries those points fall within.

    MapIt powers most mySociety sites, for example:

    • When you drop a pin on the map while using FixMyStreet, MapIt provides the site with the administrative boundaries it falls within, so that the site can then match your report with the authority responsible for fixing it.
    • When you type your postcode into WriteToThem, Mapit gives the site the information it needs to to display a list of every representative, from local councillor up to MEP, who represents your area.
    • If you search for your postcode on TheyWorkForYou, MapIt tells the site what your Westminster constituency is and the site matches that to your MP. You can then be taken to their page with a record of how they have voted and everything they’ve said in Parliament.

    Give it a try

    Find out more about MapIt here or have a go at uploading a spreadsheet into the data mapping tool.

    If you’re not sure whether it’s the right tool for your needs, feel free to drop us a line — and, as we said before, if you are already using it to good effect, please do let us know.

    Image: Thor Alvis

  4. Here’s how mySociety can help you during the 2017 General Election

    It’s official, there’s going to be a General Election in the UK on June 8th.

    As you might suspect mySociety has lots of tools and services that you might find useful during the campaign whether you just want to find out the voting record of your current MP or if you’re planning on building a website or app to cover the campaign.

    TheyWorkForYou

    First things first: TheyWorkForYou.com already covers in lots of detail who your MPs are and how they voted. This should be your first port of call so that you can evaluate your incumbent MP, especially when you’re thinking about who to vote for next.

    Over the next couple of weeks we are going to make some changes here and there to make relevant parts of the voting record more prominent, and more clearly explain how we calculate the voting records themselves.

    If you’re planning on using the data we have in TheyWorkForYou you can access information on UK politicians, parliamentary debates, written answers, and written ministerial statement via our API at theyworkforyou.com/api

    Tomorrow we’ll share a blog post explaining in a little more technical detail how to access the API and some advice on how to get the most out of the service.

    MapIt

    Building a service or website that covers all or part of the country and want an easy way to let your users identify which constituency they are in? Then MapIt is your friend.

    It already powers most of our own services and is widely used by the likes of Government Digital Services and our friends at Democracy Club.

    You can sign up for for free at mapit.mysociety.org and if you need more calls it’s easy to upgrade to a monthly plan – you can get 10,000 calls a month for free if you are a charity or working on an open project – if you think you are going to be busier than that (a) congrats and (b) drop us an email at mapit@mysociety.org

    Helping Democracy Club

    Speaking of Democracy Club we’re going to be wholeheartedly supporting their efforts to crowdsource a full set of candidate data in the run up to the election – they are gathering all of their ideas together in this Google Doc https://goo.gl/8WtZvc

    We had planned to make some updates and amends to the YourNextRepresentative service that supports Democracy Club’s WhoCanIVotefor.co.uk site in the quiet period between major elections, ahem, but with the snap election called we’ll be doing what we can to make the site run faster and make whatever UI tweaks and fixes we can in the time available.

    They will no doubt be looking for help in sourcing candidate data, so please do sign up to help and find out what you can do democracyclub.org.uk/blog/2017/04/18/its-ge2017

     

    In summary and to make it easy you can find all of our relevant #GE2017 datasets and APIs here data.mysociety.org/datasets/?category=ge2017

    It’s not too late to let your current MP know what you think on any subject of your choice via WriteToThem.com.

    And finally, don’t forget to register to vote yourself at gov.uk/register-to-vote

    Image courtesy of Maurice on Flickr.com

  5. Meet the new-ish MapIt

    MapIt has had a bit of a refresh to bring the look into line with the rest of the mySociety projects. At the same time, we thought we’d take the opportunity to make it a bit easier for non-technical folk to understand what it offers, and to make the pricing a little less opaque.

     

    New MapIt homepage from mySociety

     

    You may not be familiar with MapIt, but all the same, if you’ve ever found your MP on TheyWorkForYou, written to your representatives on WriteToThem, or reported an issue through FixMyStreet, you’re a MapIt user!

    That’s because MapIt does the heavy lifting in the background when you enter a postcode or location, matching that input to the boundaries it falls within (ward, constituency, borough, etc). It is, if you like, the geographic glue that holds mySociety services together.

    What MapIt does

    Like most of mySociety’s software offerings, MapIt is available for others to use. So for example, the GOV.UK website uses it to put users in touch with the right council for a number of services, and Prostate Cancer UK uses it on their campaign site, using MapIt’s knowledge of CCG (Clinical Commissioning Group) region boundaries.

    And you can use MapIt too: if your app or website needs to connect UK locations with areas like constituencies or counties, it will save you a lot of time and effort.

    Simple payment

    Pricing and payment is a lot slicker now: while it was previously managed manually, you can now purchase what you need online, quickly and without the need for human intervention. It’s also quite simple to see the pricing options laid out.

    We hope that this will make it easier for people to make use of the service, and better understand what level of usage they need. But if you need to experiment, there’s a free ‘sandbox’ to play about with!

    As ever, we’re happy to provide significant discounts for charity and non-profit projects: see more details on the licensing page.

    Selection_012

    The new MapIt pricing

     

    If you have any questions or comments please do get in touch.

    Save

  6. New on MapIt: Local Authority codes for England

    There’s a new piece of data on MapIt, and it wasn’t added by us. It’s tiny but useful, and it’s slightly esoteric, so bear with us and we’ll explain why it’s worth your attention.

    Local Authority codes come from the government’s set of canonical registers. They may not look much, but they’re part of a drive to bring consistency across a wide range of data sets. That’s important, and we’ll try to explain why.

    MapIt page showing the local authority codes

     

    One name can refer to more than one thing

    If you try to buy a train ticket to Gillingham in the UK, and you are lucky enough to be served by a conscientious member of staff, they will check whether you are going to the Gillingham in Kent (GIL), or the one in Dorset (GLM).

    The names of the two towns might be identical, but their three-letter station codes differ, and quite right too — how, otherwise, would the railway systems be able to charge the right fare? And more importantly, how many people would set off confidently to their destination, but end up in the wrong county?

    I mention this purely to illustrate the importance of authoritative, consistent data, the principle that is currently driving a government-wide initiative to ensure that there’s a single canonical code for prisons, schools, companies, and all kinds of other categories of places and organisations.

    Of particular interest to us at mySociety? Local authorities. That’s because several of our services, from FixMyStreet to WriteToThem, rely on MapIt to connect the user to the correct council, based on their geographical position.

    One thing can have more than one name

    I live within the boundaries of Brighton and Hove City Council.

    That’s its official name, but when talking or writing about my local authority, I’m much more likely to call it ‘Brighton’, ‘Brighton Council’, or at a push, ‘Brighton & Hove Council’. All of which is fine within everyday conversation, but which is an approach which could cause mayhem for the kind of data that digital systems need (“machine readable” data, which is consistent, structured and in a format which can be ‘understood’ by computer programs).

    Registers of Open Data

    The two examples above go some way towards explaining why the Department for Local Government & Communities, with Government Digital Services (GDS), are in the process of creating absolute standards, not just for councils but for every outpost of their diverse and extensive set of responsibilities, from the Food Standards Agency to the Foreign & Commonwealth Office, the Land Registry and beyond.

    Where possible, these registers are published and shared as Open Data that anyone can use. It’s all part of GDS’ push towards ‘government as a platform’, and in keeping with the work being done towards providing Open Data throughout the organisation. Where possible these registers are openly available, and can be used by anyone building apps, websites and systems.

    And now we come to those Local Authority codes that you can find on MapIt.

    Anyone can contribute to Open Source code

    Like most mySociety codebases, MapIt is Open Source.

    That means that not only can anyone pick up the code and use it for their own purposes, for free, but that they’re also welcome to submit changes or extensions to the existing code.

    And that’s just how GDS’ Sym Roe submitted the addition of the register.

    What it all means for you

    If you’re a developer, the addition of these codes means that you can use MapIt in your app or web service, and be absolutely sure that it will integrate with any other dataset that’s using the same codes. So, no more guessing whether our ‘Plymouth’ is the same as the ‘Plymouth’ in your database; the three-letter code tells you that it is.

    Plus, these register codes identify a local authority as an organisation, or a legal entity, as opposed to setting out the boundary, so that’s an extra layer of information which we are glad to be able to include.

    Image (showing another example of uniformly-used three-letter codes): Jim Linwood (CC by/2.0)

  7. Wazimap brings census data alive, with MapIt

    wazimap screenshot

    Census data: there’s lots of it. It contains fascinating insights.

    But as with many huge datasets, those insights are not always easy to find at first glance — nor is it easy for the untrained observer to see which parts are relevant to their own lives.

    Wazimap in South Africa takes the country’s census data and turns it into something the user can explore interactively. Originally conceived as a tool for journalists, it turned out to be so accessible that it’s used by a much wider range of the population, from school children to researchers. It’s a great example of how you can transform dry data into something meaningful online, and it’s all done using free and open source tools.

    Our points-to-boundaries mapping software MapIt is part of that mix, putting the data in context and ensuring that visitors can browse the data relevant to specific provinces, municipalities or wards.

    We asked Greg Kempe of Code for South Africa, to fill us in on a bit more.

    What exactly is Wazimap?

    Wazimap helps South Africans understand where they live, through the eyes of the data from our 2011 Census. It’s a research and exploration tool that describes who lives in South Africa, from a country level right down to a ward, including demographics such as age and gender, language and citizenship, level of education, access to basic services, household goods, employment and income.

    It has helped people understand not just where they work and live, but also that data can be presented in a way that’s accessible and understandable.

    Users can explore the profile of a province, city or ward and compare them side-by-side. They can focus on a particular dataset to view just that data for any place in the country, look for outliers and interesting patterns in the distribution of an indicator, or draw an indicator on a map.

    Of course Wazimap can’t do everything, so you can also download data into Excel or Google Earth to run your own analysis.

    Wazimap is built on the open source software that powers censusreporter.org, which was built under a Knight News Challenge grant, and is a collaboration between Media Monitoring Africa and Code for South Africa.

    Due to demand from other groups, we’ve now made Wazimap a standalone project that anyone can re-use to build their own instance: details are here.

    How did it all begin?

    Media Monitoring Africa approached Code for South Africa to build a tool to help journalists get factual background data on anywhere in South Africa, to help encourage accurate and informed reporting.

    Code for South Africa is a nonprofit that promotes informed decision-making for positive social change, so we were very excited about collaborating on the tool.

    Could MapIt be useful for your project? Find out more here

    How exactly does MapIt fit into the project?

    Mapit powers all the shape boundaries in Wazimap. When we plot a province, municipality or ward boundary on a map in Wazimap, or provide a boundary in a Google Earth or GeoJSON download, MapIt is giving Wazimap that data.

    We had originally built a home-grown solution, but when we met mySociety’s Tony Bowden at a Code Camp in Italy, we learned about MapIt. It turned out to offer better functionality.

    What level of upkeep is involved?

    Wazimap requires only intermittent maintenance. We had municipal elections in August 2016 which has meant a number of municipal boundaries have changed. We’re waiting on Statistics South Africa to provide us with the census data mapped to these new boundaries so that we can update it. Other than that, once the site is up and running it needs very little maintenance.

    What’s the impact of Wazimap?

    We know that Wazimap is used by a wide range of people, including journalists, high school geography teachers, political party researchers and academics.

    wazimap graphsIt has helped people understand not just where they work and live, but also that data can be presented in a way that’s accessible and understandable.

    Code for South Africa has been approached a number of times, by people asking if they might reuse the Wazimap platform in different contexts with different data. Most recently, youthexplorer.org.za used it to power an interactive web tool providing a range of information on young people, helping policy makers understand youth-critical issues in the Western Cape.

    We also know that it’s been used as a research tool for books and numerous news articles.

    The success of the South African Wazimap has driven the development of similar projects elsewhere in Africa which will be launching soon, though MapIt won’t be used for those because their geography requirements are simpler.

    What does the future hold?

    As we’re building out Wazimap for different datasets, we’re seeing a need for taking it beyond just census data. We’re making improvements to how Wazimap works with data to make this possible and make it simpler for others to build on it.

    Each new site gives us ideas for improvements to the larger Wazimap product. The great thing is that these improvements roll out and benefit anyone who uses it across every install.

    Thanks very much to Greg for talking us through the Wazimap project and its use of MapIt. It’s great to hear how MapIt is contributing to a tool that, in itself, aids so many other users and organisations.

    Need to map boundaries? Find out more about MapIt here

  8. MapIt: creating a more flexible service for our customers

    If you’ve visited the MapIt site this week, you might have noticed a change: we’ve introduced key-based authentication for API users.

    This enables us to be more flexible about how we provide our service, which means you can be more flexible about how you serve your users.

    MapIt is both an open source application and, via https://mapit.mysociety.org, a web service. Use of the API is free for low-volume, charitable use, while all other uses require a licence.

    For the moment API keys are optional. We’ll always offer a free level of service to support independent developers and charities.

    We’ll have more details soon about the increased flexibility this change will bring.

    Image: Clare Black (CC)

  9. Lasa’s simplifying Universal Credit, with MapIt

    The Universal Credits system is replacing many other welfare benefits… but slowly. Its roll-out won’t be complete until 2022, meaning that many are, understandably, confused about just what applies within their own local area.

    Now Lasa, in collaboration with the Low Incomes Tax Reform Group (LITRG), have launched a tool to help with that problem. Just input a postcode, and it displays information about which benefits apply — and, crucially, where to go for advice in your area.

    Lasa's Universal Credit tool

    Like Lasa’s previous tools (we wrote about their SocialCareInfo tool previously), UniversalCreditInfo uses MapIt to match postcodes to services.

    It’s part of a suite of offerings, also available as widgets that can be placed onto any website. All fall within Lasa’s remit to support organisations in the delivery of social welfare law advice to the disadvantaged communities they serve.

    Lasa's tools as widgets

    We’re always glad to see MapIt used in other people’s projects, especially those that make a complex system easier to understand.

    Apparently advice workers are already expressing their gratitude for the fact that they can have this information at their fingertips — so hats off to Lasa.


    Image: Russell Davies (CC)

  10. Which ward will you be in on May 5?

    Are you still in the same ward? Check whether your ward boundaries have changed here.

    May 5 is election day

    If you’re a UK citizen, you have an election in your near future. We can say that with confidence.

    May 5 sees elections not only for the Scottish Parliament, the National Assembly of Wales and the Northern Ireland Assembly, but also for many local councils. Londoners will be picking their London Assembly representatives and their Mayor. As if all that isn’t enough, there are also Police and Crime Commissioner Elections.

    Ward boundaries are changing

    You might think you already know where to vote, and who’s standing for election in your area.

    But both are dictated by which ward you live in — and that may not be the one you’re used to, thanks to ongoing changes in ward boundaries.

    There’s no need to worry, though. As before, we’ve provided a nifty tool that will tell you whether your ward has changed. Just enter your postcode here.

    Thanks to Democracy Club’s new project Who Can I Vote For? you’ll also be able to click through to see whatever information is available for candidates in your ward.

    Image: Matt Brown (CC)