Open Data Day resources

Image by OpenSourceWay

Note (June 2016): This post is now slightly out of date. FixMyTransport is no longer running, though all of the other APIs and tools listed are still available.

There is also one significant addition which developers should find useful: EveryPolitician, which provides data on all current politicians around the world (and, in the future, we hope, all past ones too). See more here.


 

Much of what we do here at mySociety relies on Open Data, so naturally we support Open Data Day. In case you haven’t come across this event before, here’s the low-down:

Open Data Day is a gathering of citizens in cities around the world to write applications, liberate data, create visualizations and publish analyses using open public data to show support for and encourage the adoption open data policies by the world’s local, regional and national governments.

If you’re planning on being a part of Open Data Day, you may find some of mySociety’s feeds, tools and APIs useful. This post attempts to put them all in one place.

Data, APIs and tools from mySociety

UK parliament

 TheyWorkForYou is mySociety’s parliamentary monitoring site for the UK. It contains data about every UK parliamentary representative, current and historic (approximately as far back as 1806), and publishes Hansard daily, in a record that stretches back to the ’30s.

API Functions

TheyWorkForYou’s API requires site registration, but you’ll get immediate access on sign-in. Here are some of the things you can do with it:

  • convertURL Converts a parliament.uk Hansard URL into a TheyWorkForYou one, where possible

  • getConstituency Searches for a UK Parliament constituency and returns details

  • getConstituencies Returns list of UK Parliament constituencies

  • getPerson Returns main details for a person

  • getMP Returns main details for an MP

  • getMPInfo Returns extra information for a person

  • getMPsInfo Returns extra information for one or more people

  • getMPs Returns list of MPs

  • getLord Returns details for a Lord

  • getLords Returns list of Lords

  • getMLA Returns details for an MLA

  • getMLAs Returns list of MLAs

  • getMSP Returns details for an MSP

  • getMSPs Returns list of MSPs

  • getGeometry Returns centre, bounding box of UK Parliament constituencies

  • getBoundary Returns boundary polygon of UK Parliament constituency

  • getCommittee Returns members of Select Committee

  • getDebates Returns Debates (either Commons, Westminster Hall, or Lords)

  • getWrans Returns Written Answers

  • getWMS Returns Written Ministerial Statements

  • getHansard Returns any of the above

  • getComments Returns comments

 RSS feeds

Search on any keyword on TheyWorkForYou, and you will be given the option of an RSS feed. Searches can be filtered by date, person speaking, dept/section of Parliament, party, etc.

 Data

Download a csv file of all current MPs

Download a csv file of all current or all historical peers

Parlparse is a Parliament parser – instructions are at the foot of that page. It provides:

  • XML files: list of all MPs (across two files) and list of all Lords. Includes their name and party.

  • rest.cgi – a programmer’s interface for matching names. This uses the all-members.xml, member-aliases.xml and constituencies.xml to match MP names.

  • people.xml – links together groups of MPs from all-members.xml and Lords from peers-ucl.xml who are the same real world person.

  • ministers.xml – contains ministerial positions and the department they were in.

  • member-aliases.xml – a list of alternative names for MPs. Includes abbreviations, misspellings and name changes due to marriage.

  • constituencies.xml – list of Parliamentary constituencies. Includes alternative spellings of each constituency.

  • Register of Members Interests – MPs declare conflicts of interest, and sources of their income.

  • MPs’ expenses (annual): how much each MP claimed as expenses from parliament for travel and so on.

  • Debates (Commons), Debates (Lords), Westminster Hall – XML files containing Debates in the main chambers and in Westminster Hall from the start of the 2001 parliament (Commons) or 1999 reform (Lords).

  • Written Answers (Commons), Written Answers (Lords) – XML files containing Written Answers to questions MPs and Lords have asked ministers. Data from the start of the 2001 parliament (Commons) or 1999 reform (Lords).

  • Written Ministerial Statements (Commons), Written Ministerial Statements (Lords) – XML files containing statements which ministers made to the houses in writing.

 Tools

WriteToThem is mySociety’s site that allows people to contact their representatives, from councillor level to MEP, and including MSPs, MEPs, London Assembly members, etc.

A piece of code by Robin Houston allows you to use this functionality on your own site: details here; code and Readme here.

Mapping and constituencies

MapIt maps UK postcodes and geographical points to administrative areas; MapIt Global does the same, but internationally. Here’s what you can do with them:

  • Lookup by UK postcode: Information on a particular postcode, including its location in both National Grid and WGS84 latitude/longitude, and the areas it is contained within.

  • Lookup by partial UK postcode: Geographical details for the centroid of the partial postcode specified.

  • Lookup by point: A hash of the areas that the point is contained within. If the /box version is used, only the area bounding boxes are considered.

  • Lookup by nearest UK postcode: The postcode closest to the particular point.

  • Lookup by area: Information on a particular area, including name, type, parent, and any associated codes

  • Lookup by related areas: A hash of areas that match the requested lookup.

  • Lookup multiple areas: Information on multiple areas that match the parameters provided, as a dictionary indexed by area ID.

  • Lookup generations: A list of all generations in this installation of MapIt.

Street fault reports

 FixMyStreet.com is mySociety’s site for reporting common street faults, such as potholes and broken streetlights, to local councils.

RSS feeds

Feeds  for a specific ward/council, or for areas within 2km, 5km, 10km or 20km of a set postcode, or for an area centred on a specific postcode and containing roughly 200,000 people.

FixMyStreet platform contains the open-source code and guidance needed for setting up your own version of the site. In theory, this code could be modified to cater for any location-based reporting service.

Freedom of Information

WhatDoTheyKnow is mySociety’s site which makes submitting an FOI request both simple and transparent. The API page allows you to create draft requests to specific bodies, and also has a downloadable list of public bodies in the UK – worth checking out if your project would benefit from information, or even just from a list of publicly-accountable bodies.

RSS feeds

There are RSS feeds for every authority, every individual FOI request, every user, and any keyword.

Alaveteli platform

The open source code that helps you set up a site like WhatDoTheyKnow, anywhere in the world, is called Alaveteli – you’ll find documentation and support there.

Public transport (UK)

FixMyTransport is our site for reporting problems with public transport in the UK. It covers private transport firms and local councils (the latter because of, eg, subsidised bus routes and the upkeep of bus stops and shelters)

RSS feeds

Input any postcode on this page to access an RSS feed of reports made in that area. Zoom in or out of the map, or click and drag to change the area seen and tailor your feed.

Search for any transport operator on this page to access an RSS feed of reports made to that operator.

Search for any route on this page to access an RSS feed of reports made on that route.

Search for any stop or station by searching for a route as above, then clicking on the individual stop/station.

Transcript publication

Take a look at SayIt, our new means of publishing transcripts that can are fully searchable, linkable and where people are represented as interoperable entities. Take a look at our demo transcripts, and note that you can see the underlying code, in the Akomo Ntosa standard, by adding .an to the end of any URL: here’s an example of the mark-up and here’s the resulting page.

Unfortunately, right now, you will need mySociety help to set up a new instance, but you may wish to keep it in mind if you find yourself working on a project that would benefit from SayIt’s capabilities.

All our code

Repositories for all our open source projects can be found at: https://github.com/mysociety.

Good luck! Don’t forget to get in touch and let us know how you’ve used our code or services.

Photo by Open Source Way (CC)