All of us at mySociety love the fact that there are so many interesting new civic and democratic websites and apps springing up across the whole world. And we’re really keen to do what we can to help lower the barriers for people trying to build successful sites, to help citizens everywhere.

Today mySociety is unveiling MapIt Global, a new Component designed to eliminate one common, time-consuming task that civic software hackers everwhere have to struggle with: the task of identifying which political or administrative areas cover which parts of the planet.

As a general user this sort of thing might seem a bit obscure, but you’ve probably indirectly used such a service many times. So, for example, if you use our WriteToThem.com to write to a politician, you type in your postcode and the site will tell you who your politicians are. But this website can only do this because it knows that your postcode is located inside a particular council, or constituency or region.

Today, with the launch of MapIt Global , we are opening up a boundaries lookup service that works across the whole world. So now you can lookup a random point in Russia or Haiti or South Africa and find out about the administrative boundaries that surround it. And you can browse and inspect the shapes of administrative areas large and small, and perform sophisticated lookups like “Which areas does this one border with?”. And all this data is available both through an easy to use API, and a nice user interface.

We hope that MapIt Global will be used by coders and citizens worldwide to help them in ways we can’t even imagine yet. Our own immediate use case is to use it to make installations of the FixMyStreet Platform much easier.

Thanks OpenStreetMap!

We’re able to offer this service only because of the fantastic data made available by the amazing OpenStreetMap volunteer community, who are constantly labouring to make an ever-improving map of the whole world. You guys are amazing, and I hope that you find MapIt Global to be useful to your own projects.

The developers who made it possible were Mark Longair, Matthew Somerville and designer Jedidiah Broadbent. And, of course, we’re also only able to do this because the Omidyar Network is supporting our efforts to help people around the world.

From Britain to the World

For the last few years we’ve been running a British version of the MapIt service to allow people running other websites and apps to work out what council or constituency covers a particular point – it’s been very well used. We’ve given this a lick of paint and it is being relaunched today, too.

MapIt Global is also the first of The Components, a series of interoperable data stores that mySociety will be building with friends across the globe. Ultimately our goal is to radically reduce the effort required to launch democracy, transparency and government-facing sites and apps everywhere.

If you’d like to install and run the open source software that powers MapIt on your own servers, that’s cool too – you can find it on Github.

About the Data

The data that we are using is from the OpenStreetMap project, and has been collected by thousands of different people. It is licensed for free use under their open license. Coverage varies substantially, but for a great many countries the coverage is fantastic.

The brilliant thing about using OpenStreetMap data is that if you find that the boundary you need isn’t included, you can upload or draw it direct into Open Street Map, and it will subsequently be pulled into MapIt Global.  We are planning to update our database about four times a year, but if you need boundaries adding faster, please talk to us.

If you’re interested in the technical aspects of how we built MapIt Global, see this blog post from Mark Longair.

Commercial Licenses and Local Copies

MapIt Global and UK are both based on open source software, which is available for free download. However, we charge a license fee for commercial usage of the API, and can also set up custom installs on virtual servers that you can own. Please drop us a line for any questions relating to commercial use.

 Feedback

As with any new service, we’re sure there will be problems that need sorting out. Please drop us an email, or tweet us @mySociety.

This post was written by mySociety volunteer Peter Dixon, who is part of the FixMyTransport team.

There are many reasons nowadays for you to travel across the UK for business, with meetings and relocations being a key reason. Both have affected my personal circumstances within the past few months, so I felt it would be beneficial for me to show how I had used FixMyTransport and FixMyStreet to see more of the areas I am visiting or relocating to.

One of the biggest frustrations when arriving at a hotel on business is that you can be stranded in a strange town or city with no idea about what is in the area. Once you have factored in dinner, the rest of the time is spare and very few people enjoy being stranded in a hotel with only a few TV channels to entertain.

To discover an area, you need to have a look round and a purpose for making a journey can make it easier to undertake a gentle stroll.

When faced with this on a recent trip to the West Midlands, I used FixMyStreet and FixMyTransport to locate issues that were in the area around the hotel and planned a quick walk that allowed me to look at issues that had not been marked as fixed.

As a result of this walk, I was able to mark a couple of the reports as fixed and update some of the others. This doesn’t just limit you to walking, a cycle or a drive can be a great stress reliever for some and a reason to do them makes it so much easier.

I have recently moved house too, and both of the websites provided a great opportunity to explore my new local area. To relieve the boredom of endless housing estates, I used the two websites to find something to look at when having an explore and added new issues as I looked for the existing issues. It’s great to see things getting fixed and know that you have already added to your new community.

So when you are next in a new area and looking for something to do, log on to FixMyStreet or FixMyTransport and see what you can add to the local community.

Image credit: Checking the map by Shaun Dunmall.

On the 21st of February 2012 Alex Skene, representing mySociety’s Freedom of Information website WhatDoTheyKnow, appeared in front of the UK Parliament’s Justice Select Committee. The MPs on the committee were holding an evidence session as part of their post-legislative scrutiny of the Freedom of Information Act.

Video of the session can be viewed online via ParliamentLive.TV and the BBC’s Democracy Live. A transcript of the session will become available via TheyWorkForYou, typically these take a week or two to be produced.

Prior to the session WhatDoTheyKnow had submitted written evidence to the review making three main points:

• The scope of the act should be extended to cover a wider range of public bodies.
• Time limits should be introduced for public interest tests and internal reviews.
• There is a need for more proactive publication of information, and a culture of openness and transparency needs to continue to be nurtured and extended within the UK’s public sector

The committee appeared genuinely interested in finding out how FOI has performed to-date and how it can be improved.

Supercharging FOI

Alex told the committee that FOI enables evidence based policy making and empowers citizens; he said the WhatDoTheyKnow.com website supercharges the provisions of the FOI Act making it easier for people to take advantage of the right to access information which it gives them.

Ghosts

Elfyn Llwyd MP raised the question of vexatious and frivolous requests through the medium of ghosts. Asked if requests about ghosts could ever be justified Alex told MPs that it was hard to draw the line between acceptable and unacceptable requests. He noted that one council had spent public money on an exorcism, so in that case there would be information held and an FOI request justified. He questioned if requests on ghosts were to be deemed unacceptable, what other areas might be excluded. UFOs? The MoD for a long period did have an office collating UFO reports, again there was public spending, and recorded information held, in this area. Homeopathy was also highlighted, that’s about as real as ghosts or UFOs, but again FOI requests about it must surely be permitted as significant amounts of taxpayers money are spent on it.

Maurice Frankel, the director of the Campaign for Freedom of Information, who was giving evidence alongside WhatDoTheyKnow took a stronger line. He described those who made FOI requests about ghosts as “idiots”; but also accepted it was hard, and undesirable, to try and outlaw requests on certain subjects. He added that such requests did not generally cost large amounts of money to deal with.

Time Limits

MPs on the committee appeared sympathetic to calls from the representatives of WhatDoTheyKnow and the Campaign for Freedom of Information to introduce stricter time limits. The need for time limits was brought into focus during the discussion of the time limits for prosecutions under S.77 of the Act (Offence of altering etc. records with intent to prevent disclosure), very few requests have gone through a response, and internal review, and the Information Commissioner within the time limit for launching a prosecution. An MP suggested making offences under S.77 triable in either a magistrates or a crown court so as to extend the time period while retaining consistency with the rest of the justice system.

Fees

When asked to comment on the idea of introducing fees for all FOI requests Alex said such proposals would be “devastating” and would deter many from making requests. Alex noted that the public had paid for the information in question already, via general taxation, and ought be able to access it.

Exempting Universities

When asked to comment on lobbying from universities to be exempted from FOI, Alex robustly defended their inclusion in the act, pointing to their role in controlling access to professions and awarding degrees. Maurice Frankel and Alex noted the universities’ argument that they were being funded by a decreasing fraction of public money wasn’t really relevant, as that is not the basis on which bodies are deemed to be covered by the Act.

Extending Coverage of FOI

The reach of FOI into commercial organisations carrying out work on behalf of public bodies was briefly discussed however notably there was little further discussion of extending the coverage of FOI, perhaps suggesting this may be a dedicated subject for future evidence session. This session was been described as the committee’s first, suggesting there will be more. At least one of these will presumably hear from the Information Commissioner.

The written evidence we submitted can be read on page 81 of the compendium of submitted evidence (PDF).

When TheyWorkForYou was built by a group of volunteer activists, many years ago, it was a first-of-a-kind website. It was novel because it imported large amounts of parliamentary data into a database-driven website, and presented it clearly and simply, and didn’t supply newspaper-style partisan editorial.

These days dozens of such sites exist around the world. But today sees the launch of a rather-special new transparency site: Mzalendo, covering the Parliament of Kenya.

Mzalendo (which means ‘Patriot’ in Swahili) has been around for a few years too, as a blog and MP data website founded by volunteer activists Conrad and Ory. However, over the last few months mySociety’s team members Paul, Jessica and Edmund, plus the team at Supercool Design have been helping the original volunteers to rebuild the site from the ground up. We think that what’s launched today can stake a claim to being a true ‘second generation’ parliamentary monitoring site, for a few reasons:

• It is entirely responsively designed, so that it works on the simplest of mobile web browsers from day one.
• All the lessons we learned from storing political data wrongly have been baked into this site (i.e we can easily cope with people changing names, parties and jobs)
• Every organisation, position and place in the system is now a proper object in the database. So if you want to see all the politicians who went to Nairobi University, you can.
• There is lots of clear information on how parliament functions, what MPs and committees do, and so on.
• It synthesizes some very complex National Taxpayer’s Association data on missing or wasted money into a really clear ‘scorecard‘, turning large sums of money into numbers of teachers.

The codebase that Mzalendo is based on is free and open source, as always. It is a complete re-write, in a different language and framework from TheyWorkForYou, and we think it represents a great starting point for other projects. Over the next year we will be talking to people interested in using the code to run such sites in their own country. If this sounds like something of interest to you, get in touch.

Meanwhile, we wish Ory and Conrad the best of luck as the site grows, and we look forward to seeing what the first users demand.

December 23rd

If you haven’t got a penny,

A ha’penny will do,

If you haven’t got a ha’penny,

Then God bless you.

We wish you all a merry and prosperous Christmas – and for those of you who are already feeling quite prosperous enough, may we point you in the direction of our charitable donations page?

mySociety’s work is made possible by donations of all sizes and from all sorts of people. Those donations help fund all the online projects we create; projects that give easy access to your civic and democratic rights. If that’s important to you, show your appreciation, and we promise we’ll make the best use of every penny.

Thank you for sticking with us through this month-long post. We hope you’ve found it interesting and we wish you the very merriest of Christmases.

We hope you’ll continue to follow us on Twitter, Facebook, or Google+ – see our Contacts page to find individual projects’ social media links.

December 22nd

What’s behind the door? A letter to Santa.

Dear Santa,

We think we’ve been pretty good this year. We’ve tried to keep our local neighbourhood clean, help with problems, and aid those in need, so we’re hoping there are a few presents coming our way.

If you can fit them down the chimney, here’s what we’re dreaming of:

More publicly available data Of course, we were delighted to hear in Mr Osborne’s autumn statement that all sorts of previously-inaccessible data will be opened up.

We’re wondering whether this new era will also answer any of our FixMyStreet geodata wishes. Santa, if you could allocate an elf to this one, we’d be ever so pleased.

Globalisation …in the nicest possible way, of course. This year has seen us work in places previously untouched by the hand of mySociety, including Kenya and the Philippines. And we continue to give help to those who wish to replicate our projects in their own countries, from FixMyStreet in Norway to WhatDoTheyKnow in Germany.

Santa, please could you fix it for us to continue working with dedicated and motivated people all around the world?

A mySociety Masters degree We’re lucky enough to have a team of talented and knowledgeable developers, and we hope we will be recruiting more in the coming year. It’s not always an easy task to find the kind of people we need – after all, mySociety is not your average workplace – so we’ve come to the conclusion that it’s probably easiest to make our own.

Back in February, Tom started thinking about a Masters in Public Technology. It’s still something we’re very much hoping for. Santa, is it true you have friends in academic circles?

FixMyTransport buy-in – from everyone! Regular users of FixMyTransport will have noticed that there are different kinds of response from the transport operators: lovely, fulsome, helpful ones, and formulaic ones. Or, worse still, complete refusal to engage.

Santa, if you get the chance, please could you tell the operators a little secret? Just tell them what those savvier ones already know – that FixMyTransport represents a chance to show off some fantastic customer service. And with 25,000 visitors to the site every week, that message is soon spread far and wide.

(more…)

One of the most popular features on TheyWorkForYou is the plain English, non-judgemental vote analyses on MP pages that say things like “voted strongly against introducing a smoking ban“.  We call these ‘policies’, and they are authored by skilled people using the volunteer run PublicWhip website.

Making each one of these policies is a painstaking task, requiring good knowledge of how the Parliamentary voting system works, good writing skills, patience, and the strength of character not to let your own views about the issues cloud the analysis. It is of utmost importance to both mySociety and our users that these policies are fair and trustworthy.

Earlier this year we started to update the process by which we made new policies to make it even more rigorous, which we wrote about here. Marcus Fergusson and Stephen Young came onboard and did sterling work, but they have now moved on to greater things, and so we’re looking to recruit two to three new people to do this job. Uber volunteer Richard Taylor has been helping out recently, but this is really a job for two or more people.

You might very well ask ‘why two people, given the work is part time?’. The answer is that we really want every new policy to be cross-checked by two different people every time it is added or amended. This is to help eliminate possible mistakes, and prevent any unconscious biases.

We pay for this work on a piece work basis – £160 a time for a combination of one new policy authored, and one other policy double-checked. This money comes mainly from people making small donations, which I think helps keep everyone focussed on how important it is to get these right. We hope to add about two new policies a month, once the new team is up to speed.

If you’d like to be considered, please email hello@mysociety.org with ‘mspolicies’ in the subject line. Applications close 22nd November 2010.

We’ve added a variety of new features to our postcode and point administrative area database, MaPit, in the past month – new data (Super Output Areas and Crown dependency postcodes), new functionality (more geographic functions, council shortcuts, and JSONP callback), and most interestingly for most people, a way of browsing all the data on the site.

• Firstly, we have some new geographic functions to join touches – overlaps, covered, covers, and coverlaps. These do as you would expect, enabling you to see the areas that overlap, cover, or are covered by a particular area, optionally restricted to particular types of area. ‘coverlaps’ returns the areas either overlapped or covered by a chosen area – this might be useful for questions such as “Tell me all the Parliamentary constituencies fully or partly within the boundary of Manchester City Council” (three of those are entirely covered by the council, and two overlap another council, Salford or Trafford).
• As you can see from that link, nearly everything on MaPit now has an HTML representation – just stick “.html” on the end of a JSON URI to see it. This makes it very easy to explore the data contained within MaPit, linking areas together and letting you view any area on Google Maps (e.g. Rutland Council on a map). It also means every postcode has a page.
• From a discussion on our mailing list started by Paul Waring, we discovered that the NSPD – already used by us for Northern Ireland postcodes – also contains Crown dependency postcodes (the Channel Islands and the Isle of Man) – no location information is included, but it does mean that given something that looks like a Crown dependency postcode, we can now at least tell you if it’s a valid postcode or not for those areas.
• Next, we now have all Lower and Middle Super Output Areas in the system; thanks go to our volunteer Anna for getting the CD and writing the import script. These are provided by ONS for small area statistics after the 2001 census, and it’s great that you can now trivially look up the SOA for a postcode, or see what SOAs are within a particular ward. Two areas are in MaPit for each LSOA and MSOA – one has a less accurate boundary than the other for quicker plotting, and we thought we might as well just load it all in. The licences on the CD (Conditions of supply of SOA boundaries and Ordnance Survey Output Area Licence) talk about a click-use licence, and a not very sraightforward OS licence covering only those SOAs that might share part of a boundary with Boundary-Line (whichever ones those are), but ONS now use the Open Government Licence, Boundary-Line is included in OS OpenData, various councils have published their SOAs as open data (e.g. Warwickshire), and these areas should be publicly available under the same licences.
• As the UK has a variety of different types of council, depending on where exactly you are, the postcode lookup now includes a shortcuts dictionary in its result, with two keys, “council” and “ward”. In one-tier areas, the values will simply by the IDs of that postcode’s council and ward (whether it’s a Metropolitan district, Unitary authority, London borough, or whatever); in two-tier areas, the values will again be dictionaries with keys “district” and “council”, pointing at the respective IDs. This should hopefully make lookups of councils easier.
• Lastly, to enable use directly on other sites with JavaScript, MaPit now sends out an “Access-Control-Allow-Origin: *” header, and allows you to specify a JSON callback with a callback parameter (e.g. put “?callback=foo” at the end of your query to have the JSON results wrapped in a call to the foo() function). JSONP calls will always return a 200 response, to enable the JavaScript to access the contents – look for the “error” key to see if something went wrong.

Phew! I hope you find this a useful resource for getting at administrative geographic data; please do let us know of any uses you make of the site.

A number of non-departmental government bodies / quangos have been named as being up for abolition, merger, privatisation or absorption into parent departments, as part of the Coalition Government’s Spending Review, due this autumn. This has been widely dubbed in the press as a “bonfire of the quangos“.  The list of quangos up for review is still being compiled by the government, and there have been a number of clarifications, amendments and retractions as further details come to light.

The Telegraph has obtained and published today a leaked list of 177 quangos up for abolition, plus a further 200 that are still being reviewed.

This is a great opportunity to highlight that mySociety’s Freedom of Information site WhatDoTheyKnow covers nearly all of these little-known bodies that spend public money (we currently have just over 3,800 public authorities listed on the site).  Given their impending doom, there is little time left to find out what they spent public funds on, as only their most important records will be transferred to the National Archives or successor bodies for permanent storage.  The remainder are likely to be shredded, or deleted, as only “records identified as valuable for future administrative need” are kept.

You can see our annotated list of the Telegraph’s list here – our volunteers have added links to most of the bodies’ pages on WhatDoTheyKnow, so you can more easily make your final FOI requests to them…

Please send any missing contact details to the WhatDoTheyKnow team.

As you may know, TheyWorkForYou are conducting a survey of candidates for Parliament.

Quite a few people have been asking how we worked out the questions. There are two parts to this, one local and one national.

Local questions

We used the power of volunteers.

Thousands of DemocracyClub members were asked to suggest local issues in there area. These were then edited by other volunteers, to have consistent grammar, and be worded as statements to agree/disagree with, and filtered to remove national issues. The full criteria and examples are available.

You can view the issues for any constituency on the DemocracyClub site. They are in the “local questions” tab.

We’ve ended up with local issues for about 85% of constituencies. They’re really interesting and high quality, and quite unique for a national survey.

Thank you to all the volunteers who helped make this happen!

National questions

This was hard, because we felt that asking more than 15 questions would make the survey too long. We also wanted to be sure it was non-partisan.

We convened a panel of judges, either from mySociety/Democracy Club or with professional experience in policy, and from across the political spectrum. They were:

• James Crabtree, chair of judges, trustee of mySociety, journalist for Prospect magazine
• Tim Green, Democracy club developer, Physics student, Cambridge University.
• Michael Hallsworth, senior researcher, Institute for Government.
• Will Davies, sociologist at University of Oxford, has worked for left of centre policy think tanks such as IPPR and Demos.
• Andrew Tucker, researcher at Birkbeck, worked for Liberal Democrats from 1996-2000.
• Robert McIlveen, research fellow, Environment and Energy unit at Policy Exchange, did PhD on Conservative party election strategy.

They met at the offices of the Institute for Government, and had a 3 hour judging session on 29th March 2010. They were asked to think of 8-15 questions, with multiple choice answers, which could usefully be answered both by members of the public and prospective candidates for national office.

To ensure maximum transparency, the discussions of the judges were recorded. You can download the recordings in two parts: part 1, part 2 (2 hours, 20 mins total).

Details of the broad framework the judges operated under are given by the chair of judges, James Crabtree, a trustee of mySociety, in the opening to the recordings.

Please do ask any questions in the comments below.

The following is a message that we’d like to see emailed around within political parties of all stripes. If you work for a party, or know anyone who does, please send it along:

———-

Hi there,

TheyWorkForYou.com has sent online surveys to nearly 3000 candidates across the UK, including most of your party’s candidates. If you don’t know it, TheyWorkForYou is probably the largest politician transparency website in the UK, with about 3m visitors last year.

The survey we’ve sent is a rigorously neutral attempt to clarify candidates positions on many of the biggest issues at the election. It is also a long-term document – the data that comes from candidate responses will be viewed millions of times between now and the general election after this one. It also contains both local and national questions.

There are 6000+ volunteers now nagging non-responsive candidates. You can help your party improve its responsiveness rating, here, by passing on the word that TheyWorkForYou’s survey is not push-polling, not single issue, not short-termist.

Please help us by passing on the message that TheyWorkForYou will be one of the main ways that new MPs from all parties (and none) will be scrutinised and neither we nor new MPs want to start our relationship with a “refused to go on the record” badge on their pages.

If you are a candidate, and you want to do the survey, check your email for TheyWorkForYou (no spaces). If you don’t have it, drop a mail to developers@democracyclub.org.uk and it’ll be sent along shortly.

many thanks,

The staff and volunteers at TheyWorkForYou and Democracy Club