For an organisation whose members normally work from home, we’ve been pretty sociable recently, with meet-ups, conferences, and our annual retreat. We’re glad to discover that we haven’t actually lost the ability to communicate face to face…

If you’d like to come and sample our sharply honed social skills for yourselves, there are a number of opportunities still to come.

Every Wednesday: London meet-ups

[Above : mySociety designer Jed, chilling in front of the Mozilla Firefox]

If you’re in London, do feel free to drop by and say hello, any Wednesday from 6:00 pm – 9:00 pm. We meet at the Mozilla London space – and there are often other interesting things going on too.

Meet-ups are not just for coders – they’re for anyone who would like to talk more about mySociety projects or the wider eDemocracy field. On October 30th, we’re tying in with the Open Government Partnership event; you’re welcome to attend then or any other week.

Fancy coming along? Add your name to our Lanyrd pages here.

[Above: Our meet-ups are not always this busy! On this night, we happened to coincide with a Mozfest planning event.. speaking of which, see below]

25th – 27th October: Mozfest

Mozfest in Ravensbourne, London, is Mozilla’s annual innovative open web event for ‘technologists and creators’.

The event kicks off on the night of Friday 25th with a Science Fair. We’ll be there, showing our wares – in this case, we’ll be hoping to meet many of the internatonal attendeees and let them know about our open source software. But if you’re not an international attendee, you should totally swing by and say hello too.

30th October: Edinburgh

The next non-London mySociety meet-up will be in Edinburgh – watch this blog, our Twitter stream and Facebook page for details of precisely where (it’ll be a nice, central pub that serves food… suggestions are welcome).

That’s in advance of our attendance at the Channel Shift conference – but you don’t have to be a council employee to drop in. Come and share a pint and have a chat, whichever aspect of our work interests you.

20th November: Online Information conference

mySociety’s Director Tom will be giving the keynote presentation at the Online Information conference, the theme of which is “adapting to disruptive technologies and creating value with people, platforms and information”. Feel free to grab Tom afterwards for a chat!

4th December: Manchester

As with Edinburgh, we’re pitching up in Manchester for a Channel Shift conference, and will be taking the opportunity to mingle with lovely locals the night before. Again, pub suggestions are more than welcome.

—

We hope to see you soon at one of these events. And, if you’re wondering what we look like, well, you’re in luck. At our recent retreat we took a photo of the entire team (plus a few guests). Here we are in all our glory – click to see a larger version, if you dare.

Thanks to everyone who dropped into last night’s meet-up: we hope you had a good time. From our point of view, it was a fine mix of technical and social, with just as much chatting as coding.

mySociety’s developers Mark and Dave, project manager Ben, and director Tom were on hand, and there were plenty of new folk, who’d come along simply to find out more about mySociety and the work we do.

Dave reports:

Quite often these discussions spill over into the whole Open Source world and how that works. After all, we’re at the Mozilla London space because we’re part of that sector, and Mozilla are a great example of people who generously share their (splendid) resources to make wonderful things happen in the tech community.

Meanwhile, Andrew continued to chip away at issues on FixMyStreet, and we also enjoyed Viktoria’s impromptu explanation of how graphene works (she works with it) by ingeniously using the Moz space’s hexagonal-patterned flooring.

These meet-ups are open to everyone. So whether you’re an open source veteran, or just a curious newbie who is bemused to discover that there is such a thing as free software – if you’re interested in anything you see on mysociety.org, please come along.

Every Wednesday, 6:00 -9:00 pm. Sign up here.

Photo by Michael Statham (CC)

We love to hear that someone has taken our code and is launching a new website. But it is even better to see that site become fully established.

How can you tell when a site has reached that milestone? Hard to say – but we reckon that processing over 1,000 Freedom of Information requests is a pretty good sign that an Alaveteli install has succeeded.

Ki Mit Tud in Hungary passed that mark back in April; in Spain, Tu Derecho A Saber has also dealt with just over 1,000 requests. And in New Zealand, FYI.org.nz is also celebrating the big thou, under the care of Rowan Crawford.

We made Alaveteli to be used in any jurisdiction, no matter what Freedom of Information laws are in place, and no matter what the prevailing culture – and we are always fascinated to know how the resulting sites are impacting their respective communities. When it comes to New Zealand, some aspects are familiar; some are distinct to the region.

Rowan was kind enough to tell us a little about how it’s been, watching those first thousand FOI requests roll through FYI.org.nz.

How is it running the site? Do you have a volunteer team helping you, like we do with WhatDoTheyKnow in the UK?

“It’s just me at the moment — it hasn’t been a high burden at all. Generally things “just work”.

“As the Admin, I do get unusual questions directed at me sometimes, that strain my knowledge of the law. Fortunately I have several lawyer contacts on Twitter I can run to for help!”

Is there anything we should know about the background to FOI in NZ? 

“The Official Information Act came in to force almost exactly thirty years ago, on 1 July 1983. It is used extensively, though just how much is hard to say.

“The only body which reports OIA statistics is the Ombudsman, which only reports the numbers of complaints – about 1200 in 2012. I know journalists use it routinely.”

How has NZ reacted to having its own Right To Know site?

“The response from people using it has been very, very positive. The press haven’t covered it that much, but bloggers have. One measure of success is that it has its own section in the Ombudsman’s FAQ.

Can you recall any particularly noteable requests that have gone through the site?

“I have an affection for a couple of successful ones from the police: they replied to one request with their interrogation manual, and to another with information on how much they paid informants.”

How are you finding Alaveteli?

“Installing it has improved hugely since the first time I tried. The current method of theming works pretty well, too.

“There are several things on my road map: integrating documentcloud instead of the current HTML conversion, using S3 for resources, and a mobile version.”

What kind of user numbers are you seeing?

“There are just under 500 users registered in the system, and we’re getting 5,000 unique visitors a month. It’s growing month by month by about 15% at the moment — who knows how long that will keep up!”

Run a Freedom of Information website

Rowan makes it look pretty easy!

We won’t pretend that you can run a Freedom of Information site with no effort at all, but we have made it as simple as we possibly can. We have the code, we have the documentation, and most important of all, we have a community waiting to give you support and advice.

If you’re interested in finding out more, drop us a line at hello@mysociety.org.

Today, we are using the phrase “Alaveteli upgrade” rather a lot – and not just because it’s such a great tongue-twister. It’s also a notable milestone for our open-source community.

Alaveteli is the software that underlies WhatDoTheyKnow, our Freedom of Information website. The code can also be deployed by people in other countries who wish to set up a similar site. If you’re a ‘front-end user’, someone who just uses WhatDoTheyKnow to file or read FOI requests, this upgrade will go unnoticed… assuming all goes well at our end, that is. But if you’re a developer who’d like to use the platform in your own country, it makes several things easier for you.

Alaveteli will now be using the Rails 3 series – the series we were previously relying on, 2, has become obsolete. One benefit is that we’re fully supported by the core Rails team for security patches. But, more significant to our aim of sharing our software with organisations around the world, it makes Alaveteli easier to use and easier to contribute to. It’s more straightforward to install, dependencies are up-to-date, code is clearer, and there’s good test coverage – all things that will really help developers get their sites up and running without a problem.

Rails cognoscenti will be aware that series 4.0 is imminent – and that we’ve only upgraded to 3.1 when 3.2 is available. We will be upgrading further in due course – it seemed sensible to progress in smaller steps. But meanwhile, we’re happy with this upgrade! The bulk of the work was done by Henare Degan and Matthew Landauer of the Open Australia Foundation, as volunteers – and we are immensely grateful to them. Thanks, guys.

Find the Alaveteli code here – or read our guide to getting started.

 

Image credit: Sashi Manek (cc)

This is the third of our recent series of Open311 blog posts: we started by explaining why we think Open311 is a good idea, and then we described in a non-techie way how Open311 works. In this post we’ll introduce our proposed extension to Open311, and show how we use it in FixMyStreet.

The crux of our suggested improvement is this: normal people want to know what has happened to their problem, and Open311 currently isn’t good enough at telling them whether or not it has been dealt with. To be more specific, our additions are all about reports’ status change, by which we mean something like this:

That’s robot-311 from the previous post, if you’ve dropped in here without reading the previous posts. Once again we’re blurring the distinction between client and user (the girl you’ll see below) a little, to make things simpler to follow.

Create→send→fix→update

Every month in the UK, thousands of problems are reported on www.fixmystreet.com and, moments later, sent on to the councils who will fix them. Here’s what happens with a problem report for something like a pothole or a flickering streetlight:

1. You create the report on FixMyStreet.
2. FixMyStreet sends that report to the right department at the right council.
3. That body puts it into its own back-end system.
4. Later, when the council fixes the problem, FixMyStreet is updated, and everyone knows it’s fixed.

On the face of it, you might think we need only care about 1 and 2. But really, FixMyStreet isn’t just about dispatching reports, it’s about helping to get things like potholes actually fixed. And neither citizens nor local governments benefit if work gets done but nobody finds out about it – which is part 4 on the list above.

What do we mean by “status change”?

The example at the top of the page shows the robot effectively changing a problem’s status to “fixed”.

Actually, statuses can be simple, such as either OPEN or CLOSED, or more detailed, such as “under investigation”, “crew has been dispatched”, “fixed”, and so on. But since we’re only concerned here with the status changing, that specific vocabulary deployed doesn’t really matter – it can be anything.

In situations where FixMyStreet is not integrated with council systems (i.e we just send email problem reports) FixMyStreet problems still frequently get marked as fixed, because anyone can change the status of a report just by visiting the page and clicking the button.  Obviously, though, we prefer to have FixMyStreet directly connected to the local government back-end databases, so that news of a fixed report can be automatically bubbled from the back-office up into FixMyStreet and out onto the net.

And here’s where the problem lies: Open311 doesn’t quite support this business of getting problem updates from the back office out to the public. So first, we’ll show you how it can be done today, using Open311, and we’ll explain why this isn’t a good option. Then we’ll show our preferred solution, which we’ve proposed as an extension.

Looking at everything just to spot one change (bad)

One way to notice if any problems’ statuses have changed is to use Open311 to ask for every single service request, and see if any of them have a different status since the last time you checked.

Obviously there are some problems with this. Even though Open311 lets you ask for quite specific service requests, you have to ask for all of them, because by definition you don’t know which ones might have changed. Remember, too, that problems can potentially change status more than once, so just because it’s been marked as CLOSED once doesn’t mean it won’t become OPEN again later. This exchange is very wasteful, very slow and ultimately (with enough reports) may become de facto impossible.

Asking for just the changes (good)

So here’s a better way of doing it. We’ve actually been doing this for some months, and now seems the time to share.

The client asks the server for just the updates on a regular basis, so any requests that have recently changed get updated on FixMyStreet automatically, usually just a few minutes later.

Or, more practically for keeping FixMyStreet up to date:

This is handled by our extension to Open311, GET Service Request Updates. There’s also an optional equivalent call for putting updates into the server (POST Service Request Update), which would apply if the client changed the status after the service request had been submitted.

Note that the server identifies the problem with its own reference (that is, 981276 is the council’s reference, not a FixMyStreet ID, for example). This is important because not all these requests necessarily came from this particular client. Remember that all service requests are available through the Open311 GET Service Requests call anyway (as shown above). So the server doesn’t send each service request back in its entirety: just its ID, the new status, when it changed, and a brief description.

In practice the client wouldn’t usually ask for “today”. In fact, we typically send a request asking for any updates in the last 15 minutes, and then at the end of the day ask for the whole day’s updates, just to check none were missed.

The technical bit

From a client’s point of view, this is simply an extra call like others in the Open311 API. So it’s just a request over HTTP(S) for XML (or JSON, if required).

We deliberately make the client poll the server for updates and pull them in, rather than expecting the server to push updates out. This frees the server from any obligation to track which clients (for there may be more than just one) care about which updates. The requests themselves are sent with unique IDs, allocated by the server, so the client can dismiss duplicates. It’s also robust in the event of connection failures, so if there are timeouts or retry logic, that’s for the clients to worry about, not the server. Basically, this is all to make it as light on the server as possible: the only real issue is that it must be able to provide a list of updates. This usually means adding a trigger to the database, so that when a problem’s status is updated a record of that update is automatically created. It’s the table of those “service request update” records that incoming requests are really querying.

We have published our own recommendation of this mechanism, which FixMyStreet already implements, alongside the FixMyStreet codebase.

Is that it?

Yup, that’s it.

This extension is in addition to the Open311 specification — it doesn’t break existing implementations in any way. Obviously this means FixMyStreet’s Open311 implementation is compatible with existing Open311 servers. But we hope that others working on Open311 systems will consider our extension so that clients are kept better informed of the status of the problems being fixed.

Why are statuses so important that it is worth extending the Open311 spec?

mySociety didn’t originally build FixMyStreet because we wanted to get potholes fixed. We built it because we wanted nervous, politically inexperienced people to know what it felt like to ask the government to do something, and to be successful at that. We wanted to give people the buzz of feeling like they have a bit of power in this world, even if the most tiny amount.

If the government fixes a problem and the citizen doesn’t find out it’s a double loss. The citizen becomes disillusioned and weakened, and the government doesn’t get the credit it is due. Everyone loses. We think that Open311 is a key mechanism for making large numbers of people feel that the government does respond to their needs. It just needs a bit of an upgrade to do it better. We hope very much that the wider community tests and endorses our extensions, and it can be folded in to the next official version of the Open311 standard.

Find out more about FixMyStreet for Councils

In the previous blog post we explained why we think Open311 is a good idea. In this post we’ll explain what it actually does.

Open311 is very simple, but because it’s fundamentally a technical thing it’s usually explained from a technical point of view. So this post describes what Open311 does without the nerdy language (but with some nerdy references for good measure). At the end there’s a round-up of the terms so you can see how it fits in with the actual specification.

We’re using an unusual example here — a blue cat stuck up a tree — to show how applicable Open311 is to a wide range of problems. Or, to put it another way, this is not just about potholes.

So… someone has a problem they want to report (for this discussion, she’s using a service like FixMyStreet).

There’s one place where that report needs to be sent (in the UK, that’s your council). That administrative body (the council) almost certainly has a database full of problems which only their staff can access.

In this example, FixMyStreet is an Open311 client and the council is an Open311 server. The server is available over HTTP(S), so the client can access it, and the server itself connects to the council’s database. In reality it’s a little bit more complicated than that (for now we’ll ignore clients that implement only part of Open311, multiple servers, and decent security around these connections), but that is the gist of it.

Although it’s not technically correct to confuse the client with the user, or the server with the council, it makes things a lot easier to see it this way, so we’ll use those terms throughout.

Service discovery

To start things off, the client can ask the server: what services do you provide?

Until the client has asked the server what problems it can fix, it can’t sensibly request any of them.

FixMyStreet can use the response it gets from such a service discovery to offer different categories to people reporting problems. We actually put them into the drop-down menu that appears on the report-a-problem page.

In the Open311 API, this is handled by GET Service List. Each service has its own service_code which the client must use when requesting it. Note that these services and their codes are decided by the server; they are not defined by the Open311 specification. This means that service discovery can easily fit around whatever services the council already offers. The list of services can (and does) vary widely from one council to the next.

Service definitions

Some services require specific information when they are requested. For example, it might be important to know how deep a pothole is, but it’s not relevant for a streetlight repair.

In the Open311 API, this is handled by the GET Service Definition method. It’s not necessary for a simple Open311 implementation. In fact, it only makes sense if the service discovery explicitly told the client to ask about the extra details, which the server does by adding metadata="true" to its response for a given service.

Requesting a service

This is where it gets useful. The client can request a service: this really means they can report a problem to the server for the body to deal with. Some submissions can be automatically rejected:

If the report is in good order, it will be accepted into the system. Open311 insists that every problem has a location. In practice this is usually the exact position, coordinates on planet Earth, of the pin that the reporter placed on the map in the client application (in this case FixMyStreet.com).

In the Open311 API, this is handled by POST Service Request. You need an API key to do this, which simply means the server needs to know which client this is. Sometimes it makes sense for the server to have additional security such as IP address restriction, and login criteria that’s handled by the machines (not the user).

Listing known requests

The server doesn’t keep its reports secret: if asked, it will list them out. The client can ask for a specific report (using the ID that the server gave when the report was submitted, for example) or for a range of dates.

In the Open311 API this is handled by GET Service Request(s). The client can indicate which requests should be listed by specifying the required service request id, service code, start date, end date or status.

Does Open311 work?

Oh yes. On the Open311 website, you can see the growing list of places, organisations, and suppliers who are using it.

The technical bit

In a nutshell: Open311 responds to HTTP requests with XML data (and JSON, if it’s wanted). There’s no messing around with SOAP and failures are reported as the HTTP status code with details provided in the content body.

You can see the specification for Open311 (GeoReport v2). It doesn’t feature blue cats, but if you look at the XML examples you’ll be able to recognise the same interaction described here. And remember the specification is an open standard, which means anyone can (and, we think, should) implement it when connecting a client and server in order to request civic services.

Coming next…

In the next blog post we’ll look at how FixMyStreet uses Open311 to integrate with local council systems, and explain why we’re proposing, and utilising, some additions to the Open311 specification.

Find out more about FixMyStreet for Councils

Illustrated especially for us by René Carbonell.

The Internet has thrown up a host of challenges for governments, large and small. Most people are familiar with the problems presented by issues like hacking, but there is another challenge which probably worries local governments just as much.

The challenge is this – how can a local government cheaply and efficiently cope with the fact that the public wants to request many services through a rapidly expanding plethora of different channels – phones, websites, email, apps, and Twitter? And how can it keep control of costs when new channels are being invented all the time?

The good news is there’s an answer that can prevent each new channel leading to ever-greater costs – a free technology called Open311. The bad news is not many people know it exists, let alone how to use it, or how it works.

In this post, and two more to follow, we’ll explain how Open311 can help governments (and citizens), how it functions, and what mySociety is doing to make Open311 work a bit better.

Background – the status quo

At mySociety, we’ve been running services for years that send messages of different kinds to government bodies, on behalf of our users. Since the very beginning we’ve always been keen that any public servant or politician who receives a message via one of our systems gets it in a familiar form that doesn’t require any special knowledge or training to read or reply to. That’s why for the first few years FixMyStreet sent all its problem reports via email, WhatDoTheyKnow sent all its FOI requests via email, and WriteToThem sent all its letters to politicians via email and fax (remember fax?).

However, despite the fact that reading and responding to emails doesn’t require governments to procure any new technology or any new skills, these days this approach can clearly be bettered. Today, an email report of a broken paving slab will typically be received by a public servant working in a call centre. This person will normally cut and paste text from the email into a new database, or into a new email, before dispatching it for someone else to consider, and action.

Now, imagine that instead of this, a problem report about a broken paving slab could be sent directly from a citizen and placed into the electronic to-do list for the local government team who fix paving slabs. This would do more than just cut costs – it would make it much easier for the citizen to get sent a notification when their problem is marked as ‘resolved’ in the official database.

This is not an original idea. The team at mySociety are not the only people who think that enabling citizens to directly slot requests, messages and problem reports into local government ‘to do’ databases is desirable. In the USA a group of civic minded technologists at OpenPlans were concerned by the same issue. They decided to do something about it – and they launched a project under the banner of Open311.

Why ‘311’?

In the USA a number of cities have non-emergency government telephone helplines, accessible at the phone number 311. As a consequence ‘311’ has come to refer to more than just a phone line – it has come to mean the entire process of handling service requests from citizens around a whole range of non-emergency issues, from garbage to noisy neighbours.

To the ears of some American public servants the name ‘Open311’ consequently conjures up an image of a better, nicer more ‘open’ way of handling such non-emergency requests from citizens.

So what is Open311?

Beyond a brand, what is Open311? The answer is simple: Open311 is standardised way for computers to report problems (like potholes or fallen trees) to the computers run by the bodies that can fix them (like local governments or city departments). It’s an open standard that was started by the lovely people at OpenPlans, and which is now slowly iterating with the help of people inside and outside of governments.

In other words, Open311 is the mechanism through which citizens can slot their service requests directly into the computerised ‘to do’ lists of local government staff, and the way they those citizens can get back progress updates more quickly and easily.

Why is an Open Standard a good thing?

An open standard is just a way of communicating that anyone can implement it, without paying any money for permission to use the technology. The good thing about open standards is that once several technology systems start using the same ones, different systems from different manufacturers can talk to each other. When you phone someone else’s telephone, you are using an open standard – this means you don’t have to have the same brand of phone as the person at the other end.

What this means for a government is that if you can make your database of pothole reports speak to the outside world using Open311 then you don’t have to worry if reports are coming from two, ten or a thousands different websites or apps. You just run one system and it copes with all of them. This is not actually a new idea at all – local government call centres don’t worry what telephone network people are phoning from, or what brand of phone they are using.

However, it is a new idea in the realm of government IT systems for storing things like pothole repair requests, or school-admission applications. Traditionally these systems have not been set up to speak a common language with the outside world. Unfortunately, this failure to speak a common language has not always been by accident. Unscrupulous suppliers will sometimes intentionally set up systems so that the government has to pay extra money if they want any new channels to be added. Using Open311 is both a way to lower your future costs, and a way to make sure your current supplier can’t lock you into expensive upgrades.

Isn’t opening our systems to the outside a security nightmare?

Open311 is not about opening up private data, such as exposing the home addresses of vulnerable children. Open311 can be configured to open up government systems where that is appropriate, and everything that needs to stay private will stay private. There are no fundamental security problems to using an Open311 system.

Is this just about pothole reports?

No. Open311 isn’t limited to street-fixing services like FixMyStreet, even though that kind of problem is where Open311 started. As more and more public bodies offer their services online, they all face the same problem of spiralling costs as the public demands access through more and more diverse channels. In the future it should be possible to renew parking licenses, pay local taxes and do other complex transactions via Open311. But for today we encourage everyone interested to start at the simpler end of the scale.

How does mySociety use Open311?

When a local government anywhere in the world contracts mySociety to deliver a version of FixMyStreet for them (like Bromley, Barnet and Stevenage ) we recommend Open311.

We will still happily connect FixMyStreet to systems that don’t use Open311, but we always explain to clients that Open311 is the most desirable way of connecting their new FixMyStreet deployment with their current problem databases. We even offer lower prices to governments who use Open311.

We offer lower prices this partly because our costs go down, but also because we want to leave local governments with street-fault reporting systems that can connect not just to FixMyStreet, but to any new services in this area that emerge in the future. If Google maps or Twitter suddenly add street fault reporting, why should the local governments have to pay more money to handle those problems, when it could get them for free using Open311?

In short, we see Open311 as a solid foundation for building local government services without locking our clients into a relationship with mySociety as suppliers. In future we will also recommend the use of Open311 for services like ‘Please send me a new recycling bin’, ‘Please tell me what jobs you have open’ and ‘Please answer this FOI request’.

In the next post we’ll cover how Open311 works in a bit more detail (but still as clearly as we can), and in a third post we’ll explain how our work with FixMyStreet for Councils has led us to propose some improvements to the Open311 standard.

Find out more about FixMyStreet for Councils

Photo by Rupert Ganzer (CC)

Summer may seem like a long time ago, but despite the cold outside, we’ve been looking back over our participation in Google’s Summer of Code project. It’s almost enough to warm us up!

This post is an attempt to record the process from our point of view. We hope it will be useful for other organisations considering participating next year, and for students who want to know more about how the scheme works.

What is Google Summer of Code?

It’s a programme sponsored by Google’s philanthropic arm, giving students the chance to experience real-life coding on open source software.

The scheme is open to students all over the world, who are then paired up with open source organisations like us. The students gain paid work experience and mentoring; the organisations gain willing workers and some fresh new perspectives; the world gains some more open source code to use or develop further.

Everyone’s a winner, basically.

The beginnings

2012 was our first year on the programme: once we had been accepted on the scheme, we were given two student slots – the maximum allowed for a first-time organisation.

Given mySociety’s wide suite of codebases, there were several projects that could have benefited. We listed all our ideas, and let people apply for the ones they found appealing.

Goodness, there were a lot of applicants! It was very heartening to discover that there is such an enthusiastic community of young coders all around the world – even if it did take us a long time to sift through them all and make our choices.

You might remember our post back in May, when we announced that we’d made our choices. We were delighted to get working with Dominik from Germany and Chetan from India.

The project

As things turned out, our students ended up working on a project that wasn’t even on our original list: PopIt, our super-easy ‘people and positions’ software.

That’s because once we spoke to our chosen students, we realised they had the skills that could really help us forge ahead with this project – and once we discussed it with them, they were keen. So PopIt it was.

Logistics

Germany and India are a bit of a commute away, but fortunately development work can be managed remotely. We know this particularly well at mySociety: our core team work from home and are scattered across the UK.

The only difference here was the 6+ hour time difference between us and India: it was important to be rigorous about checking in at times when Chetan would be awake!

We communicated via IRC (instant chat), email, and occasionally Skype, and it all worked well.

Edmund, the team member chosen to be mentor, broke the required tasks down into big pieces so that the students would have realistic work units of several days each.

What was achieved

PopIt is primarily a tool for helping people create and run parliamentary monitoring websites (like TheyWorkForYou) with minimal coding knowledge/effort, though we anticipate that it will have many other uses too.

Our students spent the first half of the summer learning and improving the PopIt codebase. Once they were confident in it, they created their own sites using PopIt as a datasource to test the API, and, hopefully, create a valuable reference resource for the community.

Dominik added a migration tool to PopIt, which lets you upload data as a CSV. This means that you can start a site with a database of names, positions and dates at its heart – within seconds.

His test site was a professors’ database (the code is here). Dom also wrote some helpful posts on the dev blog like this one.

Chetan created an image proxy that lets us serve images in a smart way that makes sense for APIs. His test site was for Indian representatives (here’s the code).

Neither site is being maintained now, which just confirms that it is harder to run a site than to start it. This is not a failing, though. The creation of these sites, along with Chetan and Dom’s feedback, helped us understand where improvements needed to be made. In the course of one summer, PopIt became much more mature.

Looking back on the Summer of Code

Edmund attended a follow-up ‘mentors’ summit’ at the Googleplex in California – he found it very helpful to compare notes with other organisations and find out what had worked best for them all, and he made some good contacts too.

Assuming we get the chance again, would we participate in 2013? Our experience was very positive, but as yet we are undecided, purely because of the fluid nature of our workflow: we don’t yet know whether time and resources will permit.

Obviously, we have enjoyed great benefits from the scheme, but that has depended on quite a bit of input from our side, and we need to be sure that we can ensure that happens again.

Edmund has compiled a list of advice, from the practical (ask students to treat the placement like a full-time job; test coding skills before acceptance) to the desirable (a weekly blog post from participants; make sure you over-estimate the time you’ll spend mentoring). If you’re thinking of participating next year, he’d be happy to pass on his tips for ensuring that you, and your assigned students, get the best out of the Google Summer of Code. Just drop him a line.

Today sees the official launch of FixMyStreet’s open source codebase as a proper tool that we hope people will want to deploy in cities and countries around the world. It is based on FixMyStreet.com which we believe is the most usable, most mature street problem reporting tool in the world, but which is only available to British users.

We’re shouting about this launch a bit because we need your help to make the service ever better. First, we need feedback from programmers about whether we’ve got the install process right – whether it’s as easy and clear as we want it to be. And for non-coders who want to get involved, we want to ask for help with the process of translating the site’s text into different languages.

Over the years there have been many copies of FixMyStreet set up in many countries, often using the site’s original name, but always written by developers from scratch. We’re delighted to have inspired people, but all too often the people trying to build copies have stumbled as they realise just how hard it is to build a tool like this with the polish that users expect. We think that people everywhere would be better off if they could have a local FixMyStreet that was really usable, and really connected to the right people.

So we’re very happy to be able to open up a codebase that has been extensively modified in the last year, to help users around the world manage easy, successful deployments. Steps we have taken include:

• Putting the translation text into Transifex, so that non-technical translators can get started whenever they feel like it
• Developing Amazon Machine Images so people who want to tinker can get started in the minimum possible time
• Rewriting the entire codebase in order to make it a less confusing installation
• Building a global version of our MapIt political boundaries web service, so you can get going without having to wrestle administrative data out of your government before you get started.

Plus with the help of the wonderful OpenStreetMap, you can get maps without licensing hassles too.

Calling it version 1.0 is our way of saying two things. First, that the tool still has a lot of evolution left to do, and a long way to go before it is as good as we want it to become. But more ambitiously, calling it 1.0 is also our way of saying that it’s no longer just a codebase dumped into Github. It’s a real open source project, which we plan to support, and which we hope will make a real difference in the lives of ordinary people. Check it out.

All of us at mySociety love the fact that there are so many interesting new civic and democratic websites and apps springing up across the whole world. And we’re really keen to do what we can to help lower the barriers for people trying to build successful sites, to help citizens everywhere.

Today mySociety is unveiling MapIt Global, a new Component designed to eliminate one common, time-consuming task that civic software hackers everwhere have to struggle with: the task of identifying which political or administrative areas cover which parts of the planet.

As a general user this sort of thing might seem a bit obscure, but you’ve probably indirectly used such a service many times. So, for example, if you use our WriteToThem.com to write to a politician, you type in your postcode and the site will tell you who your politicians are. But this website can only do this because it knows that your postcode is located inside a particular council, or constituency or region.

Today, with the launch of MapIt Global , we are opening up a boundaries lookup service that works across the whole world. So now you can lookup a random point in Russia or Haiti or South Africa and find out about the administrative boundaries that surround it. And you can browse and inspect the shapes of administrative areas large and small, and perform sophisticated lookups like “Which areas does this one border with?”. And all this data is available both through an easy to use API, and a nice user interface.

We hope that MapIt Global will be used by coders and citizens worldwide to help them in ways we can’t even imagine yet. Our own immediate use case is to use it to make installations of the FixMyStreet Platform much easier.

Thanks OpenStreetMap!

We’re able to offer this service only because of the fantastic data made available by the amazing OpenStreetMap volunteer community, who are constantly labouring to make an ever-improving map of the whole world. You guys are amazing, and I hope that you find MapIt Global to be useful to your own projects.

The developers who made it possible were Mark Longair, Matthew Somerville and designer Jedidiah Broadbent. And, of course, we’re also only able to do this because the Omidyar Network is supporting our efforts to help people around the world.

 

From Britain to the World

For the last few years we’ve been running a British version of the MapIt service to allow people running other websites and apps to work out what council or constituency covers a particular point – it’s been very well used. We’ve given this a lick of paint and it is being relaunched today, too.

MapIt Global is also the first of The Components, a series of interoperable data stores that mySociety will be building with friends across the globe. Ultimately our goal is to radically reduce the effort required to launch democracy, transparency and government-facing sites and apps everywhere.

If you’d like to install and run the open source software that powers MapIt on your own servers, that’s cool too – you can find it on Github.

About the Data

The data that we are using is from the OpenStreetMap project, and has been collected by thousands of different people. It is licensed for free use under their open license. Coverage varies substantially, but for a great many countries the coverage is fantastic.

The brilliant thing about using OpenStreetMap data is that if you find that the boundary you need isn’t included, you can upload or draw it direct into Open Street Map, and it will subsequently be pulled into MapIt Global.  We are planning to update our database about four times a year, but if you need boundaries adding faster, please talk to us.

If you’re interested in the technical aspects of how we built MapIt Global, see this blog post from Mark Longair.

Commercial Licenses and Local Copies

MapIt Global and UK are both based on open source software, which is available for free download. However, we charge a license fee for commercial usage of the API, and can also set up custom installs on virtual servers that you can own. Please drop us a line for any questions relating to commercial use.

 Feedback

As with any new service, we’re sure there will be problems that need sorting out. Please drop us an email, or tweet us @mySociety.