One of the key differences between the UK’s national parliament and its local governments is that Parliament produces a written record of what gets said – Hansard.
This practice – which has no actual legal power – still has a huge impact on successful functioning of Parliament. MPs share their own quotes, they quote things back to one-another, journalists cite questions and answers, and every day TheyWorkForYou sends tens of thousands of email alerts to people who want to know who said what yesterday in Parliament. Without freely available transcripts of Parliamentary debates, it is likely that Parliament would not be anything like as prominent an institution in British public life.
No Local Hansards
Councils, of course, are too poor to have transcribers, and so don’t produce transcripts. Plus, nobody wants to know what’s going on anyway. Those are the twin beliefs that ensure that verbatim transcripts are an exceptional rarity in the local government world.
At mySociety we think the time has come to actively challenge these beliefs. We are going to be building a set of technologies whose aim is to start making the production of written transcripts of local government meetings a normal practice.
We believe that being able to get sent some form of alert when a council meeting mentions your street is a gentle and psychologically realistic way of engaging regular people with the decisions being made in their local governments. We believe transcripts are worth producing because they show that local politics is actually carried out by humans.
The State of the Art Still Needs You
First, though – a reality check. No technology currently exists that can entirely remove human labour from the production of good quality transcripts of noisy, complicated public meetings. But technology is now at a point where it is possible to substantially collapse the energy and skills required to record, edit and publish transcripts of public meetings of all kinds.
We are planning to develop software that uses off-the-shelf voice recognition technologies to produce rough drafts of transcripts that can then be edited and published through a web browser. Our role will not be in working on the voice recognition itself, but rather on making the whole experience of setting out to record, transcribe and publish a speech or session as easy, fast and enjoyable as possible. And we will build tools to make browsing and sharing the data as nice as we know how. All this fits within our Components strategy.
But mySociety cannot ourselves go to all these meetings. And it appears exceptionally unlikely that councils will want to pay for official transcribers at this point in history. So what we’re asking today is for interest from individuals – inside or outside councils – willing to have a go at transcribing meetings as we develop the software.
It doesn’t have to be definitive to be valuable
Hansard is the record of pretty much everything that gets said in Parliament. This has led to the idea that if you don’t record everything said in every session, your project is a failure. But if Wikipedia has taught us anything, it is that starting small – producing little nuggets of value from the first day – is the right way to get started on hairy, ambitious projects. We’re not looking for people willing to give up their lives to transcribe endlessly and for free – we’re looking for people for whom having a transcript is useful to them anyway, people willing to transcribe at least partly out of self interest. We’re looking for these initial enthusiasts to start building up transcripts that slowly shift the idea of what ‘normal’ conduct in local government is.
Unlike Wikipedia we’re not really talking about a single mega database with community rules. Our current plans are to let you set up a database which you would own – just as you own your blog on Blogger or WordPress, perhaps with collaborators. Maybe you just want to record each annual address of the Lord Mayor – that’s fine. We just want to build something that suits many different people’s needs, and which lifts the veil on so much hidden decision making in this country.
Get in touch
The main purpose of this post is to tell people that mySociety is heading in this direction, and that we’d like you along for the ride. We won’t have a beta to play with for a good few months yet, but we are keen to hear from anyone who thinks they might be an early adopter, or who knows of other people who might want to be involved.
And we’re just as keen to hear from people inside councils as outside, although we know your hands are more tied. Wherever you sit – drop us a line and tell us what sort of use you might want to make of the new technology, and what sort of features you’d like to see. We’ll get back in touch when we’ve something to share.
The local press in Islington has just reported the accidental release of quite a bit of sensitive personal data by Islington council.
One of our volunteers, Helen, was responsible for spotting that Islington had made this mistake, and so we feel it is appropriate to set out a summary of what happened, to inform journalists and citizens who may be interested.
Note – Concerned residents should contact Islington Council or the Information Commissioner’s Office.
On 27th May a user of our WhatDoTheyKnow website raised an FOI request to Islington Borough Council. On the 26th June the council responded to the FOI request by sending three Excel workbooks. Unfortunately, these contained a considerable amount of accidentally released, private data about Islington residents. In one file the personal data was contained within a normal spreadsheet, in the two other workbooks the personal data was contained on four hidden sheets.
All requests and responses sent via WhatDoTheyKnow are automatically published online without any human intervention – this is the key feature that makes this site both valuable and popular. So these Excel workbooks went instantly onto the public web, where they seem to have attracted little attention – our logs suggest 7 downloads in total.
Shortly after sending out these files, someone within the the council tried to delete the first email using Microsoft Outlook’s ‘recall’ feature. As most readers are probably aware – normal emails sent across the internet cannot be remotely removed using the recall function, so this first mail, containing sensitive information in both plain sight and in (trivially) hidden forms remained online.
Unfortunately, this wasn’t the only mistake on the 26th June. A short while later, the council sent a ‘replacement’ FOI response that still contained a large amount of personal information, this time in the form of hidden Excel tabs. As you can see from this page on the Microsoft site , uncovering such tabs takes seconds, and only basic computer skills.
At no point on or after the 26th June did we receive any notification from Islington (or anyone else) that problematic information had been released not once, but twice, even though all mails sent via WhatDoTheyKnow make it clear that replies are published automatically online. Had we been told we would have been able to remove the information quickly.
It was only by sheer good fortune that our volunteer Helen happened to stumble across these documents some weeks later, and she handled the situation wonderfully, immediately hiding the data, asking Google to clear their cache, and alerting the rest of mySociety to the situation. This happened on the 14th July, a Saturday, and over the weekend mySociety staff, volunteers and trustees swung into action to formulate a plan.
The next working day, Monday 16th July, we alerted both Islington and the ICO about what had happened with an extremely detailed timeline.
The personal data released by Islington Borough Council relates to 2,376 individuals/families who have made applications for council housing or are council tenants, and includes everything from name to sexuality. It is for the ICO, not mySociety, to evaluate what sort of harm may have resulted from this release, but we felt it was important to be clear about the details of this incident.
Speaking at today’s Activate conference in London, Stephen King from the Omidyar Network just announced a grant of extraordinary importance to mySociety.
Over the next three years, the Omidyar Network is granting mySociety an amazing $2.9m.
This unprecedented donation is tied to clear targets which, when translated relate to the following goals:
- Internationalising our current British websites, and helping people around the world to build sites and apps that will drive greater transparency and accountability.
- Growing an ever-stronger commercial team, to help cover our costs which remain unfunded (still substantial).
- Continuing to grow the impact of our UK sites.
If you’re someone who’s ever given us £5 or £10 to support our work, or who’s given your time volunteering on any of our projects, we have a special message for you: we would never have been able to get to the sort of scale of support that today’s grant represents if people like you didn’t believe in us all along. Thank you, and thank you for your continued support – our growing ambitions to make ever greater positive impacts on the world means we need more friends than ever, not fewer.
We’re Open for Business, Partnerships and Conversations
The overall impact of this grant, plus continued support from groups like the Open Society Foundation, Hivos, the Esmée Fairbairn Foundation, Indigo Trust – and many smaller donations – is a huge increase in our overall capacity. We can build more software, help more partners, work with more clients, give more advice.
So if you’ve thought about working with mySociety, or using our open source tech for your own goals, there’s never been a better moment. Get in touch or talk to us on Twitter.
Ah, summer: walks in the park, lazing in the long grass, and the sound of chirping crickets – all overlaid with the clatter of a thousand keyboards.
That may not be your idea of summer, but it’s certainly the ways ours is shaping up. We’re participating in Google’s Summer of Code, which aims to put bright young programmers in touch with Open Source organisations, for mutual benefit.
What do the students get from it? Apart from a small stipend, they have a mentored project to get their teeth into over the long summer hols, and hopefully learn a lot in the process. We, of course, see our code being used, improved and adapted – and a whole new perspective on our own work.
Candidates come from all over the world – they’re mentored remotely – so for an organisation like mySociety, this offers a great chance to get insight into the background, politics and technical landscape of another culture. Ideas for projects that may seem startlingly obvious in, say, Latin America or India would simply never have occurred to our UK-based team.
This year, mySociety were one of the 180 organisations participating. We had almost 100 enquiries, from countries including Lithuania, India, Peru, Georgia, and many other places. It’s a shame that we were only able to take on a couple of the many excellent applicants.
We made suggestions for several possible projects to whet the applicants’ appetite. Mobile apps were popular, in particular an app for FixMyTransport. Reworking WriteToThem, and creating components to complement MapIt and PopIt also ranked highly.
It was exciting to see so many ideas, and of course, hard to narrow them down.
In the end we chose two people who wanted to help improve our nascent PopIt service. PopIt will allow people to very quickly create a public database of politicians or other figures. No technical knowledge will be needed – where in the past our code has been “Just add developers”, this one is “Just add data”. We’ll host the sites for others to build on.
Our two successful applicants both had ideas for new websites that would use PopIt for their datastore, exactly the sort of advanced usage we hope to encourage. As well as making sure that PopIt actually works by using it they’ll both be creating transparency sites that will continue after their placements ends. They’ll also have the knowledge of how to set up such a site, and in our opinion that is a very good thing.
We hope to bring you more details as their projects progress, throughout the long, hot (or indeed short and wet) summer.
PS: There is a separate micro-blog where we’re currently noting some of the nitty gritty thoughts and decisions that go into building something like PopIt. If you want to see how the project goes please do subscribe! The Components mailing list is also a good way of staying in touch.
Top image by Elaine Millan, used with thanks under the Creative Commons licence.
mySociety’s sites are all open-source, which means that anyone can take our code and build their own sites with it. That’s been a core mySociety principle from the very beginning, but as time has passed, we’ve realised that we could have made the process easier.
Until recently, you had to be pretty techie to use our code. But now, under the banner DIY mySociety, we’re actively working to lower the bar. Firstly, and most importantly, we’re in the process of rewriting much of our code so that it’s nearer ‘plug and play’ status than previously. Then, we’re backing it up with documentation in the form of easy-to-read handbooks, and supportive communities.
If you’ve ever thought of replicating a mySociety-style site in your own neck of the woods, take a peek over at diy.mysociety.org. We’ll be regularly updating with news and advice.
Image by Kenny Louie, used with thanks under the Creative Commons licence.
FixMyStreet, our site for reporting things like potholes and broken street lights, has had something of a major redesign, kindly supported in part by Kasabi. With the help of Supercool, we have overhauled the look of the site, bringing it up to date and making the most of some lovely maps. And as with any mySociety project, we’d really appreciate your feedback on how we can make it ever more usable.
The biggest change to the new FixMyStreet is the use of responsive design, where the web site adapts to fit within the environment in which it’s being viewed. The main difference on FixMyStreet, besides the obvious navigation changes, is that in a small screen environment, the reporting process changes to have a full screen map and confirmation step, which we thought would be preferable on small touchscreens and other mobiles. There are some technical details at the end of this post.
Along with the design, we’ve made a number of other improvements along the way. For example, something that’s been requested for a long time, we now auto-rotate photos on upload, if we can, and we’re storing whatever is provided rather than only a shrunken version. It’s interesting that most photos include correct orientation information, but some clearly do not (e.g. the Blackberry 9800).
We have many things we’d still like to do, as a couple of items from our github repository show. Firstly, it would be good if the FixMyStreet alert page could have something similar to what we’ve done on http://planningalerts.barnet.gov.uk/, providing a configurable circle for the potential alert area. We also are going to be adding faceted search to the area pages, allowing you to see only reports in a particular category, or within a certain time period.
Regarding native phone apps – whilst the new design does hopefully work well on mobile phones, we understand that native apps are still useful for a number of reasons (not least, the fact photo upload is still not possible from a mobile web app on an iPhone). We have not had the time to update our apps, but will be doing so in the near future to bring them more in line with the redesign and hopefully improve them generally as well.
The redesign is not the only news about FixMyStreet today
As part of our new DIY mySociety project, we are today publishing an easy-to-read guide for people interested in using the FixMyStreet software to run versions of FixMyStreet outside of Britain. We are calling the newly upgraded, more re-usable open source code the FixMyStreet Platform.
This is the first milestone in a major effort to upgrade the FixMyStreet Platform code to make it easier and more flexible to run in other countries. This effort started last year, and today we are formally encouraging people to join our new mailing list at the new FixMyStreet Platform homepage.
Coming soon: a major upgrade to FixMyStreet for Councils
As part of our redesign work, we’ve spoken to a load of different councils about what they might want or need, too. We’re now taking that knowledge, combining it with this redesign, and preparing to relaunch a substantially upgraded FixMyStreet for Councils product. If you’re interested in that, drop us a line.
Kasabi: Our Data is now in the Datastore
Finally, we are also now pushing details of reports entered on FixMyStreet to Kasabi’s data store as open linked data; you can find details of this dataset on their site. Let us know if it’s useful to you, or if we can do anything differently to help you.
On a mobile, you can see that the site navigation is at the end of the document, with a skip to navigation link at the top. On a desktop browser, you’ll note that visually the navigation is now at the top. In both cases, the HTML is the same, with the navigation placed after the main content, so that it hopefully loads and appears first. We are using display: table-caption and caption-side: top in the desktop stylesheet in order to rearrange the content visually (as explained by Jeremy Keith), a simple yet powerful technique.
If you have any technical questions about the design, please do ask in the comments and I’ll do my best to answer.
MPs are about to review the first five years of the operation of the Freedom of Information Act 2000. We’d like to encourage users of mySociety’s Freedom of Information website WhatDoTheyKnow.com to share their views and experiences with the MPs who are to carry out the review.
The review is being conducted by the House of Commons’ Justice Select Committee.
The committee is currently inviting people to make submissions to it. The deadline for submissions is Friday 3 February 2012.
A memorandum from the Ministry of Justice has been prepared to brief the committee, that document notes, in paragraph 67:
Very little research has been published detailing the views of requesters of information.
Particularly in light of this we thought it would be worthwhile alerting our users to this review; if we could encourage our users to make submissions to the committee that might help ensure they receive balanced evidence: from outside, as well as within, the public sector.
While the committee is interested in any comments on the act’s operation, specific questions the committee has asked for comment on are:
- Does the Freedom of Information Act work effectively?
- What are the strengths and weaknesses of the Freedom of Information Act?
- Is the Freedom of Information Act operating in the way that it was intended to?
Responses can be emailed to: email@example.com
Details of how responses should be formatted and technical details relating to submission are available on the webpage announcing the call for submissions.
Some time in the middle of last night, our Freedom of Information site WhatDoTheyKnow.com was used to send its 100,000th FOI request. It was a simple one, made to the Queen Victoria Hospital NHS Foundation Trust.
WhatDoTheyKnow was launched in February 2008, with these aims: to make it easy to file a FOI request, and to keep a public archive of the requests and (more importantly) the responses received from public bodies. The Freedom of Information Act had been in force since 2005, but we wanted to make it fully accessible to people who were not journalists, lobbyists or professional operatives – it is a law that gives us all a right, not just those experts.
At base, mySociety is about giving people power to people who don’t believe that they have any way of affecting the world around them. Giving practical access to the right enshrined in this Act was and is a meaningful way of advancing that goal.
Then, thanks to a flash of inspiration from our late colleague Chris, we saw a great opportunity to increase the value created by the existence of the Act: we built a system that published the entire exchange of messages between users and public bodies online.
We believe that because of this decision to publish all exchanges with public bodies, WhatDoTheyKnow represents a very unusual phenomenon: a third-party web site that takes an existing piece of legislation and makes it better value for money for the taxpayer. Public money was already being spent answering FOI, but by running WhatDoTheyKnow we could magnify the value generated by each request by making it public, without requiring anyone who worked in a public sector to retrain, buy a new computer system or spend any new money.
And this theory turned out to be right. For every request made on the site, around twenty people come to read materials contained on WhatDoTheyKnow. The multiplier is remarkable, and one of the things that we think is most worth celebrating about this site.
WhatDoTheyKnow’s success is only possible because of a team of fantastically dedicated volunteers. These loyal enthusiasts have helped countless users, and do a simply amazing amount of maintenance work to keep the site friendly, helpful and effective. They are astonishingly talented, principled and knowledgeable, and mySociety owes them a debt of gratitude it will never really be able to pay back.
However, to give them a bit of the credit they deserve, and to highlight some of the countless uses of WhatDoTheyknow, we asked them to pick out some notable requests from the last four years.
Helen “The use of the site by campaign groups like the Campaign for Better Transport to find out about bus subsidy cuts as part of their save our buses campaign.”
John “There was the accidental release of how tax is applied to the Royal Family – which resulted in a Daily Mail front page story.”
Alex “This request about the Warmfront boiler installation scheme has a significant number of annotations. What makes it different is that the user patiently persisted with her original FOI requests, and then has carried on by continuing to help loads more people with details of how to complain and lobby for help and general warm encouragement.”
WhatDoTheyKnow is one of mySociety’s most visited sites, with one and a half million unique visitors in 2011. Like our other projects, it was built as an open source project. Thanks to the Open Society Foundation, we are in the process of making it much easier to re-deploy around the world, under the brand name ‘Alaveteli’. As we speak, there are sites based on our code in places as far apart as New Zealand, Kosovo, Brazil, and the EU, and we’re looking forward to helping people from around the world create more grandchild sites in the years ahead.
Our 100,000 request milestone comes at an interesting time for the Freedom of Information Act. It’s currently under scrutiny by the Justice Select Committee, who are investigating whether it works effectively and in the way that it was intended.
As you might expect, at mySociety, we’re passionate about the right to information. We’ll be submitting evidence to the Justice Select Committee to show just how vital FOI is to good government and a good society. If FOI has touched your life, you might want to do the same.
So. Yesterday we officially launched FixMyTransport, a site that has been in ‘quiet beta’ for a few weeks. Not such a big event, you might think – after all, the site has been open for public use; the only difference was that we were announcing it.
I think we’ve all been gratifyingly taken aback by just how much use the site has seen in the last 24 hours. Thanks to mentions in some of the mainstream press, but equally because of a veritable outpouring of tweets and retweets, word spread quickly. We experienced a 550% rise in visitor numbers (the servers took it in their stride, we are glad to say). Over the course of the day, the number of reports on the site doubled, with more than 70 totally new campaigns being created and many more problems being sent to operators. With each report came more tweets, more blog posts and more users signing up to campaigns.
We’re seeing the idea we worked on become a reality, and that’s both exciting and full of surprises. We knew what we would use such a site for, but we had no idea which issues would most motivate our users (at the latest reckoning, it’s poor air conditioning, delays, and, above all, a lack of decent information).
If you haven’t had a chance to see what FixMyTransport is all about yet, take a look at some of these examples:
Many of these examples see users (not just the operators, but ordinary people) who know a lot more than we do about public transport in this country weighing in with useful insights, which is fantastic.
Don’t forget you can search your local region for reports. If you find one you agree with, lending your support is as easy as clicking a single button, and then spreading the word with a tweet, a Facebook status or however you see fit.
Excuse the puns – they are hard to avoid – but we have the sense that we’re at the beginning of a very exciting journey here. And we’re sure we’re going to enjoy the ride. Thanks to everyone who’s come on board so far.
Everyone at mySociety is quite bubbling with excitement at the news that we’re today officially launching FixMyTransport.com , mySociety’s first new core charitable website since WhatDoTheyKnow launched in 2008. We’ve never before launched a site that took so much work to build, or that contained so much data.
What is it for?
FixMyTransport has two goals – one in your face, and the other more subtle.
The first goal, as the site’s name suggests, is to help people get common public transport problems resolved. We’re talking broken ticket machines, gates that should be open and stations without stair-free access. We’ll help by dramatically lowering the barrier to working out who’s responsible, and getting a problem report sent to them – a task that would have been impossible without the help of volunteers who gathered a huge number of operator email addresses for us. Consequently the service works everywhere in Great Britain, our database has over 300,000 stops and routes for train, tube, tram, bus, coach and ferry.
The second goal – the subtle one – is to see if it is possible to use the internet to coax non-activist, non-political people into their first taste of micro-activism. Whilst the site intentionally doesn’t contain any language about campaigning or democracy, we encourage and provide tools to facilitate the gathering of supporters, the emailing of local media, the posting of photos of problems, and the general application of pressure where it is needed. We also make problem reports and correspondence between operators and users public, which we have frequently seen create positive pressure when used on sister sites FixMyStreet and WhatDoTheyKnow.
Who made it?
FixMyTransport was largely built by one remarkable coder – Louise Crow, who started as a volunteer and who is now one of our longest serving core developers. She spent 18 months coding the site almost entirely by herself, wrestling with truly tortuous data problems and collaborating with Birmingham’s fantastic SuperCool design to make it look lovely (you should hire them, they’re great). She also tolerated my ‘aspirational scattergun’ school of project management with remarkable good humour. She really is the king of transport coding.
Credit must also go to mySociety core dev Dave Whiteland, who made the Facebook integration work, despite not having an account himself!
Why is it dedicated to Angie Martin?
Angie Martin was a mySociety coder for an all-too-brief period before she succumbed to cancer at a devastatingly early age. We’re dedicating this site to her in remembrance of a great, self taught perl monger who should still be here.
We’ll be posting further blog posts about the development process, the data challenges, and the overall project philosophy. In the mean time, please keep arms and legs inside the carriage – FixMyTransport is just about to depart.