We want to give a big shout out to Bytemark, the York-based hosting company who are generously sponsoring a large portion of our server space requirements. Thanks guys: as a result, mySociety sites will be more robust and more responsive than ever before.
Why is this gesture such a huge help to us? Well, to start with, our sites keep getting bigger! Yes, our websites are transactional, but most are also archives – so for example you can summon up FixMyStreet reports dating back to the site’s launch in 2007.
Of course, FixMyStreet grows each day, as more reports are submitted. So does WhatDoTheyKnow, and FixMyTransport. As for TheyWorkForYou, well, MPs keep on talking, and we keep on archiving their words.
Second, it’s incredibly important for us to have hosting from a provider who will quickly and competently respond to our requests. Bytemark have been brilliant on this front during the migration phase.
Our sites should be fast – whether you’re looking at archived content or submitting a new report. You, as a user, should never have to think about the capacity of our servers, or their load tolerance. The pages you want to view should simply be there.
The move over to higher capacity servers should help with all of these aims. Bytemark has two separate data centres, and our sites are now split across them.
If you’ve been enjoying a smooth ride on our sites just recently, you might want to thank Bytemark – although of course, the strongest sign of good hosting is that you’ll never notice a thing.
Thank you to everyone at Bytemark – your generosity is really important to us.
Image (CC): Karl-Ludwig G. Poggeman
It’s high time for another mySociety pubmeet. If you’re in London on 31st of October, please do come for a chat and a drink at the Banker pub.
When: 6:30pm onwards
Where: The Banker, Cousin Lane, London EC4R 3TE. Google map here
The Banker has several separate areas – just ask for mySociety at the bar, or look for someone wearing a hoodie with our logo on it.
Why: No agenda, just a chance to meet and ask any questions, float new ideas, find out more about volunteering for mySociety… or simply enjoy a drink and a natter with nice people.
How: Just turn up. If you’re the organised type, you can add yourself to our Lanyrd page, and see who else is coming too. And if you like to Tweet, Instagram, etc, we use the hashtag #mySocial.
Image (CC): Steve James
One of the projects we’ve been working on at mySociety recently is that of making it easier for people to set up new versions of our sites in other countries. Something we’ve heard again and again is that for many people, setting up new web applications is a frustrating process, and that they would appreciate anything that would make it easier.
To address that, we’re pleased to announce that for both FixMyStreet and MapIt, we have created AMIs (Amazon Machine Images) with a default installation of each site:
You can use these AMIs to create a running version of one of these sites on an Amazon EC2 instance with just a couple of clicks. If you haven’t used Amazon Web Services before, then you can get a Micro instance free for a year to try this out. (We have previously published an AMI for Alaveteli, which helped many people to get started with setting up their own Freedom of Information sites.)
Each AMI is created from an install script designed to be used on a clean installation of Debian squeeze or Ubuntu precise, so if you have a new server hosted elsewhere, you can use that script to similarly create a default installation of the site without being dependent on Amazon:
In addition, we’ve launched new sites with documentation for FixMyStreet and MapIt, which will tell you how to customize those sites and import data if you’ve created a running instance using one of the above methods.
These documentation sites also have improved instructions for completely manual installations of either site, for people who are comfortable with setting up PostgreSQL, Apache / nginx, PostGIS, etc.
Another notable change is that we’re now supporting running FixMyStreet and MapIt on nginx, as an alternative to Apache, using FastCGI and gunicorn respectively.
We hope that these changes make it easier than ever before to reuse our code, and set up new sites that help people fix things that matter to them.
Photo credit: duncan
One of the key differences between the UK’s national parliament and its local governments is that Parliament produces a written record of what gets said – Hansard.
This practice – which has no actual legal power – still has a huge impact on successful functioning of Parliament. MPs share their own quotes, they quote things back to one-another, journalists cite questions and answers, and every day TheyWorkForYou sends tens of thousands of email alerts to people who want to know who said what yesterday in Parliament. Without freely available transcripts of Parliamentary debates, it is likely that Parliament would not be anything like as prominent an institution in British public life.
No Local Hansards
Councils, of course, are too poor to have transcribers, and so don’t produce transcripts. Plus, nobody wants to know what’s going on anyway. Those are the twin beliefs that ensure that verbatim transcripts are an exceptional rarity in the local government world.
At mySociety we think the time has come to actively challenge these beliefs. We are going to be building a set of technologies whose aim is to start making the production of written transcripts of local government meetings a normal practice.
We believe that being able to get sent some form of alert when a council meeting mentions your street is a gentle and psychologically realistic way of engaging regular people with the decisions being made in their local governments. We believe transcripts are worth producing because they show that local politics is actually carried out by humans.
The State of the Art Still Needs You
First, though – a reality check. No technology currently exists that can entirely remove human labour from the production of good quality transcripts of noisy, complicated public meetings. But technology is now at a point where it is possible to substantially collapse the energy and skills required to record, edit and publish transcripts of public meetings of all kinds.
We are planning to develop software that uses off-the-shelf voice recognition technologies to produce rough drafts of transcripts that can then be edited and published through a web browser. Our role will not be in working on the voice recognition itself, but rather on making the whole experience of setting out to record, transcribe and publish a speech or session as easy, fast and enjoyable as possible. And we will build tools to make browsing and sharing the data as nice as we know how. All this fits within our Components strategy.
But mySociety cannot ourselves go to all these meetings. And it appears exceptionally unlikely that councils will want to pay for official transcribers at this point in history. So what we’re asking today is for interest from individuals – inside or outside councils – willing to have a go at transcribing meetings as we develop the software.
It doesn’t have to be definitive to be valuable
Hansard is the record of pretty much everything that gets said in Parliament. This has led to the idea that if you don’t record everything said in every session, your project is a failure. But if Wikipedia has taught us anything, it is that starting small – producing little nuggets of value from the first day – is the right way to get started on hairy, ambitious projects. We’re not looking for people willing to give up their lives to transcribe endlessly and for free – we’re looking for people for whom having a transcript is useful to them anyway, people willing to transcribe at least partly out of self interest. We’re looking for these initial enthusiasts to start building up transcripts that slowly shift the idea of what ‘normal’ conduct in local government is.
Unlike Wikipedia we’re not really talking about a single mega database with community rules. Our current plans are to let you set up a database which you would own – just as you own your blog on Blogger or WordPress, perhaps with collaborators. Maybe you just want to record each annual address of the Lord Mayor – that’s fine. We just want to build something that suits many different people’s needs, and which lifts the veil on so much hidden decision making in this country.
Get in touch
The main purpose of this post is to tell people that mySociety is heading in this direction, and that we’d like you along for the ride. We won’t have a beta to play with for a good few months yet, but we are keen to hear from anyone who thinks they might be an early adopter, or who knows of other people who might want to be involved.
And we’re just as keen to hear from people inside councils as outside, although we know your hands are more tied. Wherever you sit – drop us a line and tell us what sort of use you might want to make of the new technology, and what sort of features you’d like to see. We’ll get back in touch when we’ve something to share.
Exactly one year ago today, we quietly put FixMyTransport.com live. We’d built it as a place where you could contact transport operators, and receive their responses, in public. But would it work?
That depended, of course, on the transport companies, and how they would rise to the challenge. A year on, we’re in a position to see how things have panned out.
As you will know if you have submitted a message to them, there are a handful of operators who refuse to engage via FixMyTransport, even though this requires less effort for them than holding the conversation in private. Worst offenders include Northern Rail, Scotrail, Arriva Trains Wales and South West Trains.
These operators are starting to look as if they might have some customer service secrets to hide. You can see some of their excuses in our archive of correspondence, and frankly, they aren’t all that persuasive:
Scotrail: “We encourage our customers to contact us directly to help give them the service they expect and deserve.”
South West Trains: “In order to guarantee a full and consistent response to the concerns raised, would you please advise our customer to use one of our established methods of contact.”
Arriva Trains Wales: “Receiving feedback from our customers is important to us, and I am grateful for you taking time to report these issues. However, we would ask any customer wishing to log an issue with us to make direct contact with us, rather than submit it to us via a third party.”
Meanwhile, Northern Rail – perhaps not coincidentally one of our most-contacted operators – has a policy of sending a one-liner to say that comments have been ‘passed on to the relevant teams’. That does not comfort those who submit some of their more upsetting or important complaints.
While we are disappointed by this lack of communication, we still think it’s worthwhile using FixMyTransport to make initial contact with such companies.
Why? Because you gain the benefit of comments, advice and support from other users – and your complaint is in public for everyone to see. Even if the operator doesn’t respond, that has to make a difference. Plus, FixMyTransport users will often suggest next steps, such as contacting pressure groups or passenger watchdogs.
You see, while we may have faced difficulties with some operators, there were no such issues with the general public. You came to the site, and you quickly understood what FixMyTransport was trying to achieve. And you chose to use it in preference to the transport companies’ own channels. Perhaps the operators might like to think about why that is.
But let’s not dwell on the negatives. We have to give kudos to East Midlands Trains, First Capital Connect, First Great Western, London Midland, Southern and Virgin, all of whom stepped up to the mark and had no problems whatsoever replying to you via FixMyTransport. Equally, praise is due to Transport for London who act as the central contact for a variety of operators across the city, and Stagecoach Buses’ many subsidiaries.
These companies, along with many other smaller outfits, have consistently responded to your complaints via the site. As a result they have created a large public archive of their good customer service.
A helpful, friendly community has grown, too, aided by our team of volunteers. Over 3,500 people have sent messages through FixMyTransport, and with monthly visitors to the site now coming in at over 180,000, each of those messages has had an average of 50 readers.
This is our first year of many. We’re certainly here for the long haul, and confident that eventually, even the most reluctant operators will come on board. If they don’t, increasingly, their customers are going to be asking why. The last year has shown that there is a demand for our service, and we see ourselves as part of a wider shift towards holding companies to account in public. Think how often you’ve seen a disgruntled customer tweeting or blogging their experience.
Meanwhile, we hope you’ll keep using the site, and telling others about it. You might even consider telling your local transport operators how FixMyTransport can work for them.
We hope, too, that you’ll carry on telling us what works or doesn’t work, via the feedback button at the top of every FixMyTransport page. We’re still in active development, and every suggestion is discussed and considered.
Thanks for helping make FixMyTransport what it is. Now, have a piece of birthday cake.
Image credit: Magnus Franklin
This post is cross-posted from the FixMyTransport blog.
The local press in Islington has just reported the accidental release of quite a bit of sensitive personal data by Islington council.
One of our volunteers, Helen, was responsible for spotting that Islington had made this mistake, and so we feel it is appropriate to set out a summary of what happened, to inform journalists and citizens who may be interested.
Note – Concerned residents should contact Islington Council or the Information Commissioner’s Office.
On 27th May a user of our WhatDoTheyKnow website raised an FOI request to Islington Borough Council. On the 26th June the council responded to the FOI request by sending three Excel workbooks. Unfortunately, these contained a considerable amount of accidentally released, private data about Islington residents. In one file the personal data was contained within a normal spreadsheet, in the two other workbooks the personal data was contained on four hidden sheets.
All requests and responses sent via WhatDoTheyKnow are automatically published online without any human intervention – this is the key feature that makes this site both valuable and popular. So these Excel workbooks went instantly onto the public web, where they seem to have attracted little attention – our logs suggest 7 downloads in total.
Shortly after sending out these files, someone within the the council tried to delete the first email using Microsoft Outlook’s ‘recall’ feature. As most readers are probably aware – normal emails sent across the internet cannot be remotely removed using the recall function, so this first mail, containing sensitive information in both plain sight and in (trivially) hidden forms remained online.
Unfortunately, this wasn’t the only mistake on the 26th June. A short while later, the council sent a ‘replacement’ FOI response that still contained a large amount of personal information, this time in the form of hidden Excel tabs. As you can see from this page on the Microsoft site , uncovering such tabs takes seconds, and only basic computer skills.
At no point on or after the 26th June did we receive any notification from Islington (or anyone else) that problematic information had been released not once, but twice, even though all mails sent via WhatDoTheyKnow make it clear that replies are published automatically online. Had we been told we would have been able to remove the information quickly.
It was only by sheer good fortune that our volunteer Helen happened to stumble across these documents some weeks later, and she handled the situation wonderfully, immediately hiding the data, asking Google to clear their cache, and alerting the rest of mySociety to the situation. This happened on the 14th July, a Saturday, and over the weekend mySociety staff, volunteers and trustees swung into action to formulate a plan.
The next working day, Monday 16th July, we alerted both Islington and the ICO about what had happened with an extremely detailed timeline.
The personal data released by Islington Borough Council relates to 2,376 individuals/families who have made applications for council housing or are council tenants, and includes everything from name to sexuality. It is for the ICO, not mySociety, to evaluate what sort of harm may have resulted from this release, but we felt it was important to be clear about the details of this incident.
Since its launch in 2005, WriteToThem has always covered all parts of the United Kingdom, and the Northern Ireland Assembly was the first body added to TheyWorkForYou after the UK Parliament, in late 2006. So whilst we certainly have not ignored Northern Ireland, it had always been an irritant of mine (and a cause of infrequent emails) that FixMyStreet only covered Great Britain.
This was due to the way it had originally been funded and set up, but those issues were in the past, due to a myriad of changes both internal and external, and it was now more a case of being able to find the resources to implement the necessary work. Late last year, mySociety worked with Channel 4 on the website for their series of programmes on The Great British Property Scandal. This used, in part, code similar to FixMyStreet to let people report empty homes, and it was required to work in all parts of the UK. So as part of that process, code was written or generalised that let aspects of FixMyStreet like the maps and place name lookup work for Northern Ireland locations.
It’s taken a few months since then to allocate the time, but we’ve now been able to take the code written back then, add various other bits, and incorporate it into FixMyStreet – which now covers the 26 councils of Northern Ireland, and the central Roads Service. Issues such as potholes, graffiti, and broken street lighting can be reported to Antrim or Newry and Mourne as easily as Aberdeen or Wyre Forest, and just as in the rest of the UK you can sign up for alerts based around your location or to your council.
21st to 29th July is Love Parks Week – a campaign that raises awareness of the importance of green spaces in local communities.
This year, they are focusing on “healthy” parks – that is to say, parks that are safe, clean, well-maintained, a home for wildlife, and a hub for the local community.
The Love Parks website features a quick online health check which you can complete for your local facilities.
If your local park isn’t up to scratch, this is a great week to report it via FixMyStreet. You might think that you can only report problems on streets and roads, but in fact the automatic geolocation function, and the zoomable, draggable maps mean that you can also pinpoint issues very precisely, even if they’re on open ground.
So, if your playground equipment is rusty, the pond is full of algae, or the dog bin’s overflowing, you know where to turn.
Image by Barbara Mazz, used with thanks under the Creative commons licence.
Quick question – don’t think too hard about it: what is Amazon?
At one level, Amazon is the world’s largest online retailer, a public company listed on the NASDAQ. At another level – the physical – it is a collection of over 50,000 employees, hundreds of warehouses and zillions of servers.
But for most people Amazon is fundamentally a website.
Sure, it’s an extremely impressive website that can send you parcels in the post, and which can relieve you of money with terrifying ease. But to most people the company has very little reality beyond the big white-blue-and-orange website and the brown cardboard packages.
Vanishing Act
The same process is happening to the bits of the government that I interact with – the physical reality of bricks and mortar and people and parks is starting to disappear behind the websites.
Government is increasingly a thing I don’t have any mental images of. I don’t know what my local council looks like, nor am I even clear where it is. I’m sure you all have plenty of interactions with HM Revenue and Customs, but do you know where it is or what it looks like?
Increasingly, when I form a mental image of a branch of government in my head, what I see is the website. What else am I supposed to picture?
Governments no longer just ‘own‘ websites, they are websites.
Heartless Bourgeois Pig
Wait! Stop shouting! I know how this sounds.
I am not so out of touch that I don’t know that there are plenty of people out there who are only too familiar with the physical manifestations of government. They see the government as manifested through prison, or hospital, or the job centre. They have no problem forming a vivid mental image of what government means: a waiting room, a queue, a social worker.
And I also know that most of the poorest people in the UK aren’t online yet. It’s one of the great challenges for our country in the next decade.
But…
The majority of citizens don’t have deep, all encompassing, everyday interactions with the state – at most they drop their kids at school every day, or visit the GP a few times a year. That’s as physically close as they get.
To these people, interacting with government already feels somewhat like interacting with Amazon. It sends them benefits, passports, recycling bins, car tax disks from mysterious dispatch offices and it demands money and information in return. The difference is in emotional tone – the Amazon online interactions tend to be seamless, the government online interactions either painful or impossible – time to pick up the phone.
Increasingly, when a modern citizen looks at a government website, they’re literally seeing the state. And if what they see is ugly, confusing or down-right-broken, increasingly that’s how they’re going to see the state as a whole.
This change in public perception means that a previously marginal problem (bad websites) is now pointing towards a rather more worrying possibility. As government websites continue to fall behind private sector websites, governments will slowly look less and less legitimate – less and less like they matter to citizens, less and less like we should be paying any taxes to pay for them. Why pay for something you can’t even navigate?
It is time for the directors and CEOs of public bodies everywhere to wake up to this possibility, before the ideologues get hold of it.
Governments have the wrong management structures for a digital future
I don’t buy the argument that government websites are bad because all the ubermensch have gone off to work for the private sector. The public sector can often teach the private sector a lot about information design, like British road signs and tube maps, which are fantastic. And, of course, there’s the super team at Gov.uk, who represent the kind of change I’m writing about here.
The real difference is one of management structure and focus. At Amazon, CEO Jeff Bezos and his executive colleagues worry all the time about whether their site or app or Kindle are as good as the competitors. But in central and local governments around the world, the top bosses do not stress every day about whether the user experience of their website is up to scratch, or whether conversion rates are lower than desirable.
The main reason that they don’t worry is because their management boards don’t historically contain anyone whose job it is to worry about the performance of digital services. A council chief exec will worry about finance because their finance director will constantly be nagging them about money. But a council CEO won’t be worrying about whether 10,000 people left their website bitterly disappointed last week, because such issues are not ‘normal things to discuss’ at a board level.
Getting digital people to the top table
The solution, at least in the near term – is to recruit or promote people with digital remits and experience right to the top tier of decision making in government bodies. It means creating new roles like ‘CIO’ or ‘Head of Digital’ which have the same seniority as ‘Head of Adult Social Care’ or ‘Head of HR’. And it means empowering those people to make painful changes that are required to make digital services become brilliant and user-centric.
Clearly, this presents dangers. How do you know what powers to give the new role? How do you stop them damaging critical services? And, most problematic of all – how can you tell that a digital expert isn’t a charlatan? After all, they have niche expertise that you don’t have – how are you supposed to sniff them out?
The answer is that it isn’t easy, and that a lot of knowledge sharing and learning from mistakes will be required. As a shameless plug – we can help here – we can help vet candidates and define their roles in Britain and abroad. But none of this hides the fact that becoming digital – learning to run a public organisation that is a website, will be a fraught affair. The reward, though, is nothing less than helping to guarantee the ongoing legitimacy of government (quite apart from all the happier customers). To me that seems well worth going through some pain for.
If you live anywhere in Britain, it won’t have escaped your attention that it’s been raining a bit, recently.
This has been causing quite a bit of flooding. And when flooding happens, people need to know if it is going to affect them.
Unfortunately, the Environment Agency flood warning website leaves something to be desired. It is, quite frankly, a usability dogs’ breakfast, with problems including:
- It doesn’t answer the main question: Most users arriving at this page simply want to know if they might be in danger. The page should be all about answering that question.
- It is trying to serve national and local needs: Information about flooding across the whole country might be useful to journalists or civil servants, but it shouldn’t be the main element.
- Clutter, clutter: A massive grid of numbers which don’t really mean anything, plus lots of sidebar links.
- Confusing graphics: The page contains a national map which doesn’t actually make it clear that the colours relate to the seriousness of flooding, or that it provides links to further content.
There are also some non-design problems with the postcode lookup, but today we want to stick to just the design issues.
Not just moaning minnies
At mySociety we try to be constructive in our criticism, and so whilst the flood waters are still draining from many people’s homes, we thought that we could do something positive. We want to show that a flood warning page could be an exemplar of clear, user-centered information design. So we made a mockup.
Some of the improvements we’d like to point out are:
- A big page title that makes it obvious what this page is, and the fact that it is official information.
- All the main elements on the page are now focussed on the most likely needs of potential flood victims – journalists can follow a link to a different page for their needs.
- We’ve removed roughly 90% of the links on the page for clarity.
- We’ve removed all numerical data because it wasn’t adding value. Nobody can know if ’5 warnings’ is a lot or a little without some context. As a nod to the overall context we’ve put in a simple graph, similar to a sparkline.
- It presents a clear button to click on if you’re actually endangered by a flood.
- It gives you a way to find out if other people near you are talking about local flooding via social media.
We hope you like this. It’s just the product of a couple of hours’ work, so if you have any suggestions on how it could be better, please let us know.
And, of course, we’re always happy to do similar work for other people.
