I don’t think it is too controversial to make the following – rather boring – assertions: Greenpeace is part of the environmental movement. Oxfam is an international development charity. Human Rights Watch is part of the human rights movement. Obama for America is a political campaign. Facebook dominates the social networking sector. I hope none of these simple, descriptive statements has caused you to turn purple with semantic rage.
But what primary movement or sector is mySociety part of? Or Avaaz? Or Kiva? Or Wikileaks? When I ask myself these questions, no obvious words or names race quickly or clearly to mind. There is a gap – or at best quite a bit of fuzziness – where the labels should go.
This lack of good labels should surprise us because these groups definitely have aims and goals, normally explicit. Also, it is unusual because social and political movements tend to be quite good at developing names and sticking to them. If you were given a time machine you could tell a Victorian that you were ‘pro-democracy’ or ‘anti-slavery’ and the locals would have no trouble understanding you. Terms like ‘gender equality’, ‘small government’, ‘cancer research’, ‘anti-smoking’, even ‘anti-capitalist’, can comfortably be used by news media companies without fear of baffling the audience. The public can also easily understand terms that referred to methods of achieving change, rather than goals, terms like ‘political TV advertising’, ‘protests’, ‘petitions‘ and ‘telethons’.
But now let’s look at some of the common terms that are used to talk about the (very) wide field of digital social change projects. These include ‘digital transparency’, ‘hacktivism’, ‘peer production’, ‘edemocracy’, ‘clicktivism‘ and ‘open data’. But if you tried to slip one into a newspaper headline, the terms would definitely fall beneath the sub-editor’s axe before they could make it to print. They are too niche, and too likely to confuse readers.
The first thing to note about most of these terms is the way that they refer to methods, rather than goals of social change. But this isn’t completely unprecedented, and isn’t a reason to dismiss these terms out of hand. The name ‘Chartists‘ does indeed refer to people who used the publication of a charter as a political tool, but the name signified a huge bundle of values, methods and goals which went way beyond the deployment of that document.
Nevertheless, to me it still just doesn’t feel like the broad, loosely coupled fields of human endeavour which stretch from Anonymous to JustGiving have decent labels yet – especially not labels that signify the ways in which two things can be both similar and different (e.g. ‘rail station’ and ‘bus station’). And this worries me because consistent names help causes to persist over time. If the field of AIDS research had been renamed every 6 months, could it have lasted as it did? Flighty, narrowly used language confuses supporters, prevents focus and is generally the enemy of long term success.
So, why does this dearth of decent sector labels exist, and can we do anything about it? The short version is, I don’t know. But I do know that the easy answer, ‘It’s all too new to have names’ cannot be right any more, not now that millions have signed petitions, joined Avaaz, donated to Obama online and so on.
I don’t know why the category terms in these sectors are so weak and changeable, but I am posting today because I would love to hear the thoughts of other people who might have some ideas as to the causes, and possible solutions. Here are some theories about the lack of good labels, off the top of my head:
-
I think some of the terms currently in circulation were coined in anticipation of the development of possible projects, not after retrospectively reviewing them. So the category terms sometimes seem to define what a field might look like, rather than what it ends up looking like (think ‘edemocracy‘, from a decade ago). This means the terms often feel like they don’t describe real projects very well.
-
In the traditional (for-profit) internet industry a certain amount of money can be made from coining or becoming associated with new terms (think of IBM and ‘smarter cities’). Because there is a profit motive, there may be a structural incentive to rapidly create new terms which displace older ones which haven’t been widely adopted yet. There are probably similar incentives in some academic fields too – career rewards for coining a key term.
-
Terms in these fields we work in are usually minted one at a time – ‘only children’ as opposed to born as whole families of interconnected terms. This is unlike the sciences which, since Linnaeus came up with his elegant way of naming living things, have been good at developing naming systems, not just one-off names. Organic chemistry and inorganic chemistry are related, but different in important ways – the names helpfully show that. To explain how 38 Degrees and mySociety are similar in some ways but different in other very significant ways needs a way of naming things that can signal both commonality and difference.
-
The knowledge-sharing disconnect between the academic and activist/practitioner communities is really, truly terrible, everywhere except data-driven voter-targeting. People who run services or campaigns normally never hear about what the brightest academics are saying about their own work. And if they do try to pay attention to the ideas coming out of academia then the signal to noise ratio is too bad and the filters are too few and too busy having day jobs.
-
And, of course, I should namecheck the sceptic’s probable theory: this would argue that good, clear terms don’t exist because all these widely differing organisations are nothing more than meaningless feel-good bunk, so language slides off them like an egg off Teflon. I don’t subscribe to this theory, of course, but it’s worth noting because I’m sure some people would provide this answer to my question.
I am planning to write a follow-up blog post to this containing some suggested terms we might use to reflect what the many digital projects out there have in common, and how they are different.
But before I do, I would like to hear people’s thoughts on whether this is a real problem at all, and if so why that might be, and what we might do about it. Who knows, maybe someone will even write a blog post about it, like we’re back in 2003 or something…
The last few weeks since the US election have seen an explosion in articles and blog posts about how Obama’s tech team pulled out the stops in their race against the Republicans. It’s been an exciting time to learn about the new techniques dreamed up, and the old ones put to the test.
For those of us who develop non-partisan services to help people report broken street lights or make Freedom of Information Requests, such stories certainly seem unimaginably glamorous: I don’t think any of my colleagues will ever get hugged by Barack Obama!
But it has also been an interesting time to reflect on the difference between choosing to use tech skills to win a particular fight, versus trying to improve the workings of the democratic system, or helping people to self-organise and take some control of their own lives.
At one level there’s no competition at all: the partisan tech community is big and economically healthy. It raises vast amounts of cold hard cash through credit card payments (taking a cut to pay its own bills) and produces squillons of donors, signers, visitors, tweeters, video watchers and so on. The non-partisan tech community is much smaller – has fewer sustainable organisations, and with the exception of some big online petitions, doesn’t get the same sort of traffic spikes. By these metrics there’s absolutely no doubt which use of tech is the most important: the partisan kind where technology is used to beat your opponent, whether they are a political candidate, a policy, company, or an idea.
But I am still filled with an excitement about the prospects for non-partisan technologies that I can’t muster for even the coolest uses of randomized control trial-driven political messaging. The reason why all comes down to the fact that major partisan digital campaigns change the world, but they don’t do it in the way that services like eBay, TripAdvisor and Match.com do.
What all these sites have in common – helping people sell stuff they own, find a hotel, or a life partner – is that they represent a positive change in the lives of millions of people that is not directly opposed by a counter-shift. These sites have improved the experience of selling stuff, finding hotels and finding life partners in ways that don’t attract equal and opposite forces, driven by similar technologies.
This is different from the case of campaigning tech: here a huge mailing list is pitted against another even huger mailing list. Epic fund-raising tools are pitched against even more epic fund-raising tools. Orca vs Narwhal. Right now, in US politics, the Democrats have a clear edge over the technology lined up against them, and I totally understand why that must feel amazing to be part of. But everything you build in this field always attracts people trying to undo your work by directly opposing it. There is something inate to the nature of partisanship which means that one camp using a technology will ultimately attract counter-usage by an opposing camp.
This automatic-counterweighting doesn’t happen with services that shift whole sectors – like TripAdvisor did. In the hotel-finding world, the customer has been made stronger, the hotel sector weaker, and the net simply doesn’t provide tools to the hotel industry to counter what TripAdvisor does.
It is this model – the model of scaleable, popular technology platforms that help people to live their lives better – that I aspire to bring to the civic, democratic and community spheres in my work. Neither I nor mySociety has yet come up with anything even remotely on the scale of a TripAdvisor, but there remains the tantalising possibility that someone might manage it – a huge, scaleable app of meaningful positive impact on democratic, civic or governance systems*. Our sites are probably about as big as it gets so far, and that’s not big enough by far.
If someone does manage to find and deliver the dream – some sort of hugely scalable, impactful non-partisan civic or democratic app & website, it is unlikely that the net will instantly throw up an equal and opposite counterweight. There is a real possibility that the whole experience of being a citizen, the whole task of trying to govern a country well will be given a shot in the arm that won’t go away as soon as someone figures out how to oppose it. I’m not talking Utopia, I’m just talking better. But what motivates me is that it could be better for good, not just until the Other Team matches your skills.
And that – in rather more words than I meant to use – is why I am still excited by non-partisan tech, and why I really hope that some of the awesome technologists who worked in the political campaigns of 2012 get involved in our scene.
I’m on Twitter if anyone wants to talk about this more.
* You can certainly make an argument that Twitter and Facebook sort-of represent non-partisan democracy platforms that have scaled. But some people disagree vehemently, and I don’t want to get into that here.
One of the key differences between the UK’s national parliament and its local governments is that Parliament produces a written record of what gets said – Hansard.
This practice – which has no actual legal power – still has a huge impact on successful functioning of Parliament. MPs share their own quotes, they quote things back to one-another, journalists cite questions and answers, and every day TheyWorkForYou sends tens of thousands of email alerts to people who want to know who said what yesterday in Parliament. Without freely available transcripts of Parliamentary debates, it is likely that Parliament would not be anything like as prominent an institution in British public life.
No Local Hansards
Councils, of course, are too poor to have transcribers, and so don’t produce transcripts. Plus, nobody wants to know what’s going on anyway. Those are the twin beliefs that ensure that verbatim transcripts are an exceptional rarity in the local government world.
At mySociety we think the time has come to actively challenge these beliefs. We are going to be building a set of technologies whose aim is to start making the production of written transcripts of local government meetings a normal practice.
We believe that being able to get sent some form of alert when a council meeting mentions your street is a gentle and psychologically realistic way of engaging regular people with the decisions being made in their local governments. We believe transcripts are worth producing because they show that local politics is actually carried out by humans.
The State of the Art Still Needs You
First, though – a reality check. No technology currently exists that can entirely remove human labour from the production of good quality transcripts of noisy, complicated public meetings. But technology is now at a point where it is possible to substantially collapse the energy and skills required to record, edit and publish transcripts of public meetings of all kinds.
We are planning to develop software that uses off-the-shelf voice recognition technologies to produce rough drafts of transcripts that can then be edited and published through a web browser. Our role will not be in working on the voice recognition itself, but rather on making the whole experience of setting out to record, transcribe and publish a speech or session as easy, fast and enjoyable as possible. And we will build tools to make browsing and sharing the data as nice as we know how. All this fits within our Components strategy.
But mySociety cannot ourselves go to all these meetings. And it appears exceptionally unlikely that councils will want to pay for official transcribers at this point in history. So what we’re asking today is for interest from individuals – inside or outside councils – willing to have a go at transcribing meetings as we develop the software.
It doesn’t have to be definitive to be valuable
Hansard is the record of pretty much everything that gets said in Parliament. This has led to the idea that if you don’t record everything said in every session, your project is a failure. But if Wikipedia has taught us anything, it is that starting small – producing little nuggets of value from the first day – is the right way to get started on hairy, ambitious projects. We’re not looking for people willing to give up their lives to transcribe endlessly and for free – we’re looking for people for whom having a transcript is useful to them anyway, people willing to transcribe at least partly out of self interest. We’re looking for these initial enthusiasts to start building up transcripts that slowly shift the idea of what ‘normal’ conduct in local government is.
Unlike Wikipedia we’re not really talking about a single mega database with community rules. Our current plans are to let you set up a database which you would own – just as you own your blog on Blogger or WordPress, perhaps with collaborators. Maybe you just want to record each annual address of the Lord Mayor – that’s fine. We just want to build something that suits many different people’s needs, and which lifts the veil on so much hidden decision making in this country.
Get in touch
The main purpose of this post is to tell people that mySociety is heading in this direction, and that we’d like you along for the ride. We won’t have a beta to play with for a good few months yet, but we are keen to hear from anyone who thinks they might be an early adopter, or who knows of other people who might want to be involved.
And we’re just as keen to hear from people inside councils as outside, although we know your hands are more tied. Wherever you sit – drop us a line and tell us what sort of use you might want to make of the new technology, and what sort of features you’d like to see. We’ll get back in touch when we’ve something to share.
The local press in Islington has just reported the accidental release of quite a bit of sensitive personal data by Islington council.
One of our volunteers, Helen, was responsible for spotting that Islington had made this mistake, and so we feel it is appropriate to set out a summary of what happened, to inform journalists and citizens who may be interested.
Note – Concerned residents should contact Islington Council or the Information Commissioner’s Office.
On 27th May a user of our WhatDoTheyKnow website raised an FOI request to Islington Borough Council. On the 26th June the council responded to the FOI request by sending three Excel workbooks. Unfortunately, these contained a considerable amount of accidentally released, private data about Islington residents. In one file the personal data was contained within a normal spreadsheet, in the two other workbooks the personal data was contained on four hidden sheets.
All requests and responses sent via WhatDoTheyKnow are automatically published online without any human intervention – this is the key feature that makes this site both valuable and popular. So these Excel workbooks went instantly onto the public web, where they seem to have attracted little attention – our logs suggest 7 downloads in total.
Shortly after sending out these files, someone within the the council tried to delete the first email using Microsoft Outlook’s ‘recall’ feature. As most readers are probably aware – normal emails sent across the internet cannot be remotely removed using the recall function, so this first mail, containing sensitive information in both plain sight and in (trivially) hidden forms remained online.
Unfortunately, this wasn’t the only mistake on the 26th June. A short while later, the council sent a ‘replacement’ FOI response that still contained a large amount of personal information, this time in the form of hidden Excel tabs. As you can see from this page on the Microsoft site , uncovering such tabs takes seconds, and only basic computer skills.
At no point on or after the 26th June did we receive any notification from Islington (or anyone else) that problematic information had been released not once, but twice, even though all mails sent via WhatDoTheyKnow make it clear that replies are published automatically online. Had we been told we would have been able to remove the information quickly.
It was only by sheer good fortune that our volunteer Helen happened to stumble across these documents some weeks later, and she handled the situation wonderfully, immediately hiding the data, asking Google to clear their cache, and alerting the rest of mySociety to the situation. This happened on the 14th July, a Saturday, and over the weekend mySociety staff, volunteers and trustees swung into action to formulate a plan.
The next working day, Monday 16th July, we alerted both Islington and the ICO about what had happened with an extremely detailed timeline.
The personal data released by Islington Borough Council relates to 2,376 individuals/families who have made applications for council housing or are council tenants, and includes everything from name to sexuality. It is for the ICO, not mySociety, to evaluate what sort of harm may have resulted from this release, but we felt it was important to be clear about the details of this incident.
Quick question – don’t think too hard about it: what is Amazon?
At one level, Amazon is the world’s largest online retailer, a public company listed on the NASDAQ. At another level – the physical – it is a collection of over 50,000 employees, hundreds of warehouses and zillions of servers.
But for most people Amazon is fundamentally a website.
Sure, it’s an extremely impressive website that can send you parcels in the post, and which can relieve you of money with terrifying ease. But to most people the company has very little reality beyond the big white-blue-and-orange website and the brown cardboard packages.
Vanishing Act
The same process is happening to the bits of the government that I interact with – the physical reality of bricks and mortar and people and parks is starting to disappear behind the websites.
Government is increasingly a thing I don’t have any mental images of. I don’t know what my local council looks like, nor am I even clear where it is. I’m sure you all have plenty of interactions with HM Revenue and Customs, but do you know where it is or what it looks like?
Increasingly, when I form a mental image of a branch of government in my head, what I see is the website. What else am I supposed to picture?
Governments no longer just ‘own‘ websites, they are websites.
Heartless Bourgeois Pig
Wait! Stop shouting! I know how this sounds.
I am not so out of touch that I don’t know that there are plenty of people out there who are only too familiar with the physical manifestations of government. They see the government as manifested through prison, or hospital, or the job centre. They have no problem forming a vivid mental image of what government means: a waiting room, a queue, a social worker.
And I also know that most of the poorest people in the UK aren’t online yet. It’s one of the great challenges for our country in the next decade.
But…
The majority of citizens don’t have deep, all encompassing, everyday interactions with the state – at most they drop their kids at school every day, or visit the GP a few times a year. That’s as physically close as they get.
To these people, interacting with government already feels somewhat like interacting with Amazon. It sends them benefits, passports, recycling bins, car tax disks from mysterious dispatch offices and it demands money and information in return. The difference is in emotional tone – the Amazon online interactions tend to be seamless, the government online interactions either painful or impossible – time to pick up the phone.
Increasingly, when a modern citizen looks at a government website, they’re literally seeing the state. And if what they see is ugly, confusing or down-right-broken, increasingly that’s how they’re going to see the state as a whole.
This change in public perception means that a previously marginal problem (bad websites) is now pointing towards a rather more worrying possibility. As government websites continue to fall behind private sector websites, governments will slowly look less and less legitimate – less and less like they matter to citizens, less and less like we should be paying any taxes to pay for them. Why pay for something you can’t even navigate?
It is time for the directors and CEOs of public bodies everywhere to wake up to this possibility, before the ideologues get hold of it.
Governments have the wrong management structures for a digital future
I don’t buy the argument that government websites are bad because all the ubermensch have gone off to work for the private sector. The public sector can often teach the private sector a lot about information design, like British road signs and tube maps, which are fantastic. And, of course, there’s the super team at Gov.uk, who represent the kind of change I’m writing about here.
The real difference is one of management structure and focus. At Amazon, CEO Jeff Bezos and his executive colleagues worry all the time about whether their site or app or Kindle are as good as the competitors. But in central and local governments around the world, the top bosses do not stress every day about whether the user experience of their website is up to scratch, or whether conversion rates are lower than desirable.
The main reason that they don’t worry is because their management boards don’t historically contain anyone whose job it is to worry about the performance of digital services. A council chief exec will worry about finance because their finance director will constantly be nagging them about money. But a council CEO won’t be worrying about whether 10,000 people left their website bitterly disappointed last week, because such issues are not ‘normal things to discuss’ at a board level.
Getting digital people to the top table
The solution, at least in the near term – is to recruit or promote people with digital remits and experience right to the top tier of decision making in government bodies. It means creating new roles like ‘CIO’ or ‘Head of Digital’ which have the same seniority as ‘Head of Adult Social Care’ or ‘Head of HR’. And it means empowering those people to make painful changes that are required to make digital services become brilliant and user-centric.
Clearly, this presents dangers. How do you know what powers to give the new role? How do you stop them damaging critical services? And, most problematic of all – how can you tell that a digital expert isn’t a charlatan? After all, they have niche expertise that you don’t have – how are you supposed to sniff them out?
The answer is that it isn’t easy, and that a lot of knowledge sharing and learning from mistakes will be required. As a shameless plug – we can help here – we can help vet candidates and define their roles in Britain and abroad. But none of this hides the fact that becoming digital – learning to run a public organisation that is a website, will be a fraught affair. The reward, though, is nothing less than helping to guarantee the ongoing legitimacy of government (quite apart from all the happier customers). To me that seems well worth going through some pain for.
If you live anywhere in Britain, it won’t have escaped your attention that it’s been raining a bit, recently.
This has been causing quite a bit of flooding. And when flooding happens, people need to know if it is going to affect them.
Unfortunately, the Environment Agency flood warning website leaves something to be desired. It is, quite frankly, a usability dogs’ breakfast, with problems including:
- It doesn’t answer the main question: Most users arriving at this page simply want to know if they might be in danger. The page should be all about answering that question.
- It is trying to serve national and local needs: Information about flooding across the whole country might be useful to journalists or civil servants, but it shouldn’t be the main element.
- Clutter, clutter: A massive grid of numbers which don’t really mean anything, plus lots of sidebar links.
- Confusing graphics: The page contains a national map which doesn’t actually make it clear that the colours relate to the seriousness of flooding, or that it provides links to further content.
There are also some non-design problems with the postcode lookup, but today we want to stick to just the design issues.
Not just moaning minnies
At mySociety we try to be constructive in our criticism, and so whilst the flood waters are still draining from many people’s homes, we thought that we could do something positive. We want to show that a flood warning page could be an exemplar of clear, user-centered information design. So we made a mockup.
Some of the improvements we’d like to point out are:
- A big page title that makes it obvious what this page is, and the fact that it is official information.
- All the main elements on the page are now focussed on the most likely needs of potential flood victims – journalists can follow a link to a different page for their needs.
- We’ve removed roughly 90% of the links on the page for clarity.
- We’ve removed all numerical data because it wasn’t adding value. Nobody can know if ’5 warnings’ is a lot or a little without some context. As a nod to the overall context we’ve put in a simple graph, similar to a sparkline.
- It presents a clear button to click on if you’re actually endangered by a flood.
- It gives you a way to find out if other people near you are talking about local flooding via social media.
We hope you like this. It’s just the product of a couple of hours’ work, so if you have any suggestions on how it could be better, please let us know.
And, of course, we’re always happy to do similar work for other people.
All of us at mySociety love the fact that there are so many interesting new civic and democratic websites and apps springing up across the whole world. And we’re really keen to do what we can to help lower the barriers for people trying to build successful sites, to help citizens everywhere.
Today mySociety is unveiling MapIt Global, a new Component designed to eliminate one common, time-consuming task that civic software hackers everwhere have to struggle with: the task of identifying which political or administrative areas cover which parts of the planet.
As a general user this sort of thing might seem a bit obscure, but you’ve probably indirectly used such a service many times. So, for example, if you use our WriteToThem.com to write to a politician, you type in your postcode and the site will tell you who your politicians are. But this website can only do this because it knows that your postcode is located inside a particular council, or constituency or region.
Today, with the launch of MapIt Global , we are opening up a boundaries lookup service that works across the whole world. So now you can lookup a random point in Russia or Haiti or South Africa and find out about the administrative boundaries that surround it. And you can browse and inspect the shapes of administrative areas large and small, and perform sophisticated lookups like “Which areas does this one border with?”. And all this data is available both through an easy to use API, and a nice user interface.
We hope that MapIt Global will be used by coders and citizens worldwide to help them in ways we can’t even imagine yet. Our own immediate use case is to use it to make installations of the FixMyStreet Platform much easier.
Thanks OpenStreetMap!
We’re able to offer this service only because of the fantastic data made available by the amazing OpenStreetMap volunteer community, who are constantly labouring to make an ever-improving map of the whole world. You guys are amazing, and I hope that you find MapIt Global to be useful to your own projects.
The developers who made it possible were Mark Longair, Matthew Somerville and designer Jedidiah Broadbent. And, of course, we’re also only able to do this because the Omidyar Network is supporting our efforts to help people around the world.
From Britain to the World
For the last few years we’ve been running a British version of the MapIt service to allow people running other websites and apps to work out what council or constituency covers a particular point – it’s been very well used. We’ve given this a lick of paint and it is being relaunched today, too.
MapIt Global is also the first of The Components, a series of interoperable data stores that mySociety will be building with friends across the globe. Ultimately our goal is to radically reduce the effort required to launch democracy, transparency and government-facing sites and apps everywhere.
If you’d like to install and run the open source software that powers MapIt on your own servers, that’s cool too – you can find it on Github.
About the Data
The data that we are using is from the OpenStreetMap project, and has been collected by thousands of different people. It is licensed for free use under their open license. Coverage varies substantially, but for a great many countries the coverage is fantastic.
The brilliant thing about using OpenStreetMap data is that if you find that the boundary you need isn’t included, you can upload or draw it direct into Open Street Map, and it will subsequently be pulled into MapIt Global. We are planning to update our database about four times a year, but if you need boundaries adding faster, please talk to us.
If you’re interested in the technical aspects of how we built MapIt Global, see this blog post from Mark Longair.
Commercial Licenses and Local Copies
MapIt Global and UK are both based on open source software, which is available for free download. However, we charge a license fee for commercial usage of the API, and can also set up custom installs on virtual servers that you can own. Please drop us a line for any questions relating to commercial use.
Feedback
As with any new service, we’re sure there will be problems that need sorting out. Please drop us an email, or tweet us @mySociety.
As you may already be aware, mySociety is putting considerable effort into making it super-easy to set up versions of our websites FixMyStreet and WhatDoTheyKnow in other countries.
These ‘websites in a box’ are a key part of our strategy to help people develop more successful civic and democratic websites around the world, but they are only the first half of our plan. Today I wanted to talk about the other half.
The Challenge
There are some use-cases for software in which most people are entirely happy to take some software off the shelf, press ‘Go’, and start using it. WordPress is a good example, and so is Microsoft Office.
However, there are some kinds of social issues that vary so much between different countries and regions that we believe one-size-fits-all tools for attacking them are impracticable.
This problem is particularly acute in the arena of sites and apps that allow people to track the activities of politicians. In this area there are several dozen different sites globally, almost all of which are powered by software that was written bespoke for that particular usage.
What drives this pattern of people re-writing every site from scratch is that people in different places care about different aspects of politics. In some countries what really counts is how politicians vote, in others the crux is campaign finance contributions, in others it is information on who has criminal records, and in others still it is whether public money has been vanishing suspiciously.
To build an off-the-shelf software platform that could handle all this data equally well in every country would be an immense coding task. And more important than that, we believe that it would create a codebase so huge and complex that most potential reusers would run away screaming. Or at least ignore it and start from scratch.
In short – we don’t believe there can be a WordPress for sites that monitor politicians, nor for a variety of other purposes that relate to good governance and stronger democracies.
The Waste
We believe that the wrong answer to this challenge is to just say “Well then, everyone should build their own sites from scratch.” Over the years we at mySociety have been witness to the truly sad sight of people and organisations around the world wearing themselves out and blowing their budgets just trying to get the first version of a transparency website out the door. All too often they fail to create popular, long lasting sites because the birthing process is just so exhausting and resource-consuming that there’s nothing left to drive the sites to success. Often they don’t even get to launch.
A painful aspect of this problem is that the people who work on such sites are genuine altruists who are trying to solve serious problems in their part of the world; too much of their passion and energy is used up on building tools, when there’s still so much work beyond that that’s needed to make such sites successful. However, as we pointed out above, giving them a complete package on a plate isn’t an option. So what can we do?
Our Proposed Answer – The Components
We start from the following observation: coders and non-coders like simple, minimal, attractive tools that help them achieve bigger goals. Simple tools don’t make anyone run away screaming – they encourage exploration and deliver little sparkles of satisfaction almost immediately. But simple tools have to be highly interoperable and reliable to form the foundation of complex systems.
Our plan is to collaborate with international friends to build a series of components that deliver quite narrow little pieces of the functionality that make up bigger websites. These include:
- PopIt – A Component to store and share the names of politicians, and the jobs they have.
- MapIt – A Component to store and share information on the locations of administrative boundaries, like counties, regions or cities.
- SayIt – A Component to store and share information on the words that public figures say or put out in writing.
There will be more, possibly many more. Our goal is to radically collapse the time it takes to build new civic and democratic (and possibly governmental) websites and apps, without putting constraints on creativity.
Characteristics of each Component
There are some crucial architecture decisions that have been baked into the Components, to truly make them ‘small pieces loosely joined’.
- Each Component is fundamentally a tool for storing and sharing one or two kinds of common data – they’re intentionally minimalist.
- As a developer, you just use the Components that make sense for your goals – you simply don’t have to look at or learn about the Components that contain functionality that doesn’t matter to you.
- You don’t have to install anything to get started – you can always begin by playing with a hosted Component.
- We won’t impose our taste in programming languages on you. You can code your website in whatever language you want. The Components are not ‘modules’ - they don’t plug into some overarching framework like Drupal or WordPress. They are stand-alone tools which just present you data over REST APIs, and which you can write data into using REST APIs.
- Each Component’s data structures will offer as much flexibility as makes sense given the goal of keeping each Component really good at one or two tasks. We’ll listen to feedback carefully to get this right.
- Each Component has a clean, simple web front end so you can explore the data held in a store without having to write lots of SQL queries. Often you will be able to edit the data this way, too.
- Get started in seconds – each Component offers at least some functionality which is available inside a minute after getting involved.
- Non coders are welcome – we are building the Components so that non-coders can start gathering, editing and sharing data straight away, possibly long before they are in a position to launch a ‘real site’.
- Data can be added to the Components both through write APIs and through manual editing interfaces, suitable for non-coders.
- Learn from our mistakes – it is really easy to get the wrong data structure for civic, democratic or governmental data. Good practice data structures are baked into the Components, to save you pain later.
- Use our hosted versions, or install open source code locally. It will normally be quicker to get started in using the Components in a hosted environment, but if you want to run them locally, you’re entirely welcome. The code will be open source, and we’ll work hard to make sure it’s attractive and easy to install.
- The Components will talk to each other, and to the rest of the web using simple open schemas which will evolve as they are built. Where possible we’ll pick up popular data standards and re-use those, rather than building anything ourselves.
What the Components Aren’t
Sometimes in life it can be easier to describe things by what they aren’t:
- The Components are definitively not modules in a framework or platform. Each one is totally independent, and they will frequently be written in different languages – partly to force us to ensure that the APIs are truly excellent.
- The Components aren’t either Hosted or Local, they’re both. We’ll always offer a hosted version and a downloadable version, and you’ll always be able to move any data you have stored on the hosted versions down to your local copies.
- The Components aren’t all about mySociety. We’re planning to build the first ones in conjunction with some friends, and we’ll be announcing more about this soon. We want the family of Components to be jointly owned by a group of loving parents.
When can I see some of the Components in Action?
We’ll be blogging more about that tomorrow…
Footnote – To see the provenance of the extremely useful ’small pieces loosely joined’ concept, see this.
Speaking at today’s Activate conference in London, Stephen King from the Omidyar Network just announced a grant of extraordinary importance to mySociety.
Over the next three years, the Omidyar Network is granting mySociety an amazing $2.9m.
This unprecedented donation is tied to clear targets which, when translated relate to the following goals:
- Internationalising our current British websites, and helping people around the world to build sites and apps that will drive greater transparency and accountability.
- Growing an ever-stronger commercial team, to help cover our costs which remain unfunded (still substantial).
- Continuing to grow the impact of our UK sites.
If you’re someone who’s ever given us £5 or £10 to support our work, or who’s given your time volunteering on any of our projects, we have a special message for you: we would never have been able to get to the sort of scale of support that today’s grant represents if people like you didn’t believe in us all along. Thank you, and thank you for your continued support – our growing ambitions to make ever greater positive impacts on the world means we need more friends than ever, not fewer.
We’re Open for Business, Partnerships and Conversations
The overall impact of this grant, plus continued support from groups like the Open Society Foundation, Hivos, the Esmée Fairbairn Foundation, Indigo Trust – and many smaller donations – is a huge increase in our overall capacity. We can build more software, help more partners, work with more clients, give more advice.
So if you’ve thought about working with mySociety, or using our open source tech for your own goals, there’s never been a better moment. Get in touch or talk to us on Twitter.
One of the most common grumbles heard within the political and governmental classes is that the public doesn’t understand the need for compromise.
The argument goes something like this: left to themselves the public will vote for low tax and high public spending, resulting in eventual bankruptcy and collapse. The State of California is usually wheeled out as exhibit A here.
Assuming that this is even true, I find it hard to blame the public for a general lack of awareness about the compromises involved in running a functional government.
This is not because big budgets are complicated (although they are) but because most governments waste hundreds of thousands of opportunities a day to explain the nature of compromises. They waste them because they’re still thinking about the world from a paper-centric mindset.
Linking to explanations
My argument is this: key compromises or decisions should be linked to from the points where people obtain a service, or at the points where they learn about one. If my bins are only collected once a fortnight, the reason why should be one click away from the page that describes the collection times.
Currently, in order to obtain an explanation for why a service functions as it does, I’d probably have to pick up the phone to my local councillor, or use this handy service to make a few FOI requests. In terms of effort and clicks, these explanations describing why a service is like it is are so far away from the service itself that they might as well be on Mars.
Here are some of the wasted opportunities to explain which I would like to see seized upon:
- A “Why aren’t there more bin collections?” link on local government waste pages, linking through to an explanation about council budgets, what would have to be sacrificed to have more bin collections, and who made the decision to adopt the current compromise.
- Updates by local governments on FixMyStreet that say “We’re not going to fix this problem because it wouldn’t be good value for money”, linking through to an appropriate analysis about money spent on street fixing, versus other things.
- On the NHS’s ‘Choose and Book’ website, I’d like to see links saying “Why can’t I get an appointment sooner?” These would then be linked to data on NHS waiting lists, budget constraints and specific decisions that set the current availability.
Obviously cynics out there will say that governments don’t want people to know that they can’t solve all the world’s ills – and that they want to preserve a mystique of omnipotence, so that people will be miserably grateful to them for the bounty bestowed. In this model, governments don’t offer explanations lest citizens see them as merely mortal, and boot them out.
Now, I don’t know about you, but servile gratitude and illusions of infinite power doesn’t sound much like the current attitude to government from most people I know. We live in politically disillusioned times where many people worry if the government can actually fix anything, never mind everything.
If ever there was a time to start routinely explaining to citizens that government is a process of ceaseless compromises it is now, in the hard times. There are plenty of those around the world right now.
I believe that citizens could be both more forgiving of governments, and more empowered to demand change if services were closely connected to explanations of why compromises have been made. I think that the reason it hasn’t happened before isn’t really politics: it’s simply because it wouldn’t have been possible on paper. On paper you can’t link through to an animated narrative, or a set of votes, or a transcript of a key decision. I think the main reason we don’t connect services with explanations is because governments haven’t really grokked the meaning of simple linking yet – not really. I’m looking for the first government, national or local, willing to give it a shot.
