-
Today sees the official launch of FixMyStreet’s open source codebase as a proper tool that we hope people will want to deploy in cities and countries around the world. It is based on FixMyStreet.com which we believe is the most usable, most mature street problem reporting tool in the world, but which is only available to British users.
We’re shouting about this launch a bit because we need your help to make the service ever better. First, we need feedback from programmers about whether we’ve got the install process right – whether it’s as easy and clear as we want it to be. And for non-coders who want to get involved, we want to ask for help with the process of translating the site’s text into different languages.
Over the years there have been many copies of FixMyStreet set up in many countries, often using the site’s original name, but always written by developers from scratch. We’re delighted to have inspired people, but all too often the people trying to build copies have stumbled as they realise just how hard it is to build a tool like this with the polish that users expect. We think that people everywhere would be better off if they could have a local FixMyStreet that was really usable, and really connected to the right people.
So we’re very happy to be able to open up a codebase that has been extensively modified in the last year, to help users around the world manage easy, successful deployments. Steps we have taken include:
- Putting the translation text into Transifex, so that non-technical translators can get started whenever they feel like it
- Developing Amazon Machine Images so people who want to tinker can get started in the minimum possible time
- Rewriting the entire codebase in order to make it a less confusing installation
- Building a global version of our MapIt political boundaries web service, so you can get going without having to wrestle administrative data out of your government before you get started.
Plus with the help of the wonderful OpenStreetMap, you can get maps without licensing hassles too.
Calling it version 1.0 is our way of saying two things. First, that the tool still has a lot of evolution left to do, and a long way to go before it is as good as we want it to become. But more ambitiously, calling it 1.0 is also our way of saying that it’s no longer just a codebase dumped into Github. It’s a real open source project, which we plan to support, and which we hope will make a real difference in the lives of ordinary people. Check it out.
-
As you may already be aware, mySociety is putting considerable effort into making it super-easy to set up versions of our websites FixMyStreet and WhatDoTheyKnow in other countries.
These ‘websites in a box’ are a key part of our strategy to help people develop more successful civic and democratic websites around the world, but they are only the first half of our plan. Today I wanted to talk about the other half.
The Challenge
There are some use-cases for software in which most people are entirely happy to take some software off the shelf, press ‘Go’, and start using it. WordPress is a good example, and so is Microsoft Office.
However, there are some kinds of social issues that vary so much between different countries and regions that we believe one-size-fits-all tools for attacking them are impracticable.
This problem is particularly acute in the arena of sites and apps that allow people to track the activities of politicians. In this area there are several dozen different sites globally, almost all of which are powered by software that was written bespoke for that particular usage.
What drives this pattern of people re-writing every site from scratch is that people in different places care about different aspects of politics. In some countries what really counts is how politicians vote, in others the crux is campaign finance contributions, in others it is information on who has criminal records, and in others still it is whether public money has been vanishing suspiciously.
To build an off-the-shelf software platform that could handle all this data equally well in every country would be an immense coding task. And more important than that, we believe that it would create a codebase so huge and complex that most potential reusers would run away screaming. Or at least ignore it and start from scratch.
In short – we don’t believe there can be a WordPress for sites that monitor politicians, nor for a variety of other purposes that relate to good governance and stronger democracies.
The Waste
We believe that the wrong answer to this challenge is to just say “Well then, everyone should build their own sites from scratch.” Over the years we at mySociety have been witness to the truly sad sight of people and organisations around the world wearing themselves out and blowing their budgets just trying to get the first version of a transparency website out the door. All too often they fail to create popular, long lasting sites because the birthing process is just so exhausting and resource-consuming that there’s nothing left to drive the sites to success. Often they don’t even get to launch.
A painful aspect of this problem is that the people who work on such sites are genuine altruists who are trying to solve serious problems in their part of the world; too much of their passion and energy is used up on building tools, when there’s still so much work beyond that that’s needed to make such sites successful. However, as we pointed out above, giving them a complete package on a plate isn’t an option. So what can we do?
Our Proposed Answer – The Components
We start from the following observation: coders and non-coders like simple, minimal, attractive tools that help them achieve bigger goals. Simple tools don’t make anyone run away screaming – they encourage exploration and deliver little sparkles of satisfaction almost immediately. But simple tools have to be highly interoperable and reliable to form the foundation of complex systems.
Our plan is to collaborate with international friends to build a series of components that deliver quite narrow little pieces of the functionality that make up bigger websites. These include:
- PopIt – A Component to store and share the names of politicians, and the jobs they have.
- MapIt – A Component to store and share information on the locations of administrative boundaries, like counties, regions or cities.
- SayIt – A Component to store and share information on the words that public figures say or put out in writing.
There will be more, possibly many more. Our goal is to radically collapse the time it takes to build new civic and democratic (and possibly governmental) websites and apps, without putting constraints on creativity.Characteristics of each Component
There are some crucial architecture decisions that have been baked into the Components, to truly make them ‘small pieces loosely joined’.
- Each Component is fundamentally a tool for storing and sharing one or two kinds of common data – they’re intentionally minimalist.
- As a developer, you just use the Components that make sense for your goals – you simply don’t have to look at or learn about the Components that contain functionality that doesn’t matter to you.
- You don’t have to install anything to get started – you can always begin by playing with a hosted Component.
- We won’t impose our taste in programming languages on you. You can code your website in whatever language you want. The Components are not ‘modules’ – they don’t plug into some overarching framework like Drupal or WordPress. They are stand-alone tools which just present you data over REST APIs, and which you can write data into using REST APIs.
- Each Component’s data structures will offer as much flexibility as makes sense given the goal of keeping each Component really good at one or two tasks. We’ll listen to feedback carefully to get this right.
- Each Component has a clean, simple web front end so you can explore the data held in a store without having to write lots of SQL queries. Often you will be able to edit the data this way, too.
- Get started in seconds – each Component offers at least some functionality which is available inside a minute after getting involved.
- Non coders are welcome – we are building the Components so that non-coders can start gathering, editing and sharing data straight away, possibly long before they are in a position to launch a ‘real site’.
- Data can be added to the Components both through write APIs and through manual editing interfaces, suitable for non-coders.
- Learn from our mistakes – it is really easy to get the wrong data structure for civic, democratic or governmental data. Good practice data structures are baked into the Components, to save you pain later.
- Use our hosted versions, or install open source code locally. It will normally be quicker to get started in using the Components in a hosted environment, but if you want to run them locally, you’re entirely welcome. The code will be open source, and we’ll work hard to make sure it’s attractive and easy to install.
- The Components will talk to each other, and to the rest of the web using simple open schemas which will evolve as they are built. Where possible we’ll pick up popular data standards and re-use those, rather than building anything ourselves.
What the Components Aren’t
Sometimes in life it can be easier to describe things by what they aren’t:
- The Components are definitively not modules in a framework or platform. Each one is totally independent, and they will frequently be written in different languages – partly to force us to ensure that the APIs are truly excellent.
- The Components aren’t either Hosted or Local, they’re both. We’ll always offer a hosted version and a downloadable version, and you’ll always be able to move any data you have stored on the hosted versions down to your local copies.
- The Components aren’t all about mySociety. We’re planning to build the first ones in conjunction with some friends, and we’ll be announcing more about this soon. We want the family of Components to be jointly owned by a group of loving parents.
When can I see some of the Components in Action?
We’ll be blogging more about that tomorrow…
Footnote – To see the provenance of the extremely useful ‘small pieces loosely joined’ concept, see this.
-
As you might know, we’ve currently got an open-call for new developers, we’re hiring quite a bit in the next six months.
Thus far our list of people interested in the job contains no women’s names at all – zip, zero, zilch – despite us having taken soundings on how to get a more diverse sample of applicants.
I’m really, really not OK with this. I understand the gender imbalance in tech as well as anyone, but I interpret this as ‘mySociety hasn’t reached out well enough’, not ‘blame the women for not applying’.
So my question to you, the world at large, is this: what can we do right now, or this week anyway, to get some women’s names on this list before we start to vet the CVs?
Applications are still very definitely open, so anyone – male, female or other – who’d like to apply should see the original blog post for how to go about it.
-
I must admit that I’m pretty happy to announce mySociety’s plans to build our first major new non-comercial website since WhatDoTheyKnow.com launched in 2008. Late in 2010 we plan to launch FixMyTransport , a site focussed on connecting and empowering people who share transport problems of different kinds. The fantabulous Louise Crow will be lead developer.
Crucially, we at mySociety are under no illusions that it is an order of magnitude more difficult to get a new ticket machine in your station than it is to get your local council to fill a pothole (FixMyStreet surveys report 2371 problems fixed in the last month alone). The difficulty of achieving even minor changes to transport services and infrastructure is why we are simultaneously announcing our plan to build FixMyTransport on top of a major new back end system called Project Fosbury.
Project Fosbury is about helping people get over difficult obstacles. It is a modular platform for breaking down a complicated civic task into pieces which can then be allocated to one or more people. So someone asking their council to change the timing on some traffic lights might be allocated the tasks of:
- Writing to their councillors
- Obtaining local policy on traffic light timings from the council
- Getting people to join a mini-campaign group
- Videoing the problem
- Sending a letter to a local newspaper
Each task will ultimately be carried out entirely within a joined up infrastructure, each module being built to mySociety’s habitually stringent rule that “it must be easy and satisfying to someone who’s never engaged politically before”. We will work to create incentive structures, peer pressure, and hopefully a sense of fun. There will be a single public home page for each mini campaign, showing recent activities on the site, as well as integrating with external social media. We hope to repeat the FixMyStreet phenomena where some ‘insoluble’ problems suddenly become soluble once they’re in the public domain.
Now for the credit where it is due. mySociety’s sysadmin Keith Garrett suggested FixMyTransport back in January 2008.
The actual mechanics of breaking the problem into pieces (the idea that became Project Fosbury) came from a wide discussion at our retreat, with excellent suggestions coming particularly from Richard Pope. But the more general idea that the Internet hasn’t yet produced a really good system for bringing people together to solve everyday problems (as opposed to chat, or win the US presidency) came from numerous Call for Proposals submissions, including ones from Mark W, Rob Shorrock, Peter Silverman, Mahmood Choudhury and more.
mySociety will be building this site using money donated by people like you, profits from commercial projects, and any specific funding we can raise around it. If you know of anyone or any organisation that you think might like to support FixMyTransport or Project Fosbury, please do get in touch.
-
Over the last weekend of November 2009 a group of 21 mySociety staff, volunteers and trustees went to a house outside of Bristol to wrestle with the question of what mySociety should build over the next 12 months. This was the fourth time we’ve done it, and these meetings have become a crucial part of our planning. This year, we were talking not just about what new features to add to our current sites, but also about the possibility of building an entirely new website for the first time in a couple of years. The discussions were lively and passionate because we know we have a lot to live up to: not only is our last major new site (WhatDoTheyKnow) likely to cross the 1 million unique visitors threshold this year, but we understood that there were people and organisations who weren’t there who would be counting on us to set the bar high.
A chunk of the weekend involved vetting the 227 project ideas that were proposed via our Call for Proposals. I’m going to write a separate post on our thoughts about that process, but if you look at the list below you may spot things that were submitted in that call.
One nice innovation that helped us whittle down our ideas from unmanageable to manageable numbers was a pairwise comparison game to help us prioritise ideas, build custom for the occasion by the wonderful and statistically talented Mark Longair. In other words, we used the technique that powers KittenWar.com to help decide our key strategic priorities for the next year: after all , if we don’t, who will?
By the end of the weekend we had not battened everything down – there are too many uncertainties around how much time we will have, and some key ideas that need more speccing. However, we were able to put various things into different buckets, marked according to size and degree of certainty. So here goes:
1. Things which were decided at the last retreat, which we are definitely building, and which (mostly) need doing before next year’s stuff starts getting built
- A top level page for each bill on TheyWorkForYou
- Future business (ie the calendar) for events in the House of Commons, including a full set of alerting options.
- Video clips on MP pages on TheyWorkForYou
- Epicly ambitious election data gathering and quiz building with the lovely volunteers at DemocracyClub
2. Small new things that we are very probably doing because there was lots of consensus
- Publish a standard that councils can use to post problems like potholes in their databases to FixMyStreet and other similiar sites.
- Template requests in WhatDoTheyKnow so that users are strongly encouraged to put in requests that are well structured.
- After the next general election, email new MPs with various bits of info of interest to them including their new login to HearFromYourMP, their page on TheyWorkForYou, explanation of how WriteToThem protects them from spam and abuse, a double check that their contact details are correct, and a introduction to the fact that we record their correspondance responsiveness and voting records.
- Add to WhatDoTheyKnow descriptions about what kind of public authority a specific entity is (ie ‘school’, ‘council’) and the information they are likely to hold if FOIed.
- Show divisions (parliamentary votes) properly on debate pages on TheyWorkForYou, ie show the results of a vote on the same page as the debate where the issue was discussed, with full party breakdowns on each division.
- Add “How to benefit from this site” page on TheyWorkForYou, inspired by OpenCongress.org
- Help Google index TheyWorkForYou faster by creating a sitemap.xml file that is dynamically updated.
- Using the data we expect to have from DemocracyClub’s volunteers, send a press release about every new MP and to all relevent local newspapers
- Incorporate a council GeoRSS problem feed into FMS
3. Slighty more time consuming things we are very probably doing because there was lots of consensus
- 1 day per month developer time that customer support guru Debbie Kerr gets to allocate as she see fit.
- Premium account feature on WhatDoTheyKnow to hide requests so that journalists and bloggers can still get scoops and then share their correspondance later.
- Add Select Committees to TheyWorkForYou, including email alerts on calls for evidence.
- Take professional advice on how to handle PR around the election
4. Much more time consuming things and things around which there is less consensus. NB – We do not currently have the resources to do everything on this list next year – it is an ambitious target list.
- Primary New site: TBA in a new post
- Add a new queue feature to WhatDoTheyKnow so that users can write requests, then table them for comments from other users and expert volunteers before they are sent to the public authority
- Relaunch our Volunteer tasks page on our sites, keep it populated with new tasks, specifically allocate resources to handhold potential volunteers. Allocate time to see if any of the ideas that we didn’t build could be parcelled into volunteer tasks.
- Secondary New site (if we have a lot more time than we expect): Exploit extraordinary richness of Audit Comission local government target data in a TheyWorkForYou-like fashion.
- FixMyStreet to become international with a) maps for most of the world b) easy to follow instructions explaining how to supply mySociety with the required data to us to enable us to turn on FixMyStreet in non UK countries or areas. This data would includ ie gettext powered text translation files, shapefiles of administrative boundaries, and lists of contact data.
- Add votes and proceedings to TheyWorkForYou (where they reveal statutory instrument titles that are not debated but where the law gets changed anyway)
- Carry out usability testing on TheyWorkForYou with then help of volunteer Joe Lanman – then implement changes recommended during a development process taking up to 10 days.
- Add to TheyWorkForYou questions that have been tabled in the house of commons but which haven’t been answered yet.
- Add a new interface for just councils so that they can say if a problem on FixMyStreet has changed status.
Phew. And that’s not even counting the projects we hope to help with in Central and Eastern Europe, our substantial commercial work, or the primary new site idea, which will be blogged in Part 2.
-
LazyWeb hear me – can I please have a service where I can send a one off fax from my web browser, paying per page with a credit card or paypal or similar?
I’ve just spent two hours examining numerous fax services, trying ones that failed to deliver the fax, rejecting ones over priced for my purposes, lugging failing faxing equipment around, and now facing a 40 minute freezing cross London journey to a fax machine because some idiot in a large company demands a fax as a proof of identity and the whole sodding internet can’t supply this noddy service (let’s leave aside for a moment the fact that anyone can buy this particular piece of identity verification for £1 making it entirely unfit for purpose).
You hear me, Lazyweb? I’m prepared to pay for this service even!
-
One of the nicest things about being involved with mySociety is seeing people in other countries starting similar organisations and building similar websites. After a Skype conversation with another eager hopeful last night, I thought I’d blog a bit about the things I think are most important to know if you’re just starting up. Here goes.
1. Absolutely the most totally essential thing is to be an organisation of amazing, politically minded coders, not an organization employing or contracting good coders*. Their skills are your lifeblood, their ideas your bread and butter, and finding the best civic hackers in your country and building your organization around them is the only path to success. And that means they should be making most of the day to day decisions, not you, you ignorant, arts-degree-clutching clot.
2.Ask the public what they think you should build. Not only will that give you access to more ideas than you have yourselves, it’ll engage people with you. Also, it’ll help you focus on the vital business of building sites that users want, not that YOU want.
3. Keep your cost base low, and put all the money you have into looking after your core staff and being nice to volunteers. Work on building a community of volunteers, even if most of them are really just friends rather than people putting in lots of time. Avoid renting offices, avoid non-essential non-coder staff, get people to donate serving infrastructure and bandwidth. Because building and running democratic websites is a fundamentally new area of human endeavor (not like blogging which at least has an analogy in journalism) there are basically no pre-existing funding streams for the type of work you’re about to do. You will have to create the buzz around yourselves that will lead to people wanting to fund you, and it will probably take years, if you get there at all (mySociety hasn’t quite done this yet, even at 5).
4. Ensure that the core of what you build can struggle on by even if your whole organization collapses. That means being open source, putting energy into sites that are as automated as possible, and making people excited about being volunteers.
5. If you aren’t pissing off at least some people all the time, you’ve probably been captured by the establishment.
6. Take whatever your first website plan is and remove 90% of the features you want. Then build it and launch it and your users will tell you which features they actually wanted instead. Build them and bask in the warm glow of appreciation.
*yes, yes I know I can’t code for toffee, and I’ve got an arts degree, but I’m still a geek, honest. (Proudest moment, working out that a batch of PCI network cards were unreliable because they’d come from the factory flashed with the EPROMs for the wrong hardware, and fixed it.)
-
In March this year Ethan Zuckerman gave a talk at ETech called The Cute Cat Theory of Digital Activism.
The summary of his theory is that web sites that successfully enable people to post nonsense like pics of their cats are the same systems that get used for activism.
The line that has motivated me to post, reflects something I’ve been noting for a while:
… She became an activist because she was forced to and she reached out for the tools she had access to – which hapened to be MSN spaces. MSN is heavily censored in China – it’s certainly not what we would have chosen for her. But you don’t get to choose the tools – activists use what’s at hand. It’s fine to build tools for activists, but even better to build tools for folks who don’t know they’re activists yet.
Then, as a sort of apology Ethan adds:
(In making this point, I should be very, very careful to point out that I have deep respect for tools that have been developed successfully for activist uses, tools like Martus or FrontlineSMS. My point is simply that there are huge numbers of web users who don’t yet think of themselves as activists who are likely to reach for the tools they have at hand, not to look specifically for tools designed for activists.)
I’m posting because I don’t think Ethan should be apologising, I think that those of us who run civic, democratic and activism websites should be thanking him for expressing a perhaps uncomfortable truth plainly. What Ethan’s pointing out is that for most people doing grass roots activism online means is using one of the megasites like Facebook, Blogger, MySpace, MSN or Hotmail to express your views to you friends and (hopefully) to more people. It’s bigger campaigns with higher starting capital that tend to use their own plaforms successfully, like Obama or Avaaz.
A few months ago it really struck me when reading Clay Shirky’s much praised Here Comes Everybody that even as he told the stories of a number of different bits of online activism, not a single one used a dedicated campaigning platform. It was Blogger, Twitter and email all the way.
Just to make things clear, I’m not posting this to moan that people don’t use the right platforms: after all mySociety doesn’t build anything that competes directly with Twitter, say. However, I would like to encourage some discussion about what role there is for smaller dedicated activist-coder groups like mySociety in a world where the first step on a just-born activist’s fight will almost always be their own IM, email, blogging or social networking tools.
Right now I’m trying to work out what sorts of path we should pursue in a universe where most users will behave like this. I don’t think the answer is as simple as ‘build widgets and plugins for all these sites’ either, none of our widgets has ever been as well used as simply providing permalinks to bits of debate in TheyWorkForYou which people link to in volumes. I hope this post can provoke some thoughts about how we can best strike a symbiotic relationship with the big beasts, especially seeking analogies from other sectors.
-
Google have launched a new UK politics site. mySociety were delighted to be asked to build a TheyWorkForYou widget for this site. There’s no doubt that this sort of modular re-purposing of our information is going to happen a lot more in the future, and it’s great to start out with the best of possible partners.
-
It’s been a long year.
A friend just pointed me to this by Danny Hillis from the Whole Earth Catalog, Winter 2000.
Like much of my generation, I grew up believing that I should try to “change the world,” presumably for the better. But I didn’t know how to do this. Looking at how other people have changed the world I concluded there are five ways of doing it:
- Some people change the world by imposing their will on it.
- Some people change the world by discovering a truth.
- Some people change the world by changing people’s minds.
- Some people change the world by creating things of great beauty.
- Some people change the world by making new tools for change.
Danny Hillis goes on to say that the last one, the making of new tools for change, is the one that appeals to him the most. I think my friend was just flattering me, as that is what mySociety tries to do.
Chris, who I started this year with and have sadly ended it without, would, I think, have been on the surface deeply cynical about even the last one. He’d have sad that tools could be used for evil – indeed, part of the point of campaigns that he took up from his heart, such as no2id, is to point out how computers are just such a tool that can be used for evil.
Nevertheless, he worked tirelessly to make other tools (e.g. WriteToThem), so that we could all use them for good. Hopefully, you can pick the tools that you make, choosing ones that maybe a few will use unwisely, but for which the many will make it up by using them wisely.
Happy Christmas!
May your New Year be full of will imposed judiciously, truth that both aches and thrills, minds changed to be more enlightened, beauty that is great, and more tools that everyone can use wisely for change.