Last time we updated you about Alaveteli professional, the Freedom of Information toolset for journalists that we’re building, we were just coming out of our discovery phase.
Since then, we’ve made strides through the alpha and early beta part of our development process. In alpha, the idea is to build dummy versions of the tool that work in the minimum way possible — no bells and whistles — to test concepts, and our assumptions. Having thought hard about the potential problems of Alaveteli Professional, now is the time for us to try the approaches that we believe will solve them, by making prototypes of how the tool might work and testing them with a very small group of users.
In the early stages of beta, our priority has been to get to the point where a Freedom of Information request can go through all its various processes, from composition to response, with the features that a journalist user would need. Once that’s in place, it allows us to add other features on top and see how they would integrate.
This pattern — discovery, alpha, beta, release — is a well-tested method by which to produce a final product that works as it should, while avoiding costly mistakes.
Alpha and beta testing, perhaps unexcitingly, are all about the reduction of risk: in the words of the startup mantra, it’s good to ‘fail fast’— or rather, it’s better to know early on if something doesn’t work, rather than spend time and money on something that doesn’t fit the bill.
So, for Alaveteli Professional, what are the risks that have been keeping us awake at night?
We think the biggest priority is to ensure that there’s actually added value for journalists in using a service like this. Clearly, the Freedom of Information process is already available to all, whether via our own site WhatDoTheyKnow, or directly.
We need to be able to demonstrate tangible benefits: that Alaveteli Professional can save journalists time; help them be more efficient in managing their requests; maybe help them get information that otherwise wouldn’t be released; and give them access to rich data they wouldn’t otherwise be able to access.
For all we said about failing fast, the alpha phase also meant committing to some fairly big technical decisions that, ideally, we wouldn’t like to reverse.
Decisions like, do we build the service into the existing Alaveteli codebase, or go for a new standalone one (we went for the former)? From the user’s point of view, should Alaveteli Professional look like a totally different site, or like a registration-only part of WhatDoTheyKnow (we chose the latter)?
And onto beta
As we move from alpha to beta, we’re finding out what happens when real users make real requests through the service, and making adjustments based on their feedback.
What do they think of the way we’ve implemented the ability to embargo requests – does it make sense to them? Do they trust us to keep embargoed requests private? Are they able to navigate between different interfaces in a way that seems intuitive? mySociety designer Martin has been figuring out how to take the cognitive load off the user and give them just the information they need, when they need it.
We’re also returning to prototyping mode to work out how to implement new features, like the ability to send round robin requests to multiple authorities, in an effective and responsible way. The other half of our design team, Zarino, has been showing us that a slideshow in presentation mode can be an effective tool for demonstrating how users might interact with an interface.
As we continue to round out the feature set in the UK, we’re also cooking up plans in the Czech Republic so that later in the year we can present the tools to a new audience of journalists there and again, use their feedback to make the tools more flexible so that they can be used in different jurisdictions.
As you can see, there’s lots going on, and we’re all really excited to be finally getting some real life users in front of the tools that we’ve been thinking about, and working on, for so many weeks. Don’t forget to sign up to the mailing list if you’d like to keep up with Alaveteli Professional as it develops.
You might already be enjoying one of the usability improvements that FixMyStreet version 2.0 has brought, though it’s possible that you haven’t given it much thought.
But for FixMyStreet, we hadn’t given much thought to letting you filter reports by more than one dimension, until Oxfordshire County Council suggested that it would be a useful feature.
For quite some time, you’d been able to filter by category and status (“Show me all pothole reports” or “Show me all ‘unfixed’ reports”), but this new functionality is more flexible.
You can now select multiple categories and multiple statuses simultaneously (“show me all pothole and graffiti reports that are unfixed or in progress”) — and all through the power of tickboxes.
If you’re a non-technical person, that’s all you need to know: just enjoy the additional flexibility next time you visit FixMyStreet. But if you are a coder, you might like to read more about how we achieved this feature: for you, Matthew has written about it over on the FixMyStreet Platform blog.
If you’ve used FixMyStreet recently — either to make a report, or as a member of a council who receives the reports — you might have noticed that the site’s automated emails are looking a lot more swish.
Where previously those emails were plain text, we’ve now upgraded to HTML, with all the design possibilities that this implies.
It’s all part of the improvements ushered in by FixMyStreet Version 2.0, which we listed, in full, in our recent blog post. If you’d like a little more technical detail about some of the thought and solutions that went into this switch to HTML, Matthew has obliged with in a blog post over on FixMyStreet.org.
Last year, when we were helping to develop YourNextMP, the candidate-crowdsourcing platform for the General Election, we made what seemed like an obvious decision.
We decided to use PopIt as the site’s datastore — the place from which it could draw information about representatives: their names, positions, et cetera. We’d been developing PopIt as a solution for parliamentary monitoring sites, but we reckoned it would also be a good fit for YourNextMP.
That turned out to be the wrong choice.
YourNextMP was up and running in time for the election, but at the cost of many hours of intensive development as we tried to make PopIt do what was needed for the site.
Once you’ve got an established site in production, changing the database it uses isn’t something you do lightly. But on returning to the codebase to develop it for international reuse, we had to admit that, in the words of mySociety developer Mark Longair, PopIt was “actually causing more problems than it was solving”. It was time to unpick the code and take a different approach.
Mark explains just what it took to decide to change course in this way, over on his own blog.
The post contains quite a bit of technical detail, but it’s also an interesting read for anyone who’s interested in when, and why, it’s sometimes best to question the decisions you’ve made.
FixMyStreet has been around for nearly nine years, letting people report things and optionally include a photo; the upshot of which is we currently have a 143GB collection of photographs of potholes, graffiti, dog poo, and much more. 🙂
For almost all that time, attaching a photo has been through HTML’s standard file input form; it works, but that’s about all you can say for it – it’s quite ugly and unfriendly.
We have always wanted to improve this situation – we have a ticket in our ticketing system, Display thumbnail of photo before submitting it, that says it dates from 2012, and it was probably in our previous system even before that – but it never quite made it above other priorities, or when it was looked at, browser support just made it too tricky to consider.
Here’s a short animation of FixMyStreet’s new photo upload, which also allows you to upload multiple photos:
For the user, the only difference from the current interface is that the photo field has been moved higher up the form, so that photos can be uploading while you are filling out the rest of the form.
Personally, I think this benefit is the largest one, above the ability to add multiple photos at once, or the preview function. Some of our users are on slow connections – looking at the logs I see some uploads taking nearly a minute – so being able to put that process into the background hopefully speeds up the submission and makes the whole thing much nicer to use.
When creating a new report, it can sometimes happen that you fill in the form, include a photo, and submit, only for the server to reject your report for some reason not caught client-side. When that happens, the form needs to be shown again, with everything the user has already entered prefilled.
There are various reasons why this might happen; perhaps your browser doesn’t support the HTML5 required attribute (thanks Safari, though actually we do work around that); perhaps you’ve provided an incorrect password.
However, browsers don’t remember file inputs, and as we’ve seen, photo upload can take some time. From FixMyStreet’s beginnings, we recognised that re-uploading is a pain, so we’ve always had a mechanism whereby an uploaded photo would be stored server side, even if the form had errors, and only an ID for the photo was passed back to the browser so that the user could hopefully resubmit much more quickly.
This also helped with reports coming in from external sources like mobile phone apps or Flickr, which might come with a photo already attached but still need other information, such as location.
Of course there were edge cases and things to tidy up along the way, but if the form hadn’t taken into account the user experience of error edge cases from the start, or worse, had assumed all client checks were enough, then nine years down the line my job would have been a lot harder.
Anyway, long story short, adding photos to your FixMyStreet reports is now a smoother process, and you should try it out.
As players were quick to notice, decisions made on our politician-sorting game Gender Balance were final. Thanks to volunteer coder Andy Lulham, that’s now been rectified with an ‘undo’ button.
Gender Balance is our answer to the fact that there’s no one source of gender information across the world’s legislatures—read more about its launch here. It serves up a series of politicians’ names and images, and asks you to identify the gender for each. Your responses, along with those of other players, helps compile a set of open data that will be available to all.
Many early players told us, however, that it’s all too easy to accidentally click the wrong button. (The reasons for this may be various, but we can’t help thinking that it’s often because there are so many males in a row that the next female comes as a bit of a surprise…)
In fact, this shouldn’t matter too much, because every legislature is served up to multiple players, and over time any anomalies will be ironed out of the data. That doesn’t stop the fact that it’s an upset to the user, though, and in the site’s first month of existence, an undo button has been the most-requested feature.
Thanks to the wonders of open source, anyone can take the code and make modifications or improvements, and that’s just what Andy did in this case. He submitted this pull request (if you look at that, you can see the discussion that followed with our own developers and our designer Zarino). We’ve merged his contribution back into the main code so all players will now have the luxury of being able to reverse a hasty decision. Thanks, Andy!
Our team member Richard has now analysed every single one of those votes, and his findings have been added to each MP’s information on TheyWorkForYou.
We hope that’s great news for users: it means that we can now present a really full picture of how your MP voted on key topics.
It’s also potentially useful for developers, eDemocracy hackers and campaign groups, who can pick up our data and use it as they please.
So what exactly is the data?
Often MPs vote on motions which are, at first glance, rather incomprehensible and cryptic. They might vote for example on a motion to accept:
Amendments (a) to (d) proposed in lieu of Lords amendments 1 to 4 and 6.
We’ve done the research to determine what MPs were actually voting on in each case, and turned their archaic language into plain English.
For every vote we’ve written a sentence to describe the effect of voting either “aye” or “no”. In relation to one MP’s vote on the evening of the 9th of July 2014 we write:
Mark Pawsey MP, Rugby voted for a residence test as an eligibility criteria for civil legal aid; subject to exceptions for refugees and those who have sought asylum.
In addition to describing every vote, we have decided whether it should be considered relevant to the topics we list on each MP’s page (see an example MP here, or check your own MP by inputting your postcode on the homepage, then clicking ‘voting record’ on your MP’s page).
If a vote was relevant to one of the statements we show on TheyWorkForYou, we then determined whether voting ‘aye’ or ‘no’ was a vote for or against the statement and if the vote was very important, or less important. By clicking on the green ‘details’ button beside each statement on an MP’s voting record you can see exactly which individual votes contributed to it as well as how we calculated which wording such as “moderately for” or “strongly against” to apply in each case.
Matters MPs have voted on since the 2010 general election have ranged from bankers’ bonuses to same sex marriage; from food banks to the “bedroom tax” (all of which have contributed to statements we show on TheyWorkForYou); from daylight saving to the regulation of hairdressers (neither included) – and plenty more. (We’ve written previously about how we select which topics to show on TheyWorkForYou.)
Of course, Parliament continues to hold votes, and we’ll be continuing to analyse the results as they come in – but it is good to know that we are bang up to date.
How can this data be used?
We have plenty of ideas ourselves, and we want to hear yours, too. With the forthcoming general election, one obvious use is for ‘who should I vote for?’ tools, which match users’ opinions with those of each party.
There’s also potential for comparisons between what constituents believe and what their elected representative has voted for.
No doubt there are many other ideas that haven’t even occurred to us yet – please do get in touch if you have ideas and you’d like to use this data.
This month we released a new version of FixMyStreet. Amongst the new features, fixes, and thingamajigs were some small improvements added by two volunteers, Andrew and Andy.
Even though these are not core pieces of functionality — in fact, precisely because they are not — we want to draw your attention to why they were included, and why this is a Very Good Thing. And perhaps, if you’re a coder who wants to put something into an Open Source project but hasn’t quite found a way in, Andrew and Andy’s work will nudge you into becoming a contributor too.
One of the axioms of Open Source is that, because anyone can read the source code, in theory anyone can contribute to it. In practice, though, it’s not really as simple as that. Both ends of the “anyone can contribute” idea require effort:
- Before contributing to a project of any complexity (as we hope you want to do), there’s often a lot to learn, or figure out, before any work can even begin;
- Before accepting contributions to such a project (keen as we are to do so), there’s an overhead of testing, checking, and managing the incoming code.
The ugly real world
The basic issue here is that software is complex — no matter how well-written, tested and documented program code is, if the problem it’s solving is in the real world, it’s not going to be simple.
This is especially true of anything used by the public, because often you can only make things seem simple at the front (such as a clean web interface or “user journey”) by working hard behind the scenes with data structures and processes that handle the underlying complexity. It’s inevitably true of any projects which have been developed over time — programmers like to use the term “legacy code” to describe anything that wasn’t written then way they’d choose to write it now.
Often the problems that software is solving are not quite as obvious as they first appear. At mySociety we’ve got a wealth of experience and actual usage data that ultimately changes the way we build, and develop, our platforms. We understand the fields we work in well (technically, the nerds like to call these the “problem domain”), whether it’s governmental practice or civic user behaviour, and that’s often knowledge that’s not encapsulated anywhere in the program code.
Furthermore, any established platform must protect against the risk that new changes break old behaviour — something that regression testing is designed to catch. This is especially important on platforms like FixMyStreet or Alaveteli where the software is already running in multiple installations.
This is why we have a team of full-time, experienced, and (thanks to our rigorous recruitment process) talented programmers who can invest the time and effort to be familiar with all these things when they set to work coding.
But this builds up to an impediment: sensitivity to any of these issues is enough to make anyone think twice about simply forking our code and starting to hack on it for us.
How it sometimes works
In practice, then, how does anything get contributed? How come it doesn’t all get written by our own coders?
The answer is, of course, we do work with major contributors outside our own team (have a look at the activity on our github repos to see them) — but it always requires a period of support and on-line discussion both before and during the process. There’s also the business of testing, and sometimes politely pushing back on, pull requests (which is how code contributions are submitted). But the fact of the matter is that this is only possible for people who are willing to spend time familiarising themselves with the specific code, technologies, and practices that we’re using on that project. These tend to be hard-code devs, and — here’s the point — they’re always experienced Open-Sourcers: this will never be the first time they’ve worked on such a project.
Which is where the little features come in.
The joy of small
We noticed this problem — that contributing code to our projects is simply not easy for us or for contributors. Importantly, it’s not just us: it’s Open Source everywhere. But we can’t simply dismiss the opportunity for contribution. We want to encourage coders to do this, because we believe that Open Source is intrinsically a good thing.
We do two things to make it easier to contribute:
- We identify small features that a coder can approach without a full understanding of the code and the problem domain;
- We help people (like you!) get started by opening up a laptop at our weekly meetups.
The first of these seems obvious now: when we add issues (an idea for a new feature, or maybe a bugfix) to our github repos, if we think they’re candidates for manageable, isolated work, we tag them with the label: Suitable for volunteers (like this).
Often these turn out to be “nice-to-haves” that one of our full-time devs can’t be pulled off more pressing problems to add just now. (Case in point: Andrew added a date-picker to the FixMyStreet admin stats page, and three of our own staff had stumbled upon and applauded the difference it had made within a week of it going live).
It means it’s much easier for you to get involved, because often it’s a little, isolated piece of code. And it’s much more manageable for us, because the change you’ll be submitting is also isolated.
So if you’re looking for something to tackle, pick one of those issues, and let us know (just to check nobody else has baggsied it already). Fork the repo, cut the code, write the tests, submit a pull request!
But wait — if that last paragraph made you gulp, here’s the second thing we do: meetups. Of course, this is less helpful if you can’t make it to London on Wednesdays, but the concept is sound. Put simply, if there is a barrier to entry to diving in, and if one-on-one time with a dev, and some pizza, is what it takes to overcome that, it’s time well spent for you to come and see us.
Not 100% confident with git? Not sure when
db/schema.sqlgets used or how we like to handle migrations? No problem: we’re happy to guide you.
If this has struck a chord with you — you’d love to be an Open Source contributor one day, and you think mySociety projects make the world a better place — perhaps you should take a poke in our repos, or come along to a meetup. Start small, but do start.
Oh, and Andrew and Andy — thanks guys 😉
Photo by Matt Katzenberger (CC)
Last week we asked what improvements you’d like to see on TheyWorkForYou. Thanks so much for all the comments on that post (do keep them coming). They’ve all been carefully documented on our development list.
Our standard way of working on a project like this is in ‘sprints’ – short periods of activity after which we can spend some time reflecting on what went well, and what could have gone better.
This system is great for ensuring that we don’t get involved in a large piece of work, only to realise that it doesn’t do what was intended, or hasn’t had the desired effect. So, for example, if we’ve added a new feature, we might be asking ourselves, ‘Is anyone using it?’, ‘Have there been any bug reports?’, and ‘Has it fulfilled our original aim?’. We’re striving to be as analytical and methodical as possible about these assessments, so part of the process has also been figuring out which types of metrics to collect, and how.
That said, what have we already done?
It’s easier to find a specific representative
Where previously our pages listing all MPs, all MSPs and all MLAs just contained one very long list of names that you had to search or scroll through, there’s now an A-Z navigation at the top. We also added the ability to find your own MP from this page.
Why? This is an example of a small usability tweak which should make a difference to a large number of people – not everyone knows how to search a web page with Ctrl+F. It’s also a fix that’s been on our to-do list for two years!
The addition of the ‘find your MP’ box helps to serve one of our core aims: to make democracy easy to understand for the uninitiated.
We’ve added ‘like’ and ‘follow’ buttons
We thought you might not notice these discreet additions to our page footers – but we’ve certainly seen an upturn on the rate at which people are ‘liking’ our Facebook page. Whereas Twitter – not so much. Maybe TheyWorkForYou users are just more Facebook-inclined?
Why? In part, this addition is for our own benefit – we welcome the opportunity that social media gives our users to spread the word. As a small organisation with no advertising budget, this kind of grass roots promotion is invaluable. Then, we are hoping that it will help us to understand our users. Clicking that ‘like’ button can be seen as a form of positive affirmation and enagement that it’s very hard to quantify by other means.
We are still considering the addition of buttons which would allow you to share specific debates with your social circles.
We have noted the comments on our last post which made it clear that some of our users do not welcome integration with social media. That’s fine – we’ll never do anything that excludes you from the core activities of the site, whether you use Facebook and Twitter or not – our intention is simply to provide the functionality for those who want it.
Those comments have been a useful reminder to us that we should continue to consult our users, because we can’t always predict what you might object to!
You can change your email address
If you have an account, now you can change your email address yourself.
Why? This was identified as a common request that often puzzled users, and took up support time on our side.
MPs’ pages will look better
You can’t see these yet, because they’re still in progress. Due to some quirks of the code in which the site was originally built, the new design for the MPs’ pages has taken longer to implement than we’d anticipated. But we’re getting there.
Why? MPs’ pages contain an awful lot of information, from voting history to recent appearances, and more. The redesign will help us present all this information more clearly, making the page just as easy to read on a mobile device as it is on a desktop, and simply bringing the (frankly, dated) pages a more current look.
Bullets are bullets
This is almost ridiculous, but we think it was worth attending to. In recent user tests, we noticed some confusion, caused by the fact that our bullet points were in the form of small squares – they were frequently mistaken for check boxes.
Why? Just to rid the world of that one small piece of frustration that occurs when you try to tick a box that is not, in fact, a checkbox.
As I say, we are still actively collecting and working on your feedback, so please do keep it coming. Comment below this post, or drop us a line on email@example.com. I’ll be reporting back after our next sprint.
Photo by William Warby (CC)
We’re good friends with the people at Mozilla. Every Wednesday, they welcome us into their London Moz space for our weekly meet-ups. They are champions of empowering possibilities of the web through Open Source software (a world we’re part of too). And they’re all so smart and lovely. So of course we’d been looking forward to this year’s Mozilla Festival for some time.
We had a table at the “Science Fair” on Friday night, where we literally had buckets of sweets (OK, they were little plastic buckets). Tom, our director, and Dave, from our international team, talked about mySociety’s work with anyone who came close. Perhaps people were drawn in by those sweets, or the FixMyStreet demo on the monitor, or even the (new!) stickers we had to give away… but regardless of the lure, we think they all learned a little bit more about how our platforms help empower people’s civic lives: from something as simple as reporting a flickering streetlight, to holding a public authority to account, to monitoring a whole parliament. (That’s FixMyStreet, Alaveteli, and Pombola, if you were wondering).
The Mozilla Festival’s venue was, once again, London’s astonishing Ravensbourne, right next to the O2 Millenium Dome. The setting magnifies the wonder of the event. Those big round windows make it feel like being in a spaceship made of Swiss cheese. The place is so open, and so vertical, that the activity and enthusiasm doesn’t just spread out, it spreads up. There is making and teaching, learning and sharing, going on across nine floors, and it’s easy to drift up and down from one themed space to another.
We met old friends. We got to hang out some more with our Chilean brothers-in-code from Ciudadano Inteligente, and the excellent Gaba from Uruguay’s DATA, together with the good people from the OKFN. We made lots of new friends too. And all this didn’t just happen at the sessions. A lot of serendipitous encounters took place by the Alchemy coffee stations. Or on the stairs (khun Toy and khun Hui — hi!). Or in the Alphabet City party venue, afterwards.
So a big “thank you” to that Fiery Fox, and an enthusiastic high five (yes, there was an unLondonlike amount of enthusiasm on show — possibly because quite a few of the attendees were over from the USA — which it is impossible not to be caught up by) to all the people we met at the event. Dave grinned his way through a wonderful Scratch tutorial from Code Club, met a whole array of cool people, got answers to some nerdy coding questions, and learnt about the awesome Hive learning networks… and lots more things besides. That already describes a great weekend. But beyond that, we hope we might see a few new mySociety-powered sites spring up elsewhere in the world due to sparks that were sparked at mozfest last weekend.