1. Alaveteli Release 0.26

    We’ve just released Alaveteli 0.26! Here are some of the highlights.

    Request page design update

    After some research in to where people enter the site we decided to revamp the request pages to give a better first impression.

    Alaveteli 0.26 request page redesign

    We’ve used the “action bar” pattern from the authority pages to move the request actions to a neater drop-down menu. We’ve also promoted the “follow” button to help other types of users interact with the site.

    Alaveteli release 0.26 request page action menu

    Since lots of users are entering an Alaveteli on the request pages, it might not be obvious that they too can ask for information. We’ve now made an obvious link to the new request flow from the sidebar of the request pages to emphasise this.

    The correspondence bubbles have had a bit of a makeover too. Its now a lot more obvious how to link to a particular piece of correspondence, and we’ve tidied the header so that its a little clearer who’s saying what.

    The listing of similar requests in the request page sidebar has been improved after observing they were useful to users.

    Also in design-world we’ve added the more modern request status icons, made the search interfaces more consistent and helped prevent blank searches on the “Find an authority” page.

    Admin UI Improvements

    As an Alaveteli grows it can get trickier to keep an eye on everything that’s happening on the site.

    We’ve now added a new comments list so that admins can catch offensive or spam comments sooner.

    Alaveteli Release 0.26 admin comments page

    For the same reasons, we’ve added sorting to the users list and made banned users more obvious.

    Alaveteli Release 0.26 admin users list

    The CSV import page layout and inline documentation has also been updated.

    More stats!

    The wonderful team at the OpenAustralia Foundation contributed an epic pull request to revamp the statistics page.

    The new statistics page adds contributor leaderboards to help admins identify users as potential volunteers, as well as a graph showing when site admins hide things to improve the transparency of the site.

    Alaveteli Release 0.26 leaderboard stats Alaveteli Release 0.26 hide events

    Extra search powers

    As if that wasn’t enough, the the OpenAustralia Foundation team also added a new advanced search term to be able to find requests to public authorities with a given tag.

    Conversion tracking improvements

    We’re constantly asking how we can improve Alaveteli. In order to answer that question meaningfully, we need good tracking of what happens when people make requests on the site. When we tried to track down whether people coming from social media sites were more likely to make a request than those coming from other sources, we found a problem with the way we tracked ‘conversions’ – the process of getting all the way through making a request. We were using a particular URL to mark the end of that process in Google Analytics. The issue was that sometimes, requesters would share that URL with other people, causing us to record the same request being made multiple times. We’re now using a little bit of javascript to make sure we only record the conversion when a new request is actually being made.

    The full list of highlights and upgrade notes for this release is in the changelog.

    Thanks again to everyone who’s contributed!

  2. Introducing EveryPolitician’s bot

    mySociety’s EveryPolitician project aims to make data available on every politician in the world. It’s going well: we’re already sharing data on the politicians from nearly every country on the planet. That’s over 68,652 people and 2.9 million individual pieces of data, numbers which will be out of date almost as soon as you’ve read them. Naturally, the width and depth of that data varies from country to country, depending on the sources available — but that’s a topic for another blog post.

    Today the EveryPolitician team would like to introduce you to its busiest member, who is blogging at EveryPolitician bot. A bot is an automated agent — a robot, no less, albeit one crafted entirely in software.

    First, some background on why we need our little bot.

    Because there’s so much to do

    One of the obvious challenges of such a big mission is keeping on top of it all. We’re constantly adding and updating the data; it’s in no way a static dataset. Here are examples — by no means exhaustive — of circumstances that can lead to data changes:

    • Legislatures change en masse, because of elections, etc.
      We try to know when countries’ governments are due to change because that’s the kind of thing we’re interested in anyway (remember mySociety helps run websites for parliamentary monitoring organisations, such as Mzalendo in Kenya). But even anticipated changes are rarely straightforward, not least because there’s always a lag between a legislature changing and the data about its new members becoming available, especially from official national sources.
    • Legislatures change en masse, unexpectedly
      Not all sweeping changes are planned. There are coups and revolutions and other unscheduled or premature ends-of-term.
    • Politicians retire
      Or die, or change their names or titles, or switch party or faction.
    • New parties emerge
      Or the existing ones change their names, or form coalitions.
    • Areas change
      There are good reasons (better representation) and bad reasons (gerrymandering) why the areas in constituency-based systems often change. By way of a timely example, our UK readers probably know that the wards have changed for the forthcoming elections, and that mySociety built a handy tool that tells you what ward you’re in.
    • Existing data gets refined
      Played Gender Balance recently? Behind that is a dataset that keeps being updated (whenever there are new politicians) but which is itself a source of constantly-updating data for us.
    • Someone in Russia updates the wikipedia page about a politician in Japan
      Wikidata is the database underlying projects like Wikipedia, so by monitoring all the politicians we have that are also in there, we get a constant stream of updates. For example, within a few hours of someone adding it, we knew that the Russian transliteration of 安倍晋三’s name was Синдзо Абэ — that’s Shinzo Abe, in case you can’t read kanji or Cyrillic script. (If you’re wondering, whenever our sources conflict, we moderate in favour of local context.)
    • New data sources become available
      Our data comes from an ever-increasing number of sources, commonly more than one for any given legislature (the politicians’ twitter handles are often found in a different online place from their dates of birth, for example). We always welcome more contributions — if you think you’ve got new sources for the country you live in, please let us know.
    • New old data becomes available
      We collect historic data too — not just the politicians in the current term. For some countries we’ve already got data going back decades. Sources for data like this can sometimes be hard to find; slowly but surely new ones keeping turning up.

    So, with all this sort of thing going on, it’s too much to expect a small team of humans to manage it all. Which is where our bot comes in.

    Hello bot

    portrait of bot

    To be honest with you, the bot doesn’t really look like this because, being software, it’s entirely non-corporeal. Sorry.

    We’ve automated many of our processes: scraping, collecting, checking changes, submitting them for inclusion — so the humans can concentrate on what they do best (which is understanding things, and making informed decisions). In technical terms, our bot handles most things in an event-driven way. It springs into action when triggered by a notification. Often that will be a webhook (for example, a scraper finishes getting data so it issues a webhook to let the bot know), although the bot also follows a schedule of regular tasks too. Computers are great for running repetitive tasks and making quantitative comparisons, and a lot of the work that needs to be done with our ever-changing data fits such a description.

    The interconnectedness of all the different tasks the bot performs is complex. We originally thought we’d document that in one go — there’s a beautiful diagram waiting to be drawn, that’s for sure — but it soon became clear this was going to be a big job. Too big. Not only is the bot’s total activity complicated because there are a lot of interdependencies, but it’s always changing: the developers are frequently adding to the variety of tasks the bot is doing for us.

    So in the end we realised we should just let the bot speak for itself, and describe task-by-task some of the things it does. Broken down like this it’s easier to follow.

    We know not everybody will be interested, which is fine: the EveryPolitician data is useful for all sorts of people — journalists, researchers, parliamentary monitors, activists, parliamentarians themselves, and many more — and if you’re such a person you don’t need to know about how we’re making it happen. But if you’re technically-minded — and especially if you’re a developer who uses GitHub but hasn’t yet used the GitHub API as thoroughly as we’ve needed to, or are looking for ways to manage always-shifting data sets like ours — then we hope you’ll find what the bot says both informative and useful.

    everypoliticianbot-small
    The bot is already a few days into blogging — its first post was “I am a busy bot”, but you can see all the others on its own Medium page. You can also follow it on Twitter as @everypolitician. Of course, its true home, where all the real work is done, is the everypoliticianbot account on GitHub.


    Images: CC-BY-SA from the EveryPolitician bot’s very own scrapbook.

  3. The Scottish Parliament returns to TheyWorkForYou

    Scottish Parliamentary proceedings are now back on TheyWorkForYou.

    Back in August 2014, the Scottish Parliament changed the way it published the Official Report of its debates.

    TheyWorkForYou works by fetching data from various parliamentary sources—and in this case, unfortunately, the change at the Scottish Parliament end meant that our code no longer worked. We replaced our ‘debates’ section with an apologetic note.

    Well, thanks to the Scottish Parliament kindly republishing the data in almost the format we used to use, we’ve managed to make some small tweaks and restore that content—including debates from the previously missing period. If you’re subscribed to alerts, you should have received an email digest with links to the backdated content (always supposing there was any that matched your chosen keywords).

    And if you’re not subscribed to alerts? Now is a great time to rectify that. We’ll send you an email every time your chosen word or phrase is mentioned in Parliament, or every time your chosen representative speaks.

    While we were doing this work, we also modified TheyWorkForYou so that it now pulls in ministerial data from the Scottish Parliament API. This is a welcome time-saver for us: previously we were creating a list manually from the official PDFs, while we can now automatically fetch it and reformat it into Popolo JSON, meaning it’s consistent with all our other data.

    Thanks for your patience; we know that many people were awaiting this repair, and for longer than we would have liked. Enjoy!

     

    Image: Mark Longair (CC)

     

  4. Everything about TheyWorkForYou’s voting information

    Back in November 2013, we asked you what improvements you’d like to see on TheyWorkForYou.

    One answer dominated: you wanted more information about how MPs vote.

    Adding information on voting has been the single biggest project on the site since its launch, and has required several different phases of development. We announced each of these as it happened, but now that we’re at the end of this large piece of work, it seems like a good time for a complete overview.

    So let’s take a look at exactly what it has involved—and, more importantly, what it means for you.

    We’ll start with a rundown of features, then go into more detail about how they are created at the end of the post, for those who are interested.

    What vote information means for you

    MP's voting record

    1. You can easily see how your MP voted

    Just how much do you know about how your MP voted on the stuff that matters? Most of us would have a hard time keeping up with every vote, simply because it isn’t information that’s widely publicised.

    On TheyWorkForYou, you can see a run-down of how any MP has voted on key policies, by visiting their page on the site and clicking the ‘voting record’ tab (see image, above). We’ve created summaries of their stance on all kinds of matters, including the EU, same-sex marriage, NHS reform and a lot more.

    Each of these summaries is compiled from every vote the MP has made on a motion that impacts on that policy.

    show votes on TheyWorkForYouYou can click ‘show votes’ (see image above) to see the specific votes that go to make up any particular stance, and we’ve laid them all out in plain English so that it’s easy to grasp exactly what the issue is.

    plain English votes on TheyWokrForYou

    And from there you can click through to the website Public Whip, where you can explore votes in more detail, including lists of who voted for or against any given motion.

    2. You can find out how strongly your MP feels

    voted consistently

    When we first presented voting information, we said that an MP had voted ‘strongly for’ or ‘moderately against’ certain policies, which led to quite a large postbag from people asking, “How can you vote strongly, surely you either vote for or against?”.

    We wrote in the second half of this blog post about the wording changes we made to clarify the fact that these stances are calculated from a number of votes.

    3. You can assess if your MP is a sheep or a lone wolf

    We’ve pulled out all the votes which differ substantially from the way that the majority of each MP’s party voted. If your MP has voted against the flow, you’ll see something like this on their page:

    how Yasmin Qureshi differs from party colleagues on TheyWorkForYou

    Why do we highlight this type of vote? Because we think they’re a really good indication of where an MP feels strongly enough about something to risk sticking their neck out. It’s also a great way to check the truth when people say, “MPs? They’re all the same”.

    4. You can understand the background to the votes

    see full debate

    Generally speaking, there’s a debate before any vote takes place in Parliament, covering all the matters which may be topmost in MPs’ minds before they cast their lot.

    Clicking on the ‘show full debate’ link from the topic pages (see image above) will give you the full context.

    How we compile vote information

    If that all seems nice and simple, well, great! That was our aim.

    Putting it all together definitely wasn’t so simple, though. Voting information has never been previously presented all in one place in quite this way before—on TheyWorkForYou or anywhere else, to the best of our knowledge—so we had to figure out how to import the data and how best to display it.

    As with much of our work, it’s a mixture of manual graft and automating whatever we can. Some things, like rewriting votes so that everyone can understand them, can’t be done by a computer. But many of our users are surprised to learn just how much of what we publish out is untouched by human hand.

    Our Developer Struan, who did the most recent round of work on the voting records, said:

    We get all our voting data from PublicWhip, a site set up by Francis Irving (once of mySociety) and Julian Todd. Public Whip takes the data we [TheyWorkForYou] produce from Hansard and extracts only the information on votes (or divisions in Parliamentary jargon) that take place in Parliament. It then allows you to look up how an MP or a Lord voted.

    Let’s just think about that for a moment. We’re looking at a process where Parliament publishes Hansard, TheyWorkForYou scrapes the data and re-presents it, Public Whip extracts the voting information and presents that, and TheyWorkForYou takes that voting information back for its own voting pages. Simple…

    One of the first things we did was to ‘translate’ the votes into plain English, so that it was very clear what was being voted for or against— and if you want to read more about that process, we talked about it in a blog post back in July 2014.

    That allowed us to move to the next phase, as Struan explains:

    Public Whip groups related votes together into policies, e.g renewing Trident, so you can see how an MP voted on the policy as a whole.

    It does this by saying which way an MP would have to vote each of the divisions in the policy if they agreed with the policy. It then takes the MP’s votes on each division in the policy and assigns a score to it based on how they voted. These scores are then added up and compared to the score they would get if they always voted in agreement with the policy. The closer the MP’s score is to the score of an MP who always voted in agreement with the policy, the more they agree with the policy.

    Thanks to Public Whip’s grouping, we were able to start compiling our MPs’ voting records along those same policy lines.

    One of the most fiddly parts of the process was figuring out how to ensure that the information we present is a true, non-biased representation of the MP’s intentions. You might think that a vote is quite a simple matter – it’s either a yes or a no for a particular motion. But as soon as we started displaying votes within a policy, things got a bit trickier.

    Some divisions in a policy can be marked as important and voting with the policy in those divisions is worth more points. This is to prevent voting in agreement on a set of minor votes, e.g “Parliament will commission a report on the future of Trident”,  outweighing voting against something important, e.g. “Renew Trident”. It also reflects the way Parliament works, often with several smaller votes on parts of a bill and then a vote on the bill as a whole.

    For clarity I should point out here that sometimes voting no in a division is a vote for the policy, e.g voting no in a “This house believes Trident should not be renewed” division would clearly be a vote for our example “Renew Trident” policy.

    This approach also helps where one vote straddles several topics: for example, consider a vote against the Budget when the Budget contains many proposals including, say, the capping of VAT. It’s quite possible that an MP may be for the capping of VAT but broadly against several other motions covered by the Budget, and so decides to vote against it on balance. So long as we mark the Budget vote as a weak vote for the capping of VAT, its significance should be properly accounted for.

    Where we don’t have enough information to show a stance, for example where an MP never voted on the topic, is too new to have had a chance to vote on the topic, or all their votes on the topic have been labelled as ‘weak’, we say so:

    not enough info

    A final little subtlety is the difference between “Never voted” on a policy and votes where the MP was absent. If it says an MP has never voted on a policy that means they were elected after all the divisions in the policy took place so did not have a chance to vote on them. Absent means they could have voted in the divisions but did not.

    Absent votes count towards your score but at half the rate of voting in agreement with the policy. This is so that an MP who votes in agreement with the policy in one division and then misses all the other divisions shows as agreeing with the policy rather than against as it would if no score was assigned to absent votes. That does currently mean that if they were always absent it shows, slightly unhelpfully, as “a mixture of for and against”.

    It’s not an ideal system as it does produce some odd results occasionally but it mostly works.

    To show where an MP has voted against the majority of their party, we have to figure out a similar score across the party as a whole.

    This is exactly the same process, only we add up all the votes by all the MPs but the maths is pretty much the same.

    All in a day’s work

    As mentioned at the top of this post, vote information was our most-requested addition. And rightly so! Our MPs represent us, so naturally we want to see their track records, quickly and easily.

    If you’re not an expert, you might not have known how to find this information before. And that’s essentially what TheyWorkForYou aims to do: make the workings of Parliament more accessible for everyone.

     

    Parliamentary copyright images are reproduced with the permission of Parliament.

     

  5. Santiago Chile, 2014: the first Poplus Conference

    14141667353_5855576937_b 14098058716_5b3d4d6b6e_z 14141335013_ef11aa83ba_z 14098099116_fd06593a21_z 14118212822_6416252c99_z 13934577759_8edde350b0_z 13934612170_1b789ecab4_z 13934567660_e54716bc0e_z 14118234182_bf04049d6a_z 14098065166_e85c5405b9_z

    Last Sunday, several mySociety team members woke up just in time to see the sun rise over the Andes – not our standard view in the morning.

    We were on our way to Santiago, Chile, as were delegates from 27 different countries, all headed to the inaugural Poplus Conference. This was a joint event organised by ourselves and our Chilean friends Fundación Ciudadano Inteligente.

    Long-haul flights don’t feature high on our list of favourite activities – so what was so important that we all happily put in the 7,000 miles to Santiago?

    Poplus

    Before we can answer that question, you need to know what Poplus is. In short, it’s a project to create and share open source code that can help civic organisations around the world. You can find out more here.

    While legal structures and ruling regimes vary from place to place, the needs of citizens are broadly constant: they require information, empowerment, and transparency.

    The digital age has provided opportunities here, and there are many organisations like mySociety and Ciudadano Inteligente around the world: groups that harness the power of the internet to open access to civic and democratic processes.

    Underpinning Poplus is the belief that we can make great efficiencies if we share our code – that way, there’s no need for each of these groups to rewrite what is effectively the same piece of software. Poplus aims to encourage the creation of ‘Components’ – small pieces of software that can be used easily, by anyone across the world.

    Poplus – the next step

    Poplus is a project in its infancy, and this conference was the next step in its growth. It brought together civic coders, and organisations with an interest in the code they create.

    The Poplus project was initially conceived by Ciudadano Inteligente and mySociety – but in order to thrive, it needs many members from all over the world to play a part. The conference allowed us to present the general idea behind Poplus, and to ask for help shaping and refining it. Once some underlying principles had been agreed, we could become a true federation.

    There were also many opportunities to listen to organisations, about what they needed from Poplus software, and their experiences running civic websites in their own territories. By the close of the conference, we had thrashed out some broad agreements, and there was a lot of excitement about carrying on the work to create Poplus Components – and the community around the movement.

    Defining a Poplus Component

    Definition of a Poplus ComponentOne important task was to define exactly what makes a Poplus Component: the conference offered us a real opportunity to get input from many different perspectives, and come away with a ‘gold standard’.

    It was very useful to have developers and end users in the same room, talking about the process of creating Components, and the experience of using them.

    You can see what we came up with in the photograph (click to see it bigger). Among other things:

    • Poplus Components are small pieces of software which provide functionality for civic or democratic websites.
    • Each Component solves a single problem.
    • They are built to work in any country, making minimal assumptions about location.
    • They are open source and free for anyone to use.
    • They slot into any website, and may also inter-operate.

    Just the start

    The Poplus Conference was a great opportunity to nail down everyone’s thoughts. Now we have agreed on our shared purpose, the real work will begin. The conference fostered communication and sense of community, and we’ll all be trying to keep that alive.

    A huge thanks to everyone who attended and contributed  – and especially to Ciudadano Inteligente for being such welcoming and generous hosts.

    Read more

     

    Header images – click to see larger. L-R: Setting up the timetable for the ‘unconference’; networking time; ample snacks for everyone; nailing down the essentials of what makes a Poplus Component; t-shirts ready for delegates; jotting down proposals for sessions; more networking; lunch among the autumn leaves; coffee time; chatting.
  6. What is Akoma Ntoso?

    Akoma Ntoso

     

    At the beginning of this year we launched SayIt, our software for bringing transcript publication into the internet age.

    In that post, we briefly mentioned that “we use a cut-down version of the Akoma Ntoso open standard for data import”.

    Well, that’s easy enough to say, but what does it actually mean?

    In a nutshell, if you want to upload transcripts to SayIt, they need to be in a format that SayIt can recognise. It can then transform them into the linked pages that make SayIt so useful.

    Akoma Ntoso is a simple way of showing (for example) which bits of the data are names, which are speeches, etc. – and how they all relate to one another. At first glance, it’s not all that different from HTML, the basic language behind many websites.

    But there are some differences. There are also some interesting ideas behind it, from how it began and where it got its name, to why the world needs another open standard. And what is an open standard, anyway?

    We pinned down Flavio Zeni, one of the people behind Akoma Ntoso’s creation, and he very patiently answered all our questions, even the most basic ones.

    And then, because it seemed silly not to, we put the whole conversation into SayIt. You can read it here.

  7. Open Data Day resources

    Image by OpenSourceWay

    Note (June 2016): This post is now slightly out of date. FixMyTransport is no longer running, though all of the other APIs and tools listed are still available.

    There is also one significant addition which developers should find useful: EveryPolitician, which provides data on all current politicians around the world (and, in the future, we hope, all past ones too). See more here.


     

    Much of what we do here at mySociety relies on Open Data, so naturally we support Open Data Day. In case you haven’t come across this event before, here’s the low-down:

    Open Data Day is a gathering of citizens in cities around the world to write applications, liberate data, create visualizations and publish analyses using open public data to show support for and encourage the adoption open data policies by the world’s local, regional and national governments.

    If you’re planning on being a part of Open Data Day, you may find some of mySociety’s feeds, tools and APIs useful. This post attempts to put them all in one place. (more…)

  8. What we’re doing on TheyWorkForYou

    Image by William WarbyLast week we asked what  improvements you’d like to see on TheyWorkForYou. Thanks so much for all the comments on that post (do keep them coming). They’ve all been carefully documented on our development list.

    Our standard way of working on a project like this is in ‘sprints’ – short periods of activity after which we can spend some time reflecting on what went well, and what could have gone better.

    This system is great for ensuring that we don’t get involved in a large piece of work, only to realise that it doesn’t do what was intended, or hasn’t had the desired effect. So, for example, if we’ve added a new feature, we might be asking ourselves, ‘Is anyone using it?’, ‘Have there been any bug reports?’, and ‘Has it fulfilled our original aim?’. We’re striving to be as analytical and methodical as possible about these assessments, so part of the process has also been figuring out which types of metrics to collect, and how.

    That said, what have we already done?

    It’s easier to find a specific representative

    Where previously our pages listing all MPs, all MSPs and all MLAs just contained one very long list of names that you had to search or scroll through, there’s now an A-Z navigation at the top. We also added the ability to find your own MP from this page.
    AZ navigation on TheyWorkForYou

    Why? This is an example of a small usability tweak which should make a difference to a large number of people – not everyone knows how to search a web page with Ctrl+F. It’s also a fix that’s been on our to-do list for two years!

    The addition of the ‘find your MP’ box helps to serve one of our core aims: to make democracy easy to understand for the uninitiated.

    We’ve added ‘like’ and ‘follow’ buttons

    social media buttons on TheyWorkForYouWe thought you might not notice these discreet additions to our page footers – but we’ve certainly seen an upturn on the rate at which people are ‘liking’ our Facebook page. Whereas Twitter – not so much. Maybe TheyWorkForYou users are just more Facebook-inclined?

    Why? In part, this addition is for our own benefit – we welcome the opportunity that social media gives our users to spread the word. As a small organisation with no advertising budget, this kind of grass roots promotion is invaluable. Then, we are hoping that it will help us to understand our users. Clicking that ‘like’ button can be seen as a form of positive affirmation and enagement that it’s very hard to quantify by other means.

    We are still considering the addition of  buttons which would allow you to share specific debates with your social circles.

    We have noted the comments on our last post which made it clear that some of our users do not welcome integration with social media. That’s fine – we’ll never do anything that excludes you from the core activities of the site, whether you use Facebook and Twitter or not – our intention is simply to provide the functionality for those who want it.

    Those comments have been a useful reminder to us that we should continue to consult our users, because we can’t always predict what you might object to!

    You can change your email address

    If you have an account, now you can change your email address yourself.

    Why? This was identified as a common request that often puzzled users, and took up support time on our side.

    MPs’ pages will look better

    You can’t see these yet, because they’re still in progress. Due to some quirks of the code in which the site was originally built, the new design for the MPs’ pages has taken longer to implement than we’d anticipated. But we’re getting there.

    Why?  MPs’ pages contain an awful lot of information, from voting history to recent appearances, and more. The redesign will help us present all this information more clearly, making the page just as easy to read on a mobile device as it is on a desktop, and simply bringing the (frankly, dated) pages a more current look.

    Bullets are bullets

    alert for kittens on TheyWorkForYouThis is almost ridiculous, but we think it was worth attending to. In recent user tests, we noticed some confusion, caused by the fact that our bullet points were in the form of small squares – they were frequently mistaken for check boxes.

    Why? Just to rid the world of that one small piece of frustration that occurs when you try to tick a box that is not, in fact, a checkbox.

    What now?

    As I say, we are still actively collecting and working on your feedback, so please do keep it coming. Comment below this post, or drop us a line on hello@mysociety.org. I’ll be reporting back after our next sprint.

    Photo by William Warby (CC)

  9. Co-brands, code additions and pull requests

    More and more people are starting to build websites to help people become more powerful in their civic and democratic lives. Some of these are on codebases that mySociety has created which is so great. There are some things which we would love to happen when you take our code and re-use it.

    We want people using our code to keep it as up to date as they can, so that they gain the benefits of any changes made to the code by us or by other users. There are a few reasons for this:

    • You can co-brand the site without breaking anything.

    Dave, one of our developers, explains how you do this. “So suppose, instead of calling it FixMyStreet you want to call it FixMyBorchester with a Borchester logo. Obviously this is a very real requirement, because people want to rebrand. One very feasible (but wrong! As you’ll see…) way of doing this is downloading the FixMyStreet code, finding the bit that paints the FixMyStreet logo and replacing it with the words <h1>FixMyBorchester</h1> and an image. This would work as far as the FixMyBorchester branding would appear on the site.

    But if you then saved and committed your change to git and passed it back to us as a push request, we would reject it. This is for the obvious reason that if we didn’t, next time we deployed FixMyStreet in the UK it would have your logo on it.

    However, say we suddenly discover there is a bug with FixMyStreet. For (a bizarre) example, if someone put the number 0 in instead of a postcode and the site returns a huge picture of a kitten. We love kittens, but that’s not what the site is trying to do. So, we make some fixes to the code that rejects zeros, commit it, update the repo, and it’s now there on the master branch. We write to everyone saying “really everyone, update to the latest (most up-to-date) place on the master branch” And you think, “yeah OK!” and you download the latest version.

    If you just download it and copy it into place, you’re going to lose your FixMyBorchester changes, because there’s a more recent version of that file from us that hasn’t got them. If you did a “git pull” (which roughly means, “git! get me the latest version of master branch”) then git will refuse because there’s a conflict on that file.

    So, instead of inserting your FixMyBorchester stuff over ours, which can’t work, you make a new directory in the right place called ‘FixMyBorchester‘, put your stuff in there and switch the FixMyStreet config — which knows this is something people want to do — to use that cobrand. Any templates FixMyStreet finds in there will now be used instead of ours. You can now safely update the codebase from our repo from time to time and FixMyStreet and git will never damage your templates, because they are in a place it doesn’t mess with.”

    • You can add new features

    Dave continues. “Say when someone uses FixMyBorchester it’s essential that you have their twitter handle, because every time a problem is updated, FixMyBorchester direct-tweets them a kitten for fun. Right now there is no capacity to store a twitter handle for a user in FixMyStreet.

    You simply add a column to the users table in your database and add some code for accepting that twitter handle when you register, and sending the kittens etc. That’s new code that isn’t in FixMyStreet at all. Sooner or later you’ll need to put at least one line into the main FixMyStreet program code to make this happen. As soon as you do that you have the same problem we had before, only this time it’s in code not in an HTML template.

    What we would encourage you to do is put all your new code in a branch that we can look at, and maybe set it to run only if there’s a config setting that says USE_TWITTER=true. That way any implementation that doesn’t want to use twitter, which is — at this point — every other FixMyStreet installation in the world — won’t be affected by it. You send that to us as a pull request and a developer checks it’s not breaking anything, and is up to scratch in quality, and has good test coverage. Then we’ll accept it.

    Even though currently nobody else in the world wants your twitter feature, it’s not breaking anything and it’s now in the repo so you can automatically update from our master when we change bits of our files, and the installation/overwrite/git-pull will work. Plus anyone that does decide they want this feature will now be able to enable it and use it.

    And all of this helps everyone using the code; you have a secure website that can be patched and updated each time we release something, other people have access to features you’ve built and vice versa. And overall, the project becomes more feature rich.

    Please do make changes and push them back to the main codebase!

    Image credit: US Coast Guard CC BY-NC-ND

  10. Pombola, a new name for Mzalendo

    We’ve got some exciting news at mySociety.

    As you know we’ve been helping with the Kenyan Mzalendo website for a while now. And, we’ve been lucky enough to gather interest in Mzalendo‘s codebase from a number of other countries.  These range from Ghana to South Africa, and even as far afield as Paraguay. It’s amazing and humbling for us, but we’ve recently realised one thing: Mzalendo has a wealth of history in Kenya, and an amazingly complex political association. It’s also the name of the website there. So we needed a new name that would allow the code to be changed without the change being associated with the original Kenyan Mzalendo.

    Enter Pombola. This is the new name for the codebase which powers TheyWorkForYou, Mzalendo, Odekro, ShineYourEye, work-in-progress Kuvakazim and other parliamentary monitoring websites across Africa and the world.

    You may ask, “Why Pombola? What does it mean?”

    Well, it is a pretty easy word to remember.  And no one else is using it (possibly because we created it!). The word is a mix of the initials PMO (Parliamentary Monitoring Organisation) and Tombola.

    A Tombola?” I hear you cry in surprise. Well, in a Tombola, you are making a choice with no information at all – just selecting a ticket and hoping that you get lucky.  A Pombola site aims to be the opposite – you’ll get as much information as possible about your elected representatives, so that when you make a choice in future, you’ll have all the facts.

    If you’re interested in using the code you’ll find the repository here on github, along with some documentation.

    If you’re not technical but still want to use this then please contact me and we can discuss what we can do!

    And remember, this may monitor parliaments now, but you could use it for anything (*)

    (*) Disclaimer: please only use for good.

    [1] Girl with Balloons from Courtney
    [2] Air map from OpenFlights.org