1. WhatDoTheyKnow Team Urge Caution When Using Excel to Depersonalise Data

    WhatDoTheyKnow logomySociety’s Freedom of Information website WhatDoTheyKnow is used to make around 15 to 20% of FOI requests to central government departments and in total over 160,000 FOI requests have been made via the site.

    Occasionally, in a very small fraction of cases, public bodies accidentally release information in response to a FOI request which they intended to withhold. This has been happening for some time and there have been various ways in which public bodies have made errors. We have recently, though, come across a type of mistake public bodies have been making which we find particularly concerning as it has been leading to large accidental releases of personal information.

    What we believe happens is that when officers within public bodies attempt to prepare information for release using Microsoft Excel, they import personally identifiable information and an attempt is made to summarise it in anonymous form, often using pivot tables or charts.

    What those working in public bodies have been failing to appreciate is that while they may have hidden the original source data from their view, once they have produced a summary it is often still present in the Excel workbook and can easily be accessed. When pivot tables are used, a cached copy of the data will remain, even when the source data appears to have been deleted from the workbook.

    When we say the information can easily be accessed, we don’t mean by a computing genius but that it can be accessed by a regular user of Excel.

    We have seen a variety of public bodies, including councils, the police, and parts of the NHS, accidentally release personal information in this way. While the problem is clearly the responsibility of the public bodies, it does concern us because some of the material ends up on our website (it often ends up on public bodies’ own FOI disclosure logs too).

    We strive to run the WhatDoTheyKnow.com website in a responsible manner and promptly take down inappropriately released personal information from our website when our attention is drawn to it. There’s a button on every request thread for reporting it to the site’s administrators.

    As well as publishing this blog post in an effort to alert public bodies to the problem, and encourage them to tighten up their procedures, we’ve previously drawn attention to the issue of data in “hidden” tabs on Excel spreadsheets in our statement following an accidental release by Islington council; one of our volunteers has raised the issue at a training event for police FOI officers, and we’ve also been in direct contact with the Information Commissioner’s office both in relation to specific cases, and trying to help them understand the extent of the problem more generally.

    Advice

    Some of our suggestions:

    • Don’t release Excel pivot tables created from spreadsheets containing personal information, as the source data is likely to be still present in the Excel file.
    • Ensure those within an organisation who are responsible for anonymising data for release have the technical competence to fulfil their roles.
    • Check the file sizes. If a file is a lot bigger than it ought to be, it could be that there are thousands of rows of data still present in it that you don’t want to release.
    • Consider preparing information in a plain text format, eg. CSV, so you can review the contents of the file before release.
  2. mySociety – BAFTA and Emmy nominations

    empty homes spotter

    We won’t insist on being addressed this way, but you can now append ‘BAFTA and Emmy nominated’ to our name. We were very chuffed to be nominated for two television awards in the last month: the BAFTA for Digital Creativity in Television Craft, and the Emmy for best Digital Non-fiction Programme.

    ‘TV?’, you might be thinking, ‘I thought mySociety were all about digital stuff.’ Well, increasingly, of course, the lines are blurred. Television programmes come bundled with their own website, Twitter hashtag, or app. These days, TV is less about being a passive viewer, more about becoming part of an active, engaged conversation online.

    Last year, we worked with Channel 4 and TV production company Tiger Aspect to create the app and the website tools that accompanied their programme about empty houses – The Great British Property Scandal. A repurposing of the software that underlies FixMyStreet, the app enabled viewers to report empty homes; the site petition amassed 119k signatures – so the audience certainly got involved.

    We were, of course, delighted to have been recognised, along with C4 and Tiger Aspect. In the end, we didn’t need the space we’d hastily cleared on the mySociety mantelpiece, but as the BAFTA went to the incomparable Paralympics, we really can’t begrudge it.

    And of course, if you’re a TV company looking for help with your digital tie-ins, we’re happy to help.

  3. What should we do about the naming deficit/surplus?

    I don’t think it is too controversial to make the following – rather boring – assertions: Greenpeace is part of the environmental movement. Oxfam is an international development charity. Human Rights Watch is part of the human rights movement. Obama for America is a political campaign. Facebook dominates the social networking sector. I hope none of these simple, descriptive statements has caused you to turn purple with semantic rage.

    But what primary movement or sector is mySociety part of? Or Avaaz? Or Kiva? Or Wikileaks? When I ask myself these questions, no obvious words or names race quickly or clearly to mind. There is a gap – or at best quite a bit of fuzziness – where the labels should go.

    This lack of good labels should surprise us because these groups definitely have aims and goals, normally explicit. Also, it is unusual because social and political movements tend to be quite good at developing names and sticking to them. If you were given a time machine you could tell a Victorian that you were ‘pro-democracy’ or ‘anti-slavery’ and the locals would have no trouble understanding you. Terms like ‘gender equality’, ‘small government’, ‘cancer research’, ‘anti-smoking’, even ‘anti-capitalist’, can comfortably be used by news media companies without fear of baffling the audience. The public can also easily understand terms that referred to methods of achieving change, rather than goals, terms like ‘political TV advertising’, ‘protests’, ‘petitions‘ and ‘telethons’.

    But now let’s look at some of the common terms that are used to talk about the (very) wide field of digital social change projects. These include ‘digital transparency’, ‘hacktivism’, ‘peer production’, ‘edemocracy’, ‘clicktivism‘ and ‘open data’. But if you tried to slip one into a newspaper headline, the terms would definitely fall beneath the sub-editor’s axe before they could make it to print. They are too niche, and too likely to confuse readers.

    The first thing to note about most of these terms is the way that they refer to methods, rather than goals of social change. But this isn’t completely unprecedented, and isn’t a reason to dismiss these terms out of hand. The name ‘Chartists‘ does indeed refer to people who used the publication of a charter as a political tool, but the name signified a huge bundle of values, methods and goals which went way beyond the deployment of that document.

    Nevertheless, to me it still just doesn’t feel like the broad, loosely coupled fields of human endeavour which stretch from Anonymous to JustGiving  have decent labels yet – especially not labels that signify the ways in which two things can be both similar and different (e.g. ‘rail station’ and ‘bus station’). And this worries me because consistent names help causes to persist over time. If the field of AIDS research had been renamed every 6 months, could it have lasted as it did? Flighty, narrowly used language confuses supporters, prevents focus and is generally the enemy of long term success.

    So, why does this dearth of decent sector labels exist, and can we do anything about it? The short version is, I don’t know. But I do know that the easy answer, ‘It’s all too new to have names’ cannot be right any more, not now that millions have signed petitions, joined Avaaz, donated to Obama online and so on.

    I don’t know why the category terms in these sectors are so weak and changeable, but I am posting today because I would love to hear the thoughts of other people who might have some ideas as to the causes, and possible solutions. Here are some theories about the lack of good labels, off the top of my head:

    1. I think some of the terms currently in circulation were coined in anticipation of the development of possible projects, not after retrospectively reviewing them. So the category terms sometimes seem to define what a field might look like, rather than what it ends up looking like (think ‘edemocracy‘, from a decade ago). This means the terms often feel like they don’t describe real projects very well.

    2. In the traditional (for-profit) internet industry a certain amount of money can be made from coining or becoming associated with new terms (think of IBM and ‘smarter cities’). Because there is a profit motive, there may be a structural incentive to rapidly create new terms which displace older ones which haven’t been widely adopted yet. There are probably similar incentives in some academic fields too – career rewards for coining a key term.

    3. Terms in these fields we work in are usually minted one at a time – ‘only children’ as opposed to born as whole families of interconnected terms. This is unlike the sciences which, since Linnaeus came up with his elegant way of naming living things, have been good at developing naming systems, not just one-off names. Organic chemistry and inorganic chemistry are related, but different in important ways – the names helpfully show that.  To explain how 38 Degrees and mySociety are similar in some ways but different in other very significant ways needs a way of naming things that can signal both commonality and difference.

    4. The knowledge-sharing disconnect between the academic and activist/practitioner communities is really, truly terrible, everywhere except data-driven voter-targeting. People who run services or campaigns normally never hear about what the brightest academics are saying about their own work. And if they do try to pay attention to the ideas coming out of academia then the signal to noise ratio is too bad and the filters are too few and too busy having day jobs.

    5. And, of course, I should namecheck the sceptic’s probable theory: this would argue that good, clear terms don’t exist because all these widely differing organisations are nothing more than meaningless feel-good bunk, so language slides off them like an egg off Teflon. I don’t subscribe to this theory, of course, but it’s worth noting because I’m sure some people would provide this answer to my question.

    I am planning to write a follow-up blog post to this containing some suggested terms we might use to reflect what the many digital projects out there have in common, and how they are different.

    But before I do, I would like to hear people’s thoughts on whether this is a real problem at all, and if so why that might be, and what we might do about it. Who knows, maybe someone will even write a blog post about it, like we’re back in 2003 or something…

     

     

  4. Changes to public authorities today

    National Health Service changes in England

    Today (1st April 2013) marks a significant change in the way that the NHS in England is structured.  Strategic Health Authorities (SHA) & Primary Care Trusts (PCT) are abolished, and their responsibilities are being taken on by newly created Clinical Commissioning Groups (CCG), the National Commissioning Board, Public Health England and local authorities.

    The split is roughly along these lines:

    • Clinical Commissioning Groups commission elective hospital care, urgent and emergency care, community healthcare and mental healthcare & learning disability services for the local areas they cover
    • The National Commissioning Board covers primary care contracting (GP Contracting, Dental, Pharmacy), specialised services, offender healthcare, secure mental health care and some armed forces healthcare
    • “Top-tier” and unitary Local Authorities take on responsibilities for these aspects of public health: sexual health services, drug and alcohol treatment, health checks, school nursing programmes, giving up smoking programmes and services to prevent childhood obesity
    • Public Health England is a national body which will work closely with local authorities’ public health teams, carrying out a range of activities to protect and improve the nation’s health, eg to co-ordinating work to combat infectious diseases such as flu or infections acquired in hospitals such as MRSA, or to carry out national publicity campaigns to prevent ill health

    This means quite a bit of change to the public authority listings on WhatDoTheyKnow:

    1) PCTs and SHAs are now marked as “defunct” to prevent new requests from being made (see below for more details).

    2) We’ve now listed all the new CCGs, but we’re missing email addresses for around 15% of them.  It’s clear that many CCGs are not quite ready to welcome FOI requests.  Even though they went live today, there are a fair number of websites still under construction (I’ve seen lots of “lorem ipsum” text today), with no contact details.  We aim to get these all up-to-date in the next few weeks as they get up to speed.

    3) The National Commissioning Board and Public Health England have been added to the site

    4) We’ll be adding local Health and Wellbeing Boards, Healthwatch organisations & Local Education & Training Boards soon.

    Police Service changes in Scotland

    Under the banner of reducing duplication and cost-saving (BBC article), police services in Scotland are being completely re-organised with 2 new central bodies replacing all the regional police forces and boards:

    Fire Service changes in Scotland

    Similar changes are taking place with Scotland’s fire services:

    Other joiners & leavers…

    The following is a round-up of other changes taking place today…

    Say hello to:

    And goodbye to:

    And although they’re officially changing, it’s pretty much business as usual for:

    Defunct public authorities

    We flag old public bodies that no longer exist as “defunct” to prevent new requests from being made.  In most circumstances FOI officers transfer across in-flight requests to the relevant replacement authority.  If you need to follow-up a request to a defunct public body (e.g. if there’s no further contact from an authority), the website will let you, however the “old” authority is no longer under any obligation to reply.  You may need to re-send your request to a new public authority which will restart the 20-day clock…

    Please help us!

    Given the scale of change, if you find any incorrect information for these public authority listings, please let us know!  Also please get in touch if you find an email address for any of those we’re still on the hunt for…

     

  5. WhatDoTheyKnow now 6% in Welsh

    Helô!

    Alaveteli (the software that runs WhatDoTheyKnow) is capable of being translated into any language, and we’ve finally switched on the ability to use the website in Welsh today. Many apologies for the long wait as this has been on our to-do list for well over 2 years…

    As you can see, we don’t yet have a complete Welsh translation, and it’s just a start:  we’ve done the help pages, and around 6% of the rest.  To take a look at what’s been done, just click the “Cymraeg” link at the top of any page.

    We’d love it if you could help us get to 100% by adding translations (or correcting any mistakes we’ve made!) at Transifex. You can read more about working with translations for Alaveteli, here and here, or just get in touch if you need a helping hand getting started or have any further questions.

    And finally, a massive thank you & diolch to the translators who have already helped us get this far!

  6. New vacancies at mySociety

    Now Is A Good Time To Explore by Minivan Ninja

    What’s on your Christmas wishlist? If ‘a meaningful job’ is high on the list, then we’ve got important news for you. We’re looking for talented, passionate and diversely skilled people to join our team.

    In 2013, we’ll be pushing out internationally, improving our core UK sites and doing more commercial business. And we need some more lovely, dilligent people to help us.

    All the details are on our jobs page. There’s plenty of time to get your application in, so why not give it some thought over the mince pies?

    Not quite for you? Then please tell your nicest friends!

    Photo by Minivan Ninja (CC)

  7. Looking back: our experience of the Google Summer of Code

    Summer may seem like a long time ago, but despite the cold outside, we’ve been looking back over our participation in Google’s Summer of Code project. It’s almost enough to warm us up!

    This post is an attempt to record the process from our point of view. We hope it will be useful for other organisations considering participating next year, and for students who want to know more about how the scheme works.

    What is Google Summer of Code?

    It’s a programme sponsored by Google’s philanthropic arm, giving students the chance to experience real-life coding on open source software.

    The scheme is open to students all over the world, who are then paired up with open source organisations like us. The students gain paid work experience and mentoring; the organisations gain willing workers and some fresh new perspectives; the world gains some more open source code to use or develop further.

    Everyone’s a winner, basically.

    The beginnings

    2012 was our first year on the programme: once we had been accepted on the scheme, we were given two student slots – the maximum allowed for a first-time organisation.

    Given mySociety’s wide suite of codebases, there were several projects that could have benefited. We listed all our ideas, and let people apply for the ones they found appealing.

    Goodness, there were a lot of applicants! It was very heartening to discover that there is such an enthusiastic community of young coders all around the world – even if it did take us a long time to sift through them all and make our choices.

    You might remember our post back in May, when we announced that we’d made our choices. We were delighted to get working with Dominik from Germany and Chetan from India.

    The project

    As things turned out, our students ended up working on a project that wasn’t even on our original list: PopIt, our super-easy ‘people and positions’ software.

    That’s because once we spoke to our chosen students, we realised they had the skills that could really help us forge ahead with this project – and once we discussed it with them, they were keen. So PopIt it was.

    Logistics

    Germany and India are a bit of a commute away, but fortunately development work can be managed remotely. We know this particularly well at mySociety: our core team work from home and are scattered across the UK.

    The only difference here was the 6+ hour time difference between us and India: it was important to be rigorous about checking in at times when Chetan would be awake!

    We communicated via IRC (instant chat), email, and occasionally Skype, and it all worked well.

    Edmund, the team member chosen to be mentor, broke the required tasks down into big pieces so that the students would have realistic work units of several days each.

    What was achieved

    PopIt is primarily a tool for helping people create and run parliamentary monitoring websites (like TheyWorkForYou) with minimal coding knowledge/effort, though we anticipate that it will have many other uses too.

    Our students spent the first half of the summer learning and improving the PopIt codebase. Once they were confident in it, they created their own sites using PopIt as a datasource to test the API, and, hopefully, create a valuable reference resource for the community.

    Dominik added a migration tool to PopIt, which lets you upload data as a CSV. This means that you can start a site with a database of names, positions and dates at its heart – within seconds.

    His test site was a professors’ database (the code is here). Dom also wrote some helpful posts on the dev blog like this one.

    Chetan created an image proxy that lets us serve images in a smart way that makes sense for APIs. His test site was for Indian representatives (here’s the code).

    Neither site is being maintained now, which just confirms that it is harder to run a site than to start it. This is not a failing, though. The creation of these sites, along with Chetan and Dom’s feedback, helped us understand where improvements needed to be made. In the course of one summer, PopIt became much more mature.

    Looking back on the Summer of Code

    Edmund attended a follow-up ‘mentors’ summit’ at the Googleplex in California – he found it very helpful to compare notes with other organisations and find out what had worked best for them all, and he made some good contacts too.

    Assuming we get the chance again, would we participate in 2013? Our experience was very positive, but as yet we are undecided, purely because of the fluid nature of our workflow: we don’t yet know whether time and resources will permit.

    Obviously, we have enjoyed great benefits from the scheme, but that has depended on quite a bit of input from our side, and we need to be sure that we can ensure that happens again.

    Edmund has compiled a list of advice, from the practical (ask students to treat the placement like a full-time job; test coding skills before acceptance) to the desirable (a weekly blog post from participants; make sure you over-estimate the time you’ll spend mentoring). If you’re thinking of participating next year, he’d be happy to pass on his tips for ensuring that you, and your assigned students, get the best out of the Google Summer of Code. Just drop him a line.

  8. mySociety Christmas Pub Meet

    Inside the Prince Edward, Bayswater, London W2 by Kake Pugh

    We hope friends, supporters and indeed anyone who fancies it will join us for a festive drink at the Prince Edward pub in London’s Notting Hill.

    When? 7.30pm onwards, Tuesday 11th December.

    Where? 73, Prince’s Square, W2 4NY. Google Map

    Who? Everyone’s welcome.

    Why? Come and chat about any of our projects, becoming a volunteer, new ideas you have – or just enjoy a drink.

    You can add your name, and see who else is planning on coming, on our Lanyrd page.

     

    Image (CC): Kake Pugh

  9. mySociety is inviting people to become trustees of its parent charity

    Summary

    mySociety is looking to recruit new trustees to help us, as we transition from being a small digital non-profit into a mature international social enterprise.

    If you are interested in helping to guide one of the earliest ‘digitally native’ charities through to its next stage of growth, this may be an opportunity of interest to you.

    Our mission is to discover how technology can (or cannot) help make people more powerful. As a team and a community we are driven by a desire to build tools that help people exert a little control over the world around them – especially people who have never tried to do so, and who don’t think they would succeed if they tried.

    If that is a goal that motivates you in the way it motivates our staff and volunteers, we should have a conversation.

    What is a trustee?

    Trustees oversee charities to ensure that they are well-run, solvent, operating within the law, and making the right strategic decisions. These are unpaid roles with an ultimate legal responsibility for the charity. To understand more about what being a trustee means legally, please see this introduction from the charities commission.

    mySociety is the public brand of the registered charity UK Citizens Online Democracy: the positions we are advertising for today are trustees of that charity.

    As a UKCOD trustee, you will advise on the organisation’s priorities, help with the approval of budgets and staffing, and assess legal matters. In concrete terms, that means attending meetings in London every three months, and dealing with the associated emails and documents – a commitment of about six hours per month.

    History

    Between 2003 and the present day, mySociety has built and grown a series of British democratic and civic websites and apps, including FixMyStreet.com, WhatDoTheyKnow.com, WriteToThem.com and TheyWorkForYou.com.

    In the last two years our organisation has experienced a great deal of growth, with our staff tripling in number (to nearly 20) and our objectives becoming ever more international. This is largely due to major investments by groups like the Omidyar Network and the Open Society Foundation, as well as an increase in our commercial software and consultancy services, which generate about half our revenues.

    Future Challenges

    We have numerous challenges to face as we approach our 10th birthday, in late 2013.

    • How do we balance the need to maintain and improve the quality of UK services, whilst working increasingly in other countries?
    • How do we ensure that the people trying to build copies of the services we run in other countries succeed in adapting them to very different environments?
    • How do we become a successful, substantial social enterprise that can drive quality improvement and higher ethical standards across the entire government IT sector?
    • How do we do all this whilst ensuring that the high standards of talent – and niceness – of the people within our organisation do not slip?

    What we’re looking for in trustees

    mySociety is still small enough that it often needs very practical support from the trustees, such as opinions on legal matters. This means we need trustees equally comfortable with big questions and small ones.

    We are interested in acquiring trustees from a range of different occupational backgrounds. If you have skills in any of the disciplines listed below, you could really help us.

    • Marketing
    • Campaigning and community organising
    • International development
    • Human resources
    • Legal
    • Governance structures
    • Digital product development
    • Finance
    • Local Government
    • Advertising
    • Quality assurance

    We also welcome applications from mySociety volunteers, whether past or present.

    Timelines

    We will be happy to meet people and arrange phonecalls for no-commitment discussions up to 21st December 2012. Please contact abi@mysociety.org if you would like to book in a conversation with someone who could tell you a bit more about the role.

    If you would actually like to apply, please send a CV and covering letter, explaining why becoming a trustee is of interest, to abi@mysociety.org by 5th January 2012 latest.

    We will conduct interviews on Monday and Tuesday 21st and 22nd January – we can arrange them in the evenings if that is necessary.

    We will notify applicants of our appointment decisions on Monday 28th January.

  10. Dear Ninja Campaigner Geek: Why I work on non-partisan tech, and why I encourage you to take a look

    The last few weeks since the US election have seen an explosion in articles and blog posts about how Obama’s tech team pulled out the stops in their race against the Republicans. It’s been an exciting time to learn about the new techniques dreamed up, and the old ones put to the test.

    For those of us who develop non-partisan services to help people report broken street lights or make Freedom of Information Requests, such stories certainly seem unimaginably glamorous: I don’t think any of my colleagues will ever get hugged by Barack Obama!

    But it has also been an interesting time to reflect on the difference between choosing to use tech skills to win a particular fight, versus  trying to improve the workings of the democratic system, or helping people to self-organise and take some control of their own lives.

    At one level there’s no competition at all: the partisan tech community is big and economically healthy. It raises vast amounts of cold hard cash through credit card payments (taking a cut to pay its own bills) and produces squillons of donors, signers, visitors, tweeters, video watchers and so on. The non-partisan tech community is much smaller – has fewer sustainable organisations, and with the exception of some big online petitions, doesn’t get the same sort of traffic spikes. By these metrics there’s absolutely no doubt which use of tech is the most important: the partisan kind where technology is used to beat your opponent, whether they are a political candidate, a policy, company, or an idea.

    But I am still filled with an excitement about the prospects for non-partisan technologies that I can’t muster for even the coolest uses of randomized control trial-driven political messaging. The reason why all comes down to the fact that major partisan digital campaigns change the world, but they don’t do it in the way that services like  eBay, TripAdvisor and Match.com do.

    What all these sites have in common – helping people sell stuff they own, find a hotel, or a life partner – is that they represent a positive change in the lives of millions of people that is not directly opposed by a counter-shift.  These sites have improved the experience of selling stuff, finding hotels and finding life partners in ways that don’t attract equal and opposite forces, driven by similar technologies.

    This is different from the case of campaigning tech: here a huge mailing list is pitted against another even huger mailing list. Epic fund-raising tools are pitched against even more epic fund-raising tools. Orca vs Narwhal. Right now, in US politics, the Democrats have a clear edge over the technology lined up against them, and I totally understand why that must feel amazing to be part of. But everything you build in this field always attracts people trying to undo your work by directly opposing it. There is something inate to the nature of partisanship which means that one camp using a technology will ultimately attract counter-usage by an opposing camp.

    This automatic-counterweighting doesn’t happen with services that shift whole sectors – like TripAdvisor did. In the hotel-finding world, the customer has been made stronger, the hotel sector weaker, and the net simply doesn’t provide tools to the hotel industry to counter what TripAdvisor does.

    It is this model – the model of scaleable, popular technology platforms that help people to live their lives better – that I aspire to bring to the civic, democratic and community spheres in my work. Neither I nor mySociety has yet come up with anything even remotely on the scale of a TripAdvisor, but there remains the tantalising possibility that someone might manage it – a huge, scaleable app of meaningful positive impact on democratic, civic or governance systems*. Our sites are probably about as big as it gets so far, and that’s not big enough by far.

    If someone does manage to find and deliver the dream – some sort of hugely scalable, impactful non-partisan civic or democratic app & website, it is unlikely that the net will instantly throw up an equal and opposite counterweight. There is a real possibility that the whole experience of being a citizen, the whole task of trying to govern a country well will be given a shot in the arm that won’t go away as soon as someone figures out how to oppose it. I’m not talking Utopia, I’m just talking better. But what motivates me is that it could be better for good, not just until the Other Team matches your skills.

    And that – in rather more words than I meant to use – is why I am still excited by non-partisan tech, and why I really hope that some of the awesome technologists who worked in the political campaigns of 2012 get involved in our scene.

    I’m on Twitter if anyone wants to talk about this more.

    *  You can certainly make an argument that Twitter and Facebook sort-of represent non-partisan democracy platforms that have scaled.  But some people disagree vehemently, and I don’t want to get into that here.