1. Gaza Ceasefire votes and voting records

    What happened?

    Yesterday in the House of Commons, there was an SNP Opposition Day debate about a ceasefire in Gaza. This meant that the SNP had an opportunity to put forward a motion for the House to vote on.

    The Labour Party’s preferred wording of a ceasefire motion replaced the SNP motion and was passed by the House of Commons based on what is known as a voice vote. 

    This is when the speaker (in this case the Deputy Speaker who was in the chair) judges the result of the vote based on the volume of shouts in the chamber. As such, there is no record of how individual MPs voted.

    This is not the same as saying the vote was unanimous – and listening to the recordings there is a clear ‘no’ present on both votes (the Deputy Speaker does later claim that “nobody called against it”, which is then contested).

    From the Speaker’s point of view, the goal is taking a read on the decision of the House (and this may have been correct in that one side was louder, if not unopposed) – and a vote in the lobbies (division), which takes around 15 minutes, serves no purpose. 

    But votes also serve the purpose of putting the opinions of individual MPs on the record, which several were frustrated to have been denied. Votes are part of the public facts about MPs’ impact in Parliament, and part of how actions are communicated to constituents. This is a factor in the democratic process that also needs balancing in these decisions. 

    This decision followed a long division for a motion to sit in private – and votes that seemed clear on a voice vote may have been seen as costly in terms of time to take to a full division. In general, it is possible to have voting processes that are much faster and fairer to MPs, that would allow getting two votes on the record without taking most of an hour of parliamentary time. 

    What does this mean for TheyWorkForYou?

    Because there was no recorded division – the approval of the motion does not appear in the recent votes tab for MPs.

    The full debate is worth a read – the general sense is of a long debate where MPs engage with a complicated situation, and reflect that the UK’s role can only be part of any solution. 

    We’re in the process of updating the processes behind our voting summaries, which includes ways to include what we’re calling “agreements” (decisions without a “division”) in summaries. But issues like last night’s decision reflect that we need to take a cautionary approach – as there is clear evidence that it was not an unambiguous decision. We will publish more on this approach soon.

    Why was the amendment process controversial?

    As it was an SNP Opposition day, (a day when an Opposition party gets to choose the main debate) they got to propose the motion. Both Labour (another Opposition party) and the Conservative Party (Government party) proposed amendments to the motion.

    The Speaker went against previous convention and allowed both a Government and Opposition amendment – which was unexpected. The motivation of this was to give most MPs a chance to vote for a motion on their preferred wording – the problem is that the amendment process is not really set up for this.

    The thinking makes sense given how Opposition day voting is supposed to work: the Opposition by definition is not supposed to win because they have fewer MPs than the government.

    What’s supposed to happen is that MPs debate a topic, hold a vote, and the motion is rejected. If the topic strikes closer to home, the Government will amend it to say “this is an important issue but the Government is doing a great job”, and that is the motion that is passed because the Government should have the numbers to win the vote.

    Government amendments come after the vote on the main motion to respect the purpose of Opposition day debates, while reflecting the reality that the government can amend the motion and win. This sequencing allows for a vote on the pure motion on the record before the amended one wins.

    The same applies for amendments from other parties or backbenchers in the Opposition – these votes should also lose, and can be put before the motion without disrupting the flow.

    So what could have happened is: Labour amendment rejected (mostly by Government MPs), SNP motion rejected (mostly by Government MPs), Conservative amendment approved and adopted (mostly by Government MPs). In this scenario, most MPs have had a chance to vote for their party’s preferred wording, but this is only possible because the first few votes are rejected.

    In practice what seems to have fallen apart is the government approach – exactly why is still unclear but one suggestion is not enough Government MPs wanted to vote against the Labour wording, so to avoid an internal conflict they pulled their amendment and stopped opposing other votes. 

    This meant that Labour’s amendment won, it replaced the SNP motion and was passed as the main motion. 

    This outcome was the opposite of the one the Speaker’s choice was intended to facilitate. The SNP (and anyone who preferred the contents of their motion) didn’t get the chance to vote on their version, and no one generated a voting record either. A ceasefire motion passed, but no individual votes were recorded for it.

    The role of the Speaker

    The core issue is different ideas of what the Speaker is supposed to do. 

    In one reading the Speaker is supposed to be an agent to draw out the collective will of MPs, in another, the emphasis is on being non-partisan and reflecting a settled (cross-party) view of how the House of Commons operates. 

    The Clerk of the House advised not to allow both amendments, but also said that this was allowed by the rules, it went against previous approaches and risked that the SNP motion wouldn’t be voted on. The Speaker didn’t do anything inherently wrong by the rulebook, but has upset the sense that he was supposed to be a speaker who “innovated” less than his predecessor John Bercow.

    The virtue of deferring to the dead hand of precedent is that it shields the Speaker from the accusations of political bias. The outcome of this decision was good for Labour in that it avoided a split over the SNP vote, leading to a perception the rules were being bent in Labour’s favour. If this had threaded the needle and everyone had got the votes they wanted, this might have paid off. As it is, there’s a big question mark over whether the Speaker is trusted by MPs to be fulfilling the role. 

    The argument made by Owen Thompson (SNP) was that “the purpose of an Opposition day is for our party to have the ability to put forward our business”.  In general, Labour has a lot more Opposition days, where they haven’t chosen to propose their version of the motion. One of the SNP’s few days has resulted in SNP MPs not being able to put their views on the record. 

    But also if a motion would be preferred by the House it doesn’t seem undemocratic to include it. The amendment process is not meant to allow voicing opinions on three different things – but working towards a single statement that has majority support. If including more amendments changes the outcome, it is reasonable to include them on this basis. 

    This gets at different ideas of what voting in Parliament is for – is it for Parliament to come together and agree a consensus view, or for political actors to signal their divergent views? Both of these are legitimate purposes for a political body – especially when the goal of the motion is signally internationally (and also domestically) what the UK political establishment’s views are. 

    What does this mean for foreign policy?

    There’s a view that this kind of vote is navel gazing – and what MPs yell about over here doesn’t affect things over there. But this view is too narrow and misunderstands Parliament’s role as a political institution and how that relates to international politics.

    Motions can be broadly “doing something” or “saying something” motions, and this was a “saying something” motion. It doesn’t commit the government to do anything, and if it did, the government doesn’t have the power to impose a ceasefire tomorrow.

    In practical terms, it doesn’t matter what the Opposition thinks except in terms of the approach it signals in a possible next government. It does matter what government MPs are thinking however, and these motions seem to have flushed out some fault lines within the Conservative Party. Even if this isn’t on the voting record, it shapes internal discussion and policy making. 

    It is broadly good for the long term project of British diplomacy and coalitions with other countries where there is widespread consensus in Parliament on an action. Even partial support for bigger approaches within the governing party gives the Foreign Secretary more flexibility, and alignment with the likely next party of government similarly empowers the kind of statements and alliances that can be made. 

    So the vote does nothing in itself, but helps reveal what the political lie of the land actually is, and empowers actors working within it. Just because something is partial and political doesn’t mean it’s pointless.

    Political violence is shaping how representatives behave

    Another running thread here is the idea of political violence impacting decisions on the parliamentary agenda. The Speaker explicitly said the decisions he made on amendments were based on conversations “about the security of Members, their families and the people involved” – where MPs were considering their personal safety in weighing up if they could oppose motions by other parties. Regardless of whether you think it would be fine if more MPs had supported the SNP motion, it’s not good that this is part of the thinking in either direction. 

    This is part of a wider problem where political violence and threats of violence are collapsing political trust and openness – making politicians more suspicious of each other (seeing each other as whipping up mobs rather than engaging in politics), and less likely to give the benefit of the doubt to what may be passionate but legitimate participation of citizens in politics. 

    The parliamentary rulebook cannot take the weight of this – there are arguments about the extent to which allowing individual expression is an important purpose, but it can’t take the weight of allowing individual expression for the purposes of safety. 

    Here is where the recent Jo Cox Foundation report  No place in politics: tackling abuse and intimidation gives constructive steps. A key argument in the JCF report is that proportionate reactions to political violence can damage the relationship between representatives and their constituents. More safety measures and less public visibility make representatives less accessible. Its impact is not just in the one act, but the chilling effect it extends through the whole system that makes us more distant and suspicious of each other. 

    Reflecting this, the report puts a lot of time into a series of very practical measures to improve policing and reporting of threats and abuse, tying together different systems of support across Parliament, parties and policing. The clearest way to take abuse and intimidation seriously is to join up support and action on the least ambiguous cases. Politicians feeling that they are safe, and that threats against them are taken seriously, helps an environment where trust and openness support a better democratic system.

    Photo: UK Parliament – Central Lobby 

  2. Democracy month notes – January

    What are month notes?

    They are notes on what we’ve been working on… each month. It’s like weeknotes for lazy people. 

    We’ve been writing them for our Climate programme, and we’re building up to the point where we’ve got enough going on in our Democracy work that it’s worth establishing the habit of being clear about what’s going on.

    It’s about being open with what we’re working on, and if we’re lucky that helps spark conversations that help move things along. 

    What is “democracy” at mySociety?

    “We should do some Democracy monthnotes” is a sentence that makes perfect sense internally, but for some quick scene setting…

    In principle, everything we do is in some way about democracy —  we consider our FOI and transparency work to be important in part because it enables and grows civic ideas of democracy. Key to our climate work is the idea that democratic and climate problems are linked, and so our climate work is very engaged with the kinds of problems of democracy we’ve been thinking about all along — but with a sharper focus.

    Internally, we tend to think about our democracy work as being around TheyWorkForYou  and WriteToThem, and internationally looking at similar “parliamentary monitoring organisations”. These are some of our longest running services, widely used, and with a long potential future ahead of them. One of the things we’ve been doing over the last year is creating a clearer idea of what we want to accomplish with PMO work. Lots of this work has been behind the scenes in funding bids – but can be seen in the work adding the Senedd as a general direction of travel. More on what we’re currently working on (and some things that didn’t work out) in future. 

    Behind the scenes, there isn’t a Democracy “team” as such because we don’t currently have the funding available for that. My estimate is that last year we probably had 1-1.5 full time equivalent (FTE) people working on Democracy – but that was spread over 5-6 actual people. Given the porous lines between the different things we do, the immediate goal isn’t to get a big team, but to be increasing the consistency with which we can use the wide range of skills already in the organisation — and in making links and making the most of opportunities across our wider work. 

    So “Democracy” at mySociety is always going to be a little fluid —  we’ll use these monthnotes to be clearer about what that means in practice. 

    All the conferences

    The Democracy Network held its second conference this January, attended by 10% of mySociety (three people). 

    This is an interesting crowd that is, for obvious reasons, moving into being quite election focused. By contrast, a lot of mySociety’s work is about an effective civic democracy between elections. Many WriteToThem and TheyWorkForYou features become less useful in an election, while traffic increases and usage changes. Once Parliament dissolves ,there are no MPs until new ones are elected, but what those MPs have been up to is important. We’re doing some thinking on our options for running the most useful version of the site during an election, and have picked up some conversations at the conference to continue. 

    Julia also went to the Democracy Classroom strategy day, where she spoke on a panel about using data in campaigning alongside our friends at Generation Rent and the Democracy Club. Our hosts, the Politics Project, brought together organisations of all sizes and types, from all across the UK. It was great to have a really practical conversation about the data needs of organisations working with young people, and after a follow up chat with Gaibhin from United Response, we’re already working on adding census disability data to the Local Intelligence Hub service we’re launching soon. 

    We’ve also had the mySociety quarterly team meeting, where we all head somewhere in the country (this time, Leeds!) to talk for a few days with colleagues we generally only see on the internet. They’re nice! 

    Register of interests

    Great to put out our spreadsheet version of the register of members interests

    Fun fact: this has been mostly ready to go since October —  part of not having a lot of funded time for Democracy work means there’s a backlog of 99% finished work to get out the door. 

    But the response to the blog post shows the value in getting that out, and in being transparent in general. Lots of nice comments from people who think they’ll find it useful – but it’s also leading to more conversations with people with an interest in the register that can help us get a better sense of what’s currently happening where, and what our role in the picture might be. 

    For keeping track: On my bit at the end about the kinds of questions people might want answered, Stuart from Open Innovations has linked me to some of their old weeknotes on work they did with PDS and the HoC Library on the kind of questions that can be answered through the current data.

    Voting records

    One of the big things we were doing behind the scenes last year is reviewing and updating our approach to voting records. This is one of TheyWorkForYou’s most notable features, and we got a grant from Newby Trust to have a good look at these and used some of our grant from the Porticus Foundation to do some more involved technical work than we may have otherwise been able to manage. January has been the fiddly final stages of getting this to launch. 

    I’m not going to go into a lot of details here (there are long blog posts to come) – but the big task has been thinking through what we’re trying to achieve, and then untangling our technical systems to make that sustainable over the long run. 

    Our current system is based on various data flows in and out of the Public Whip – which has a complicated history with TheyWorkForYou. It has some overlap with the people who founded it, it’s not run by us, but at the moment is substantially kept updated by Matthew’s work unclogging the ParlParse system the two sites have in common. Some of the things we want to change would need changes deep in the Public Whip, which we can’t do, and that’s bad for what’s such an important feature of the site. 

    As such, we’ve made a transitional replacement for the Public Whip, where we can build in the kind of analysis tools we need to have more visibility and control of the whole process.Over this year we’re going to be talking to people who want more of the kind of number-y analysis the Public Whip does well to tidy up what we’re using internally — and set it up as a useful specialist complement to TheyWorkForYou. 

    TICTeC/PMO Communities of Practice

    TICTeC is back in London! See the call for proposals about the conference itself – where we’re especially trying to think about how/if/where civic tech is relevant to themes of democracy in crisis, and democratic approaches to the climate crisis.

    As part of this funding, we are also setting up communities of practice around access to information and parliamentary monitoring organisations. A big bit of January for me has been (working with the Civic Tech Field Guide) making a good list of PMO organisations from around the world to approach to get a sense of what problems we might discuss over the next year. I’m currently working through the survey responses to that. 

    New combined authority

    Welcome to the new “York and North Yorkshire Mayoral Combined Authority”  — added to our big list of local authorities, our IMD dataset, and nearest neighbour dataset – and to CAPE – our local climate action tracker – where you can see some of these datasets in practice.

    CAPE has some features helping navigate the connections between authorities and combined authorities, and in general we’re trying to think about how we can better reflect Combined Authorities in our core work. 

    A key use of WriteToThem is a “here is the structure of government where you are” — and we haven’t added the new CAs because unlike the London Assembly they don’t have as clear a public facing representative. The long term solution here is either to lean into WriteToThem having information for people you don’t necessarily “write to” or building a clearer page for this into TheyWorkForYou – which does some version of this for devolved Parliaments/Assemblies. We’re thinking about it. 

    Making progress

    In any given month, we’re generally making incremental progress on things we think are good ideas, that might also importantly be fundable ideas on where we’re well placed to make something better

    Julia’s been developing more about what our approach to training might be — making more of the fact our tools are already used by educators, and building a better loop between that and our service development. 

    I’ve been developing our thoughts on the register of interests further, and reading through the Jo Cox Foundation’s new report “No place in politics: tackling abuse and intimidation” (which I thought was measured, and well-thought through) and making some notes on how it applies to our work. 

    We’re also thinking more about how practically we can try and increase support from the public for our services. Here moving a bit away from “Save TheyWorkForYou” language to being clearer about how what we do is part of making things better (regardless of who wins elections), and that we want to be far more ambitious than keeping the lights on.

    Anyway, if you’ve read this far — donations are welcome. But also get in touch if you’ve got something to chat with us about!

  3. Improving the register of MPs interests

    Tl;dr: We’re now releasing our register of interests data as a spreadsheet.

    High quality data about the external interests of our MPs and ministers is vital to identifying conflicts of interests, and discouraging politicians from having conflicts of interest in the first place.

    Lack of clarity on the interests and income streams of MPs is a corruption risk. The problem with second jobs and outside interests is less that MPs might be distracted from their main job – but that when they stand in Parliament, they may be representing groups beyond their constituents, asking questions (or not asking questions) depending on their outside work.

    When outside interests exist, it’s vital they are clear and transparent. The Register of Members Interests contains a list of disclosures MPs are required to make of financial interests or benefits which “others might reasonably consider to influence his or her actions or words as a Member of Parliament”. Following the Owen Patterson scandal, there was renewed interest in this data, as it was clear that there were a number of potential stories and scandals hidden in plain sight – just requiring someone to join up the data.



    Building a data ecosystem

    A key problem is that the data is not easy to work with. The data is released (roughly fortnightly) on the parliament.uk website as a HTML document for each MP. This process technically releases the information, but makes it hard to compare releases of the same MP over time, or to make comparisons between different MPs.

    TheyWorkForYou improves on this by creating structured data from the HTML release. Using this we can highlight the changes in each release from the previous release. This is useful for journalists and campaigners in quickly understanding what has changed in each release. For instance, the change in Rishi Sunak’s register over time can be seen here.

    We want to avoid people doing the same work of cleaning the data over and over. We make our version of the data available publicly, so other people can use our work to do things that we haven’t done ourselves. For instance, Open Innovations have built on top of the data we publish to link the data to other datasets and create a Register of Members’ Financial Interests Explorer.

    While projects like the Tortoise/Sky News Westminster Accounts create new value in joining up datasets and cleaning the data for their own work – ultimately the new datasets they have created are only usable by those organisations. That’s their right as the people doing the work – but we think there is a bigger (and more sustainable) impact to be had in improving the data in public.

    Making our data more accessible

    Previously, we have published our interests data as a series of XML files, which is useful for programmers, but harder for other specialists to work with. We did some thinking with OpenDemocracy last year to explore if there were small changes we could make that would make the work we already do more useful.

    As well as the XML files, we now publish an experimental spreadsheet version of all data since 2000, and the register for the current 2019 Parliament.

    These sheets show the earliest and latest disclosure of an interest, and include some (very) basic NLP analysis to extract mentioned orgs from the free text and make it easier to quickly parse when scrolling.

    This data can also be explored through Datasette, which can be used to query the datasets in the browser, and save the queries as links that can be shared.

    For instance, the following links go to specific queries (we’re using an in-browser version for prototyping and this might take a minute to load):

    We want to continue to improve our approach here – and welcome feedback from anyone this spreadsheet helps.

    Parliament can do better data publication

    A key problem run into by everyone working with the data is that it’s broken to start with. MPs fill things out in inconsistent ways that makes the overall data different to analyse without cleaning first (see both the Open Innovations and Tortoise/Sky News methodology notes). Fixing this up is a key first step towards aggregate analysis – and the easiest place to fix it is with validation when the data is collected at the start.

    While work can be done to improve the data after the fact (and experiments with Generative AI have found it to be quite good at fixing inconsistent formatting), improving the initial data collection is the most effective way of improving the quality of the data. There are active moves in Parliament to fix some of these problems. Producing more information in machine readable formats, and adding methods to make sure the data is correct to start with, will make the transparency process simpler at every stage.

    Similar issues apply to the register published for All-Party Parliamentary Groups (APPGs), which should publish as “machine readable” data the range of data that the groups are formally supposed to make publicly available. APPGs are semi-official groups that MPs can form around specific interests or issues. Many of these are useful ways of having discussions, but these can also be an avenue for corruption, with outside interests supporting the group and its activities. The register includes the officers of groups and financial assistance and gifts received by the group – but not the overall membership. APPGs are separately required to disclose their wider membership on their website (or if they don’t have a website, if someone asks) but this isn’t included in the register, and so can’t be consistently scraped to produce data. While MPs are supposed to disclose benefits from groups on their individual disclosure, clearer data on what is officially “public” memberships would help ensure that there is nothing missed between these two datasets.

    Separately there is a register of ministerial interests that applies to MPs who also have government positions. This is in principle more strict, requiring disclosures of relevant interests of family members, and avoiding even perceived conflicts of interest. However, in practice the information does not contain the specific financial value of gifts or benefits, just that they exist. The disclosure cycle is also longer, being published every six months rather than monthly. In practice – this means that relevant interests may not be public for a significant time after a minister is appointed (and potentially never published, if the minister has again moved on by then).

    There is a lot of work that can be done from the outside to build on official data. But the more Parliament does things that it is uniquely able to do, the more we can focus on analysis and data comparisons that are best done outside.

    What mySociety can do

    A very basic thing we can do is beat the drum (and work with those who have been doing this for ages) for better publication of data from Parliament.

    But if this happens or not, we can do work to make the data better. If it looks like Parliament’s data is unlikely to be fixed at the source, then a project of improving the data in public in a way that multiple projects could then build on would be useful. But if the data gets better, then we can better spend our time doing more work on top of this data. This might include joining up the official data with other datasets (including those of the UK’s other Parliaments and Assemblies) to draw out connections and better analysis.

    But our work here isn’t just about producing good data – it’s about displaying it in a way that’s useful and understandable by people. Chris Bryant MP (former Chair of the Standards Committee) has argued that Parliament’s own display of the history of registers should match what’s provided by TheyWorkForYou. If Parliament improved its own display to the public of registers of members’ interests this would be fantastic news – and we in turn would need to think about if there are new approaches that would be useful on top of that.

    One approach we are thinking about would be to find out what people wanted to know the answers to about their MPs interests, and then using volunteers to answer a set of common questions. This is the kind of editorialising that Parliament itself would find much harder to do – while providing something different from aggregate analysis of the data all together. This is something we could do with the data as it exists, but is something where better data would let us create new tools so volunteers could answer more complicated questions.

    Making MPs’ interests clearer and easier to understand is key to spotting conflicts of interest and keeping politicians accountable. We hope our new spreadsheet version of the data helps make the work we’re already doing more useful and accessible – while we think about the road we want to take in future to improve TheyWorkForYou and the project of a transparent democracy.





    If you'd like to see us extending our work in democracy further, please consider making a contribution.
    Donate now

    Image: Wilhelm Gunkel on Unsplash.

  4. How we improved CAPE this year

    CAPE is our data-packed website that makes it easier for everyone to track and improve local action on climate change, built in collaboration with Climate Emergency UK.

    When CAPE launched, it was basically a list of councils’ climate action plans — but we’ve continued to make regular improvements and additions, and it’s better, and more useful, than ever.

    If you’re a council officer with climate duties; a researcher looking for climate data; an activist needing facts to back up your climate campaign; or simply an interested citizen, you’ll find some truly invaluable facts and figures on CAPE.

    And if you haven’t visited for a while, you’re in for a surprise. Here’s what we’ve added this year:

    Improved search

    Search results for 'flooding' - these include terms such as 'drainage' and 'river'

    Now you don’t need to think of all the different ways that councils might have described the concept you’re looking for. We’ve integrated machine learning into the search function, to help surface your search term even in documents that use different words for the same concept. Interested to know more? We wrote it up in detail here

    Updated for new councils

    In April, a set of UK local authorities was abolished and merged into new unitary authorities.  When this happens, we do work to update the data that CAPE depends on. You can make use of it too!

    We’ve updated:

    And this has all fed back into CAPE to make changes of local government structure transparent in the site. 

    Local polling

    We wanted to make it easier to see where there is support for climate action, so we took a few sets of MRP polling on people’s attitudes to net zero and energy sources and converted it from per-constituency to per-local council.

    For each local authority (except in Northern Ireland) – see Croydon, for example – you can now see estimates for support for net zero (even when it is described as expensive), and for different kinds of renewable energy project.

    Support for net zero in Croydon

    Not sure what MRP polling is? We wrote something up about that

    Climate Assemblies

    Local climate assemblies at CAPEIn 2019, we were part of the UK-wide Climate Assembly, and we’ve kept an interest in citizens assemblies about climate change. There’s been one held for the whole of Scotland, and more widely we’ve found 17 (so far) held by local authorities across the UK.

    We’ve uploaded the final reports of those assemblies into CAPE. It is now easy to see which authorities have held climate assemblies, and to search the results of those assemblies. Find all this at cape.mysociety.org/assemblies.

    Hundreds of new documents have a fresh coat of paint

    Thanks to people telling us about new plans, and us conducting an extensive search for new documents across council domains, we’ve updated and added hundreds of new documents to councils’ pages. 

    With all that content, we’ve gone back and tidied up the design of our council page layout, giving us more space to add explainers and summaries to each section. Why not take a look for your area?

    Council Climate Action Scorecards

    Let’s not forget that while all this activity has taken place on CAPE, we were also helping Climate Emergency UK with the Council Climate Action Scorecards. At the time of their launch in October, we wrote about the part we played in building the site, wrangling the data, advising on the methodology and building a scoring interface.

    The result of all this is, of course, that whatever your climate interests – be you council staff, researcher, journalist, campaigner or interested member of the public – you can dive in to any council’s page to see how they’re doing against several markers of climate action.

    Since launch there has also been the significant addition of Question Pages, allowing a never before seen view of how each council scores on every individual action point. Julia wrote about that in detail, in this post.


    Your donations keep our sites running.
    Donate now

  5. Providing tech for effective crowdsourcing

    An important idea in mySociety’s history and work is that combining people-power and technology is a powerful method to make change.

    Recently we’ve had the chance to work with Climate Emergency UK on the council climate scorecards. As part of this we’ve been learning from how CE UK works with volunteers, and thinking how best we can be a technical partner to crowdsourcing projects.

    But we’ve also been thinking how we can better incorporate crowdsourcing into our core services. This has led to improving our approach to crowdsourcing within WhatDoTheyKnow, and developing a new platform (GRACE) to help volunteers crowdsource information for the scorecards.

    Developing our technical approach

    In the first round of the Council Climate Scorecards,back in 2021, the process was managed on a big Google Sheet. Each answer to a question has a line, which tracks the answer from a dropdown and keeps track of notes, and we use macros to populate and move information around. This worked and let us get started at the pace we wanted, but introduced a lot of room for error, and meant more work was needed at the end of the process to validate the data.

    For the second round of scorecards, with a more complicated set of questions, we wanted to move away from the spreadsheet approach, into something that could validate the data as we went, and integrated the right of reply process into the same platform.

    Working with CE UK, we built a crowdsourcing platform GRACE (named after CE UK’s first volunteer) to replace the old spreadsheet approach.

    The overall process worked like this:

    • Working with their advisory group, CE UK created questions across a range of different areas, with different sections for different kinds of councils.
    • These questions and data from FOI responses were uploaded into the platform.
    • Volunteers were given access to the platform, and could answer questions in the topic areas assigned to them. This would include an answer from a dropdown, a public “evidence” field, and a private notes field to share between volunteers.
    • Overall progress across all areas could be seen by the volunteer coordinators.
    • Local authorities were given a right of reply per question. Emails were sent to all local authorities, giving them access to the platform to review the scores and evidence for their section. This gave them the opportunity to submit new evidence where they felt the score did not reflect their actions. 74% submitted responses to the right of reply.
    • Responses from the right of reply were reconciled with the original answers by a group of volunteers and CE UK.
    • This data could then be reviewed and loaded into councilclimatescorecards.uk

    How did CE UK find the tool? Isaac Beevor, Co-director of CE UK:

    GRACE – our online data collection system was incredible. We would not have been able to collect the thousands of data points on local authorities’ climate action without it. It was simple to use and effective so all of our volunteers were able to use it, with minimal training. Furthermore, it allowed us to coordinate and track progress. This meant we were able to encourage volunteers who needed it and track progress in sections allowing us to plan ahead for staff and volunteer capacity. We then used it to allow access for councils in the Right of Reply and all of the feedback from officers, particularly its ease of use, was positive. The best thing is we can now use this for every Scorecards, which saves us so much time and energy.

    Effective crowdsourcing

    As we wrote in our report about public fragmented data the issue with the idea of ‘armchair auditors’ is not that they do not exist, but that they were thought about in the wrong way. People can and do use their time to support civic accountability through looking at spreadsheets. But they need to be given support and structure to work effectively together.

    Reflecting on the CE UK process and other crowdsourcing approaches we admire, like Research for Action and Democracy Club, what these projects have in common is that they involve knowledge sharing and collaboration across the country, with volunteers themselves contributing a local or specialist focus. “Armchair auditors” aren’t atomised individuals, but work together as a community.

    The Effective Crowdsourcing diamond (below) describes the aspects we think are key to successful projects, and helps us shape our thinking about future crowdsourcing/citizen science projects.

    A blue diamond with the following text "Effective crowdsouricng: working together to answer big questions". The four points of the diamond have: Volunteers: A pool of motivated supporters break the problem up Expertise: Experts and advisors to frame the questions to ask Technology: specialist tools streamline information gathering and quality assurance steps Impact: clear paths form gathering data to making change.

    In this framework, an effective process will have four features that all interrelate and reinforce each other:

    • Expertise – Helping move from high level principles to concrete questions and approaches that inform how work can be split up for volunteers, and the technical tools needed.
    • Volunteers – People who care about an issue, with the skills to contribute to answering questions.
    • Technology – Technology makes it easier for people to work together, and validation and cross checking improves the accuracy of the process, enabling more complex approaches.
    • Impact – What is the path from the result of a project to change the world? Clear routes to impact means clearer benefits of engaging to experts and volunteers.

    For Scorecards and other projects, we have provided the technical side of the diamond. But there is lots of potential for us to run crowdsourcing projects that improve our core Democracy and Transparency services. Starting with the tools we now have, we’re thinking about how we can strengthen our skills and approaches across the diamond – and how we might partner on projects to bring in other expertise and skills.

    GRACE screenshots

    Initial volunteer screen showing sections assigned:

    Screenshot of a page showing the assignments and completion rates assigned to a specific indivudal. A series of progress bars next to sections.

    Screen showing councils and progress

    Screenshot of a pages showing the overall progress for a section. It shows by council the number of questiosn completed for this group of questions

    Marking screen

    This shows the marking screen - which is a form capture the answer to a question, with boxes and links for evidence

    Right of Reply screen

    This is the right of reply screen that will be visible by councils - it shows the answers to the question in uneditable boxes -while providing options to agree/disagree with response and submit further evidence

    Admin screen so you can check that how much has been assigned to volunteers

    Volunteer management screen - showing how many sections are currently assigned to volutneerws

    Admin screen to show progress on an authority level

    Progress screen, showing the number of questions completed by council, and the overall progress

    Or on a section level

    Progress screen by question section, showing the number of councils with a complete set of questions for that section.

    Image: Clay Banks on Unsplash.

  6. CE UK and mySociety are using people power and Freedom of Information to bring transparency to local climate action

    A story in this week’s Financial Times [paywalled] has brought the EPC ratings of council-owned properties into the public conversation. This story was based on data obtained through FOI requests as part of the Council Climate Action Scorecards project, which we’ve been working on in partnership with Climate Emergency UK (CE UK).

    What you can read in the FT is one story pulled from a wealth of data, but there’s more to come. Our WhatDoTheyKnow Projects tool allowed CE UK’s team of volunteers to conduct a nationwide survey of every council through well-placed FOI requests covering the use of renewable energy, plans for retrofitting, green skills training, road expansion and more. 

    The data they gathered has allowed for the understanding of councils’ action on a nationwide scale. This level of oversight has not previously been possible: as with so much about the Scorecards project, it is allowing councils to take more informed action on climate, and individuals to clearly understand what is being done.

    Why local action matters

    In the UK, it is estimated that around one third of carbon emissions are in some way under the influence of local authorities. 80% of UK councils have declared a ‘climate emergency’ to indicate they recognise the scale of the problem of climate change, and are in a position to take practical steps to be part of the solution. To help local authorities achieve the goals they set themselves (and to push them to go further), we need to engage with the plans that local authorities are making, and the actions they are starting to take. 

    In 2021, CE UK and mySociety worked together to produce the first Council Climate Plan Scorecards. CE UK’s upcoming launch is the second iteration of the Scorecards. It is much bigger and more ambitious in scope than the last: it scores not the plans, but the climate actions of every local authority in the UK. 

    FOI requests were just one part of the process. As well as giving CE UK access to WhatDoTheyKnow Projects, we developed a crowdsourcing tool for volunteers to use while marking across the 90+ datapoints collected for each council. 

    How do you score action?

    CE UK moved from scoring plans to scoring actions. That required new approaches to gathering the information. 

    The questions CEUK used in the new Scorecards are the result of a long and thorough process of research and refinement. Building on their own research and expertise, they conducted one-on-one consultations with approximately 80 organisations and sector-specific experts. An advisory group of environmental and local government experts provided further discussion and refinement, to help build a list of questions that would practically be possible to answer, and that would reveal important information about the climate actions of councils. 

    The aim was to identify areas where information was publicly accessible; but also where gaps existed, especially in operational matters that aren’t often made public. Additionally, CE UK wanted to investigate whether councils are truly implementing the actions outlined in their climate action plans, including aspects like lobbying for additional powers.

    Making use of Freedom of Information

    Freedom of Information laws means that a huge range of information held by public authorities (including local councils) can be requested by any person who asks. This provides a legal tool to create greater transparency where information is not being published proactively.

    For CE UK, the potential of FOI for the Scorecards project was clear – but there were concerns. In consultations with council staff, there was pushback regarding the use of FOI requests due to the potential time and financial burden on council officers who work on climate – with some requests for a more informal survey approach to be used. But the drawback of that would be making good data dependent on goodwill everywhere. FOI requests provided a way to make sure the scorecards were not just effective for councils who engaged with the process and provide an approach that was fair across the country. 

    To balance a process where they want to encourage positive engagement from councils, with one that works without that, CE UK’s approach was to plan out the most efficient and least burdensome use of FOI requests. 

    Based on feedback from the advisory group, and trial runs to a small number of councils, they eliminated questions that were less important and useful, made more ‘yes/no’ or ‘single number’ responses, and learned where certain questions weren’t relevant to certain areas or groups of councils. 

    The subsequent FOI requests became more streamlined, and this resulted in quicker response times for the final requests than they had in the trial – as the information sought was more direct and concise.

    In the end, CE UK submitted a total of over 4,000 FOI requests to councils across the UK. The questions were divided into 11 categories, with some being specific to certain types of councils, such as district councils or combined authorities. The next stage was taking these 4,000 requests and getting them into a form that can be used for the scorecards. 

    Crowdsourcing and review process

    CE UK used WhatDoTheyKnow to manage their FOI request process. mySociety’s WhatDoTheyKnow acts as a public archive for requests – requests made through the site have the responses shown in public to bring more information into the open  – making it more discoverable by other people interested in the information, and reducing the need for duplicate requests being made. As of 2023, a million requests for information have been made through the site, with hundreds of thousands of pieces of information being released. 

    A feature we are trialling with a range of organisations is WhatDoTheyKnow Projects, which integrates crowdsourcing tools into WhatDoTheyKnow, and allows the task of extracting information into a new dataset to be spread out. The goal is that this helps organisations be more ambitious in finding out information and helps people work together to create genuinely new and exciting datasets, that no single organisation has ever seen. 

    As CE UK’s approach already made heavy use of volunteers and crowdsourcing, this was a natural fit.  Alongside a wider group of 200 volunteers working on getting answers to the other questions, 15 volunteers specifically worked on the FOI requests. These volunteers were a mixture of people with prior experience or professional interest in FOI requests, campaigners well-versed in FOI processes, and individuals new to the concept but eager to engage in activism.

    After the crowdsourcing of FOI data was complete, it joined the rest of the data in the new tool mySociety had developed for helping volunteers crowdsource information for the Scorecards.  

    From here, councils were given access to the data collected about them and given a right of reply to correct any inaccuracies or point towards information not previously discovered or disclosed. The results of this process will then be reviewed to produce the final Scorecards data, which will be launched this month.

    But the Scorecards data will not be the only useful thing that will come out of this process. Because of how WhatDoTheyKnow was used, to see evidence supporting the final Scorecards, people will be able to click through and see the original responses, for instance, to see what councils have lobbied on support for their climate work. 

    Some of the FOIs are being used to construct datasets that have a broader impact, and here we come back to that FT story on the Energy Performance Certificate (EPC) ratings of council-owned houses. Building these new public datasets will be useful for councils to understand their own situation, and as we see with the news story, more broadly to understand the challenges ahead for local governments to meet net zero emissions goals. 

    Onwards

    The original Scorecards project has already been influential on how local governments understand their own plans, and how organisations like the UK’s Climate Change Committee understand the role and progress of local government in the challenges ahead. When the next generation of Scorecards is released, we hope that they continue to be useful in shaping and improving local government action around climate change.

    mySociety believes that digital technology can be used to help people participate more fully in democracy, make governments and societies more transparent, and bring communities together to address societal challenges.

    The Scorecards project showcases how the combination of digital tools, people power, and the right to information produces powerful results. We hope that the impact of this project can inspire and make possible similar approaches for other problems, or in other countries.

  7. ICO advisory note on publishing spreadsheets

    Following the PSNI and other recent data breaches, the ICO has issued guidance to public authorities. This guidance suggests a temporary stop on publishing Excel-style spreadsheets in response to FOI requests made via online platforms like WhatDoTheyKnow. The full advisory note is available online

    The advisory note emphasises that this is not a reason not to disclose requested information. Instead, the ICO says to release the information from original source spreadsheets as a CSV file – a simpler format than Excel Workbooks, with less potential for including hidden sheets or metadata that can lead to an accidental breach.

    A focus on file formats is a blunt measure, and one that will need to be superseded by better procedures and technical processes.

    We support authorities releasing data in the most appropriate format for the information being requested. This may sometimes mean an extract from a table, and sometimes a complete document. Excel spreadsheets are legitimate public documents, and information released in this format can be hugely valuable. It’s important to develop processes where they can be released safely. 

    Significant data breaches involving Excel files clearly show the risks when data management and release processes fail. These include not just breaches we see through WhatDoTheyKnow, but through disclosure logs and releases made directly to requesters. This is an opportunity for public authorities, the ICO and us at WhatDoTheyKnow to reflect on how we can best deliver the huge benefits of public transparency while safeguarding personal data. 

    Modern authorities need to be good at handling data. Data breaches happen at the intersection of technical and human processes. The FOI team can be the last link in the chain of a data breach when they release the information, but the root cause often goes back to wider organisational issues with the handling of sensitive data.

    In the short run, the ICO has recommended training for staff involved with disclosing data. Many teams already have excellent processes and do excellent work, but all authorities should take this opportunity to consider their responsibility on the data they hold, and have appropriate processes in place.

    Long term progress means developing good universal processes that keep data safe, regardless of the format of the data or how the data is released. All FOI releases should in principle be treated as if they are being released to the public, because the authority’s ability to stop a data breach ends when the information is released. Making FOI responses public produces huge efficiencies for the public sector, increasing transparency in practice, and multiplying the benefit to society of the information released. 

    Technology can also be part of the solution – we need to understand more about why existing technical ways of removing hidden information from Excel spreadsheets are not being used (as described in the ICO’s established guidance on disclosing information safely), and how new tools or guidance can make it easier to release data safely. 

    A core part of our work at WhatDoTheyKnow is dealing with the practical reality of promoting public transparency while protecting personal information. We take data breaches seriously and have processes in place for dealing with them as promptly as possible. We continue to plan and work to help reduce the occurrences and impact of personal data breaches through both our procedures and technical approach. 

    By monitoring how authorities respond to requests on WhatDoTheyKnow, we will seek to understand how this guidance is working in practice, and engage with the ICO and other organisations to promote effective long term approaches to this problem. 


    Notes on the content of the advisory

    Below is our understanding of the advisory note by subject matter:

    Freedom of Information requests

    • Continue to comply with FOI responsibilities. This guidance is about releasing information in a way that reduces risk of accidental disclosure. 
    • Temporarily, do not release original source spreadsheets to online platforms like WhatDoTheyKnow. Instead – convert and release to CSV files.
    • If that is not possible, then:
      • Ask if the Excel sheet can be sent to a separate (non-public) address. Proceed with the original address if they ask for this. 
      • In all releases, go through processes to ensure there is no data breach in the material. 

    General data management

    • Excel files are unsuitable working environments when they become very large (hundreds of thousands of rows). Authorities need to switch to appropriate data management systems that are more appropriate for managing larger amounts of data.  
    • Staff who use data software and are involved in disclosing information need continuous training.  
    • Understanding of pivot tables and their risks should be incorporated into data management.

    The ICO plans to update their guidance on Disclosing Information Safely

    The checklist released accompanying the advisory has several useful steps on checking for hidden data in Excel sheets. However, on the ‘considered alternative ways to disclose’ step, refer back to the steps in the advisory note. Information converted to CSV can be released to WhatDoTheyKnow in compliance with the advisory note. The advisory note says that the source dataset should continue to be released to WhatDoTheyKnow if it cannot be converted, the requester does not want to use an alternative route, and the authority is confident it does not contain a data breach.

  8. Navigating the new constituencies

    This blog post is part of our Repowering Democracy series. This year we will be publishing a series of short pieces of writing from mySociety staff and guest writers who are thinking about how our democracy works and are at the frontlines of trying to improve it.

    This week our senior researcher, Alex Parsons, shares data we’ve produced about new constituencies, and how we should steer the process of boundary reform towards making our politics easier to navigate.

    The current set of parliamentary constituencies is being replaced for the next election. For some, the main effect is a name change, but for others the borders of constituencies will change substantially.

    Boundary reviews are carried out by four separate organisations — one for each nation — who each set their own boundaries. This means that no-one officially produces a single dataset of all the new constituencies covering the whole UK.

    That seems like a useful thing to have, so we’ve created it. We’ve made:

     

    This contains both the official GSS id, and a unique ID based on the three letter IDs for new constituencies created by Philip Brown and Alasdair Rae for their hexmap of the new constituencies. This is useful in some instances because the GSS codes are not unique to the new constituencies (some Scottish constituencies are unchanged).

    Screenshot of a postcode in mapit, showing the different kind of boundaries available.



    Political equality means more than equal seats

    There are 650 MPs in the UK Parliament, each is elected from a constituency, and each constituency only has one MP.

    The main change in the new rules is reducing the tolerance for differences in the number of registered voters between constituencies, and ending the previous separate weighting of Wales. This means that some areas are affected much more than others by the change – with the number of Welsh seats reduced by eight, and Scotland losing two seats in total.

    Boundary changes aren’t just a technical process, but have impacts on the results of elections. In the UK, politicians can’t directly draw boundaries, but this doesn’t mean they’re not a political choice. The boundary commissions follow the rules they are set, but what these rules are (and how often they happen) are the subject of political debate where everyone has one eye on the outcomes that different sets of rules produce. The debate about “equal size constituencies” versus boundaries that reflect “natural communities” is in part about different perceived partisan advantages of drawing lines in different ways.

    Advocates of constituencies of equal size argue that this is about political equality. Now, we like political equality, but taking this argument seriously should lead you way past equal size to supporting a move to proportional representation. In practice, everyone in this debate accepts trade-offs between political equality and other factors. If we’re not going to have proportional representation, we think clear lines between different levels of government are features that should have real weight.

    Effective and understandable layers of representation

    We are in favour of layers of representation that are effective and easy to understand. Through postcode lookups on TheyWorkForYou and WriteToThem we can explain the overlaps for a user’s own postcode, but the simpler the system the easier it is for people to understand what is happening – whether or not they visit one of our websites.

    When boundaries are less complex, it’s also easier for people working inside the system to understand how the pieces fit together. Tighter requirements on populations means more constituencies will cross local authority boundaries. Using our new datasets, we can say that the number of constituencies that have more than 5% of its population in at least one other local authority has increased from 26% to 38%.

    This means there are an increasing number of MPs who have a harder job than others – working with different or multiple local authorities depending on the issue at hand. Knowledge and understanding of local systems is much more complicated for these MPs, and getting problems to the right place is more challenging for their staff.

    Number of local authorities Current constituencies % Future constituencies %
    1 478 74% 404 62%
    2 155 24% 220 34%
    3 16 2% 26 4%
    4 1 0% 0%

    Tighter requirements also mean more frequent changes. The current timetable will lead to this process being repeated every other election, disrupting understanding of constituencies and processes of accountability.

    The reason we’ve produced this data in the first place is to help organisations we are working with transform information they have about current constituencies into information that is useful for new constituencies. Institutions inside and outside the formal political system develop an understanding of the country that is disrupted by changing boundaries.

    This problem also applies to the understanding MPs have of their own area – both in terms of learned understanding, and statistics and reports created to inform them. Changing boundaries means everyone has to change their understanding of what a constituency looks like. This is all bad from the point of view of effective understanding and interrelation of layered government.

    Something that should be key when designing our political system is making sure that it can be understood by citizens and representatives, and supports effective communication between layers of government.

    If we designed our institutions and boundaries to be easily navigated, rather than meet mathematical rules, what would that look like? Certainly there should be better alignment between different layers of government, but can we go further than that? We want better postcode data so that we can fix the problems of when the same postcode is in multiple areas, but from a public understanding point of view – why shouldn’t the line drawers respect postcode boundaries in the first place? They’re a lot more real to people than the process that produces our boundaries now.

    The way we draw our boundaries is part of wider arguments about the different priorities we have when we design political institutions – and the idea that things should be easy to understand and navigate is currently undervalued.





    If you'd like to see us extending our work in democracy further, please consider making a contribution.
    Donate now

    Image: Héctor J. Rivas on Unsplash.

  9. It should be easier for MPs to vote

    This blog post is part of our Repowering Democracy series. This year we will be publishing a series of short pieces of writing from mySociety staff and guest writers who are thinking about how our democracy works and are at the frontlines of trying to improve it.

    As part of this work, we are working to solidify our views on where Parliamentary reform would work in a complimentary way with the principles behind our services. This week our senior researcher, Alex Parsons, discusses how changes to how MPs vote can improve both transparency and the parliamentary working culture.

    Improving how MPs vote isn’t just about adopting new technology. The sudden move in the pandemic from MPs physically packing into voting lobbies to voting on their phones felt like catching up with a century of technology all at once. But that period was over quickly and things are now (roughly) back to how they were before.

    This is because parliamentary processes also reflect political culture. Thomas Edison proposed an electronic voting system for the US Congress in 1869, but one wasn’t actually implemented for 90 years after that. The technology existed, but that wasn’t enough.

    New technologies can change who holds power and threaten how things work. Decisions about technology become wrapped up in fights to preserve or change political culture. When thinking about technological changes, we can’t just approach it as a project of modernisation – we need to have a view on the culture we want to create.

    Our view is that when MPs vote, we want processes that create transparency on the votes of individuals, that help create an effective working culture, and are sensitive to the circumstances of representatives’ lives.

    These elements are not separate. Both transparency around voting and sensitivity to the lives of representatives are important to creating an effective democratic working culture. The approach needs to balance the fact that MPs are standing in for the rest of us, with the fact that they are also people, and are entitled to decent working conditions like anyone else.

    This leads to three recommendations on how the UK Parliament should handle its internal votes:

    • The House of Commons should in normal circumstances, defer votes to a standardised voting time (within ‘core hours’), where multiple votes are held in succession.
    • These votes should be held through a fast electronic means – whether through terminals, voting pass systems, or apps.
    • Current proxy voting schemes should be extended to personal discretion to designate a proxy – e.g. a set number of days a year a proxy vote can be allocated, no questions asked.

    In this week’s blog post, I’m going to walk through a bit of the backstory to this thinking, and the benefits of this approach.



    How votes work now

    Currently, when a decision point is reached in a debate, there is a “voice vote” in the room to test if anyone objects. If they do, MPs vote by physically moving into two rooms connected to the debating chamber. Two different systems kick in here – MPs tap their passes on readers to record how they individually voted, while Parliamentary staff count the number of people who entered to report back for the purpose of the vote. This takes quite a long time, and a vote can last around 15 minutes. Part of this includes an eight minute window for MPs to reach the lobbies from elsewhere in the Parliamentary estate.

    The current system isn’t pointlessly antiquated and slow, it is antiquated in a way that supports the way power currently works in the Commons. A slower voting process discourages too many votes – which reflects the reality that non-government approved amendments do not win, and MPs aren’t really engaged in a functional process of improving legislation. The lobbies themselves are defended as an opportunity for MPs to talk to ministers, but the influence also works the other way. The lobby system is a physical and intimidating process of sorting that enforces party unity and discourage rebellion. It is a more difficult thing for someone to walk into a room of their enemies, while being shouted at by their friends, than it is to press a button.

    Any change from this system has the potential to disrupt how power works, shifting power away from a shared consensus of party leadership (who, regardless of the specific issue, are in favour of MPs doing what parties say) and enabling more individual action by MPs.

    Our view is that this broadly would be a good thing – empowering individual MPs and Parliament collectively over the party leaderships would enable a culture closer to what citizens collectively want from MPs. Importantly, it also helps create a better working environment for MPs themselves.

    Why we care about the votes of individual MPs

    To provide better information through TheyWorkForYou, we need more complete individual voting data. The more formal mechanisms there are to manage absences, the more we can say what an MP’s vote in comparison to their party’s vote means, and produce better information for our users. Giving disclaimers and saying “it’s more complicated than that” is one approach, but it’s better to fix the problem at the source.

    Ultimately we think that MPs are responsible for how they vote and that giving more power to MPs helps them keep their promises to us. The growing amount of “rebellions” (where MPs vote against their parties) is assumed (at least in part) to result from TheyWorkForYou making individual voting records more salient. While MPs do not have strong individual mandates, they do have opinions about where leadership is drifting beyond what they said in elections. We think it is more effective for the responsibility of remembering promises to be distributed among a party, rather than seeing an election as endorsement of any and all future decisions of the party leadership.

    We’re not blindly pro-rebellions, but we think it’s important to create the space where they’re a real option available to MPs. We want better data, but we also want to give MPs more room to make decisions.

    Separating debates from votes

    Grouping votes is already the standard working pattern in the Scottish Parliament and Senedd. The Scottish Parliament groups votes into a “decision time” at the end of the day; the Senedd similarly has a “voting time”. Combined with electronic voting systems – these parliaments can handle a number of votes in the time it takes the Westminster commons time to do one vote. This was also a recommendation of Sarah Childs’ Good Parliament Report, and the Fawcett Society’s A House for Everyone.

    But is it right to split up debates and votes? The Parliament website says that parliamentary “[d]ebates are designed to assist MPs and Lords to reach an informed decision on a subject. This decision is then often expressed in a vote (called a ‘division’), for or against”. Don’t we want MPs to make decisions following the debate so their votes are informed by the arguments they’ve heard?

    The problem is this description is technically true, but fiddly in the details. While debates are an opportunity for individual MPs to talk about principle and specific issues with legislation, when we move to votes, party power asserts itself. MPs who were never part of the debate are summoned by the division bell, and the vote is decided by who holds power in Parliament, rather than strong arguments made in the moment.

    Debates and votes are connected indirectly. The best way of describing the process is that parliamentary debates are the visible portion of a wider set of conversations between groups of MPs as well as between government MPs and leadership. These conversations pre-determine the outcome of votes, by finding if there are any areas the government will accept and support amendments on, or to save face, will more quietly accept in the Lords rather than the Commons. It’s not that MPs’ concerns and opinions go nowhere, just that the method they can have impact is indirect. To describe this isn’t to say it’s good, or an ideal way of working, just that it is not important for how the Commons currently works for votes to directly follow the debate they relate to.

    The connection between debates and votes is part of a long debate about the role of MPs and technology. Jacob Rees-Mogg, in arguing against remote electronic voting, argued that if MPs didn’t “have the inconvenience of having to be here physically, they don’t necessarily take it as seriously”. This is a less substantial version of Thomas Wakley’s 1839 complaint about having division bells at all – where he found it “most mischievous, that hon. Members should come down there to vote upon a question without having heard one word of the discussion”.

    Ultimately, Wakley has lost this argument – but separating out debates and votes, and moving towards a decision time, at the least removes one source of party pressure, and helps build an effective working culture that also has good transparency to the rest of the country.

    This decision time should be held within some concept of “core hours” for Parliament – helping most MPs make the voting period within their other responsibilities, personal and professional.

    Being sensitive to the lives of MPs

    Sometimes MPs can’t make it to Parliament for reasons that are part of the normal human experience. People have children and people get sick. These are things we recognise as important to safeguard in employment law, but there are difficulties for MPs in applying this to MPs who are effectively self-employed, but with obligations to constituents.

    The way Parliament currently balances this is through proxy votes – where MPs can, in circumstances like maternity/paternity leave or prolonged sickness, designate another MP to cast a vote on their behalf. Similar mechanisms now operate in the Scottish Parliament and Senedd. In TheyWorkForYou, proxy votes are displayed the same as normal votes, with a note of who the proxy vote was cast by.

    The proxy vote system (introduced in 2019) was a result of two factors. The first is a long running trend where TheyWorkForYou (or similar analysis) highlighted individual voting records in a way that did not capture informal mechanisms of managing long absences like pairing – where parties mutually agree a list of absent MPs so the final result is not affected by absences. While in practice the absence is being accounted for, this is happening in a way that is not transparent to the public. In 2018, women MPs argued that TheyWorkForYou metrics were part of a standard where MPs had to work too soon after having children. This more broadly reflects the issue that Parliament and political life are structured in such a way that assumes the model MP is male, and TheyWorkForYou reinforced rather than challenged this.

    The second factor was that a breakdown of trust between parties on pairing led to heavily pregnant MPs needing to vote. Informal mechanisms are ultimately dependent on goodwill, something that cannot always be relied on.

    Proxy votes were a solution to this. They are formal mechanisms that are guaranteed by Parliament rather than parties. They reflect the reality that often, an MPs vote is the same as all other MPs of that party, and other MPs can easily cast it for them. By creating an explicit way MPs can be absent for sustained periods, the stream of individual data is created that preserves visibility on the impact of MPs, without requiring a daily presence in Westminster.

    This system is a good innovation, but it should go further. While circumstances that lead to prolonged absences are explicitly included (and others should be over time), people often need time off work to handle important issues in their lives. Currently, this remains managed through informal pairing approaches. Party managers can give MPs permission to be absent (“slips”), which will then be managed through party channels.

    The problem with this informal approach is another area where it gives parties arbitrary power in one aspect of an MP’s life, that can be used to encourage discipline more widely. In 2022, female Conservative MPs argued there was sexism in when slips were and weren’t allowed – with an example of a slip not being given to a mother needing to take a child to hospital, while feeling male colleagues were easily being given permission to go on holiday. In other cases, permission might be denied to punish internal critics. This is not a healthy system – it works for party management, but not for MPs, and doesn’t work transparently for the rest of the country.

    The solution to this is to manage “slips” through the same proxy vote mechanism. MPs should be given the equivalent of leave through a set number of times they can designate a proxy for the day, no questions asked. This accomplishes a double goal of removing more absences from the parliamentary record, and reducing another arbitrary way parties can hold power over MPs.

    Looking for answers that work for everyone

    We believe that transparency in how MPs vote is important. Through our voting records we want to present straightforward summaries of the impact of MPs in Parliament, that inform our users, and encourage focus on significant decisions made in Parliament. From our point of view, the more complete the data, the easier it is to create summaries that accurately reflect underlying realities without long disclaimers. As such, we’re in favour of both making voting easier and parties publishing the instructions they give MPs, to give the public more complete, and easier to understand, information about voting.

    At the same time, making voting easier improves the quality of life of MPs. Making voting more predictable, and reasonable absences more possible, fit well as part of a package of changes improving parliamentary life.

    While TheyWorkForYou will sometimes make life harder for MPs (and that’s partly the point), in other cases, our frustration at lack of transparency, and MPs’ frustrations at arbitrary and bad working conditions come from the same place: an agreement between party leaderships that power should be centralised. But there are ways of changing that, and we should talk more about it.





    If you'd like to see us extending our work in democracy further, please consider making a contribution.
    Donate now

    Image: Johann Siemens on Unsplash.

  10. Machine learning can make climate action plans more explorable

    We’ve used machine learning to make practical improvements in the search on CAPE – our local government climate information portal.

    The site contains hundreds of documents and climate action plans from different councils, and they’re all searchable.

    One aim of this project is to make it easier for everyone to find the climate information they need: so councils, for example, can learn from each other’s work; and people can easily pull together a picture on what is planned across the country.

    The problem is that these documents often use different terms to talk about the same basic ideas – meaning that using the search function requires an expert understanding of which different keywords to search for in combination.

    Using machine learning, we’ve now made it so the search will automatically include related terms. We’ve also improved the accessibility of individual documents by highlighting which key concepts are discussed in the document.

    Screenshot of CAPE: Search results for "flooding" , including 41 related terms such as drainage, river, and flooding.

    Screenshot of CAPE action plan listing. Shows a climate emergency action plan, and some of 65 topics extracted from the document: e.g. air quality, biodiversity, carbon budgets

    How machine learning helps

    We’re already using machine learning techniques as part of our work clustering similar councils based on emissions profile, but we hadn’t previously looked at how machine learning approaches could be applied to big databases of text like CAPE.

    As part of our funding from Quadrature Climate Foundation, we were supported to take part in the Faculty Fellowship – where people transitioning from academic to industrial data science jobs are partnered with organisations looking to explore how machine learning can benefit their work.

    Louis Davidson joined us for six weeks as part of this programme. After a bit of exploration of the data, we decided on a project looking at this problem of improving the search, as there was a clear way a machine learning solution could be applied: using a language model to identify key concepts that were present across all the documents. You can watch Louis’ end of project presentation on YouTube.

    Moving from similar words to similar concepts

    Louis took the documents we had and used a language model (in this case, BERT) to produce ‘embeddings’ for all the phrases they contained.

    When language models are trained on large amounts of text, this changes the internal shape of the model so that text with similar meanings ends up being ‘closer’ to each other inside the model. An ‘embedding’ is a series of numbers that represent this location. By looking at the distance between embeddings, we can identify groups of similar terms with similar meanings. While a more basic text similarity approach would say that ‘bat’ and ‘bag’ are very similar, a model that sorts based on meaning would identify that ‘bat’ and ‘owl’ are more similar.

    This means that without needing to re-train the model (because you’re not really concerned with what the model was originally trained to do), you can explore the similarities between concepts.

    There are approaches to this that store a “vector database” of these embeddings which can be directly searched – but we’ve gone for a simpler approach that doesn’t require a big change to how CAPE was already working.

    Using the documents we have, we automatically identified (and manually selected a group of) common concepts that are found across a range of documents – and the original groups of words that relate to those concepts.

    When a search is made we now consult this list of similar phrases, and search for these at the same time. This gives us a practical way of improving our existing processes without adding new technical requirements when adding new documents or searching the database.

    Because we now have this list of common concepts, we are also pre-searching for these concepts to provide, for each document, links to where that concept is discussed within it. With this change, the contents of individual documents are more visible, with it easier to quickly identify interesting contents depending on what you are interested in.

    Potential of machine learning for mySociety

    Our other websites, like TheyWorkForYou and WhatDoTheyKnow, similarly have a large amount of text that this kind of semantic search can make more accessible — and we can already see how they might be useful to those relying on data around climate and the environment WhatDoTheyKnow in particular has huge amounts of environmental information fragmented across replies to hundreds of different authorities.

    Generative AI and machine learning have huge potential to help us make the information we hold more accessible. At the same time, we need to understand how to incorporate new techniques into our services in a way that is sustainable over time.

    Through experiments like this with CAPE, we are learning how to think about machine learning, which problems we have that it applies to, and understand new skills we need to work with it. Thanks to Louis, and his Faculty advisors for his work and their support on this project.

    Image: Ravaly on Unsplash.