1. What do we know about MPs’ financial interests?

    In our new WhoFundsThem project we are making summaries of MPs registers of financial interests to add to TheyWorkForYou. We want to take these existing disclosures and add context to make them easier to understand. To do this, we are taking a hard look at how all the existing disclosure processes work (and when they don’t) to understand how we might best apply pressure for improvements. 

    One of our motivations here is that we think the rules about what MPs can and can’t do should be led by public expectations. To reflect that in our work, we’ve put together a literature review of the current picture of evidence around how MPs’ financial interests operate, and how these are perceived.

    We’ve published this review online, but here are some quick thoughts I’ve taken away from this. 

    There’s been a big shift in the role of MPs from 40 years ago – in practice and in public perception, being an MP is a full time job. There is some nuance here in public perception: while some professions are more approved in general (doctors/nurses), generally as the pay or involvement goes up, the work is considered less favourably. 

    There’s too much focus on the problem of second jobs being a distraction for the MP, and not enough on the problem of privileged access to Parliament for those who can pay for it. We should be asking questions about when MPs are selling their access rather than expertise. This encourages paying more attention to written questions – where MPs have a privileged ability to get answers to questions (and there’s indirect evidence this has been happening as part of some MPs’ second employment). 

    We need to care where donations come from, rather than being too focused on what they were spent on.  A general throughline in the discourse is catching when people are benefiting privately from their position (e.g. receiving gifts) – but there’s also the situation that private donors are supporting the public work of politicians (for instance, funding researchers in their offices). With a “follow the money” hat on, this should be seen as an investment in relationships with politicians that might pay off later rather than being purely public spirited. 

    We need to be aware that transparency in this area has been a hedge against more substantial reform (e.g. disclose bad things rather than stop doing bad things). This compromise position has usefulness for both sides. For those who want stricter rules, it encourages politicians to have one eye on public opinion through disclosure requirements, and generates a regular series of news stories helpful in future reform. 

    But for those opposed to stricter rules, transparency can be framed as approval – where the electorate is argued to have endorsed MPs’ choices. Conversations become about if the rules were followed rather than the underlying issues, and when the regime is only half-heartedly supported, non-disclosure can be common (meaning that scrutiny falls more on those correctly disclosing rather than those who do not). 

    In general, we see increasing the transparency and getting the most out of the information that is available as the tool we have been given to improve the situation. But we shouldn’t lose sight that transparency is a means, not an end in itself. 

    From this work, we’ve created a set of questions that make sure we draw out important aspects of the register. Next week our volunteers will start to answer these questions. 

    These questions cover all sections of the register. We’re asking volunteers to help us understand which industries are showing up in MPs’ registers, and whether they are declaring an interest in debates and questions when they’re supposed to be.  We’ll compare the Register of Interests against Companies’ House with support from new data from Any One Thing, and we’ll get volunteers to give MPs’ registered interests and overall transparency score. The process will also include a right of reply, so MP’s can respond to the summaries we write. 

    We do this work because we think it is possible to make politics better from the outside. Through combining the effort of volunteers with the lever of technology, we can make a real difference in how things work. 

    If you’d like to support this project – please donate today.

  2. More than 90% of APPGs have disappeared… but we think it’s probably just an admin delay

    The first register of All Party Parliamentary Groups since the general election has just been published, and 519 of the 553 groups have vanished, leaving just 34. 

    What is an APPG?

    All Party Parliamentary Groups (APPGs) are self-selecting groups of MPs and Lords with an interest in a particular policy area. Most groups are supported by a secretariat, which is usually a charity, membership body or consultancy organisation.

    The logic behind APPGs is to create legitimate avenues for experts and interested parties from outside Parliament to discuss policy with MPs and Lords – but unfortunately they can also be vehicles for corruption. 

    Our WhoFundsThem project is going to be taking a closer look at APPGs, to see which MPs are members (this information is currently not published) and a closer look at the organisations providing secretariat support. We have also updated our public APPGs spreadsheet with the new register. 

    So why have so many groups disappeared?

    A change in rules last year meant that we saw a huge drop-off from the 800+ groups registered in March to around 450 in April, and then a steady increase to 553 by the end of May. The 28th August edition has just 34 registered groups.

    Since the general election, we think are there are three factors that might be influencing the dramatic decline in registered groups:

    1. New officer rule – there’s a new rule that MPs are now only allowed to be an officer of a maximum of six groups.
    2. The reduced size of the opposition – the ‘all party’ nature of APPGs means that they must have at least one member of the official opposition as an officer. Before Parliament was dissolved for the election in May, the then Labour opposition had 206 MPs. Now, the Conservative opposition has 121 MPs. Conservative Lords are allowed to be officers of APPGs, but the APPG Chair must be an MP.
    3. Summer recess admin delay – in order to meet the deadline for this register, groups had to hold their new AGM to elect officers before summer recess began on 30 July. This gave them just a couple of weeks after the election, which was a hectic time, especially for the majority of MPs who were new to Parliament, and busy setting up their offices.

    What next?

    Given that we’ve just had one register, we can’t be sure which of these factors is having the biggest effect, but a second edition of the register should help us to understand the scale of the admin delay problem. 

    We expect a large number of groups will have used the summer to get established and recruit officers and members – but they will need to hold an AGM fairly soon after Parliament returns next week in order to make the new register, which should be published in about six weeks’ time.

    We’ll be looking in detail at the work of these groups, and the people behind them, in our project WhoFundsThem. Please consider donating to help us do more of this work. 

    Photo by Erik Mclean on Unsplash

  3. New register of interests spreadsheet – with much richer data

    Tl;dr: Parliament has released new data, which we’ve made available in a simple format

    As part of the new release of the register of financial interests (which we blogged about yesterday)  – Parliament has released CSVs of the new edition of the register. This isn’t just a better way of getting the data from each page individually, but contains much richer information than we’ve had previously. 

    Earlier this year, Parliament improved its data collection for MPs’ interests – meaning it collects much more structured data for different kinds of interests than the free text data that was released previously. 

    This is really good news – the work put in improving the data collection is so hard to do from the outside. Lots of effort has been made to clean up data in the past, but it was just fundamentally too broken. This is a big improvement on that – and means we can focus our efforts on where we can add the most value. 

    We know that Parliament is looking at creating data tools to sit on top of this – but in the meantime we’ve quickly made a single Excel file – and an analysis site to explore the data.  We’ve also added our IDs from TheyWorkForYou and information on the MPs party. The great thing about Parliament making more data available is how that data can then be expanded by other datasets – for instance, the data now contains Companies House IDs, which could be joined to a range of datasets. 

    Please email if there are tweaks that would make the spreadsheet more useful to you!

    Some example queries that are possible with this (give the site a minute to load):

    Whenever Parliament ups its game, we need to think about what we’re going to do to build on top of that. As part of our WhoFundsThem project, we’re working to create simple summaries of declarations of interests. In general, the register is full of data but lacking in context. What do these organisations who have donated do? What’s the top-line figure on outside income? Is this affecting how MPs behave in parliament?

    These are the questions we want to answer through WhoFundsThem. If you also want to know the answer, you can donate to support our work.

     

  4. TICTeC 2024: Call for proposals and registration now open

    TICTeC, our Impacts of Civic Technology Conference, will be returning for its 7th edition on 12th and 13th June 2024, in London and online. We’re delighted to announce that the call for proposals and registration for TICTeC 2024 are now open.

    TICTeC is all about sharing research, knowledge and experiences to examine and improve the impacts of civic technology, in order to strengthen democracy, public participation, transparency, and accountability across the world.

    Call for Proposals: open until 22nd March 2024

    Core themes

    After twenty years of mySociety, and approaching ten years of TICTeC – we want to think about what is needed now to match the big challenges of the next twenty years. 

    As well as examining the impact that civic technology is having upon societies around the world, the big question we want to answer through TICTeC is:

    What is needed to make civic tech on a global scale more successful and impactful, to tackle global problems around democracy and climate change?

    Through TICTeC 2024 and 2025, and our new Communities of Practice – we are going to break down this question, and work through for ourselves, and with our partners, what is needed to deliver on the radical goals of the civic technology movement. 

    This breaks down into two sub questions that we want to explore. What is the role of civic tech in:

    • safeguarding and advancing democracy/transparency where it is under threat?
    • enabling the effective and democratic change needed to meet the challenge of climate change?

    For this year’s TICTeC we encourage proposals that contribute to discussion around these two thematic questions, as well as to the overarching conference theme. Potential topic areas may include:

    • Access to Information/Freedom of Information
    • Monitoring parliaments/legislatures
    • Climate change/climate action
    • Tools for citizen participation
    • AI and Democracy
    • Civic tech as part of civil society
    • Crowdsourcing and volunteers
    • Impacts of big tech/tech giants
    • Fact checking
    • Technical infrastructure/cybersecurity

    You can propose 20 minute presentations and ideas for longer workshops.

    We encourage presentation submissions to focus on the specific impacts of technologies, rather than showcase new tools that are as yet untested. A tool doesn’t have to have mass usage to be worth talking about – we’re also interested in qualitative stories on the impacts of technology, their impacts on official processes, and how users have used platforms to campaign for change. We’re also interested in stories about obstacles and barriers to having impact. 

    Workshop proposals should be relevant to the conference themes. Technology does not have to be new, and we welcome retrospectives on long running projects. 

    The deadline for applications is the 22nd March 2024. Those selected for inclusion in the conference programme will be notified no later than 5th April 2024.

    Presenters will be required to register for the conference by 19th April in order to confirm their slot (the registration fee will be waived for individuals presenting). 

    Submit your proposals via this application form by 22nd March 2024 at the latest. 

    Register now

    Registration for TICTeC 2024 is now open and is essential in order to attend. TICTeC has sold out in previous years – so make sure you get tickets early. Early bird tickets provide a significant discount, so it’s well worth registering before early bird ticket sales end on 20th April 2024.

    Attending TICTeC 2024 in-person will allow attendees access to all conference sessions, including main plenary sessions, presentation/Q&A sessions, workshops, networking sessions, lunches and drinks reception. Attending online will allow remote attendees access to all main plenary sessions and some breakout presentation/Q&A sessions.

    The TICTeC 2024 Eventbrite page contains further information about the conference, as well as FAQs, but do let us know if you have any questions by emailing tictec@mysociety.org.


    In the following months, we will be publishing full details of proceedings as they are announced over on the TICTeC website. If you’d like to hear of TICTeC 2024 updates first, please sign up for email updates.

    And in the meantime, if you’d like to see what TICTeC is all about, you can browse all the resources from previous TICTeC events over on the TICTeC Knowledge Hub.

    We look forward to welcoming you to TICTeC 2024!

     

  5. Event recap: How can we make local climate data more useful for everyone?

    At the end of November, we were delighted to be joined by over 80 people at our webinar about making local climate data more useful. The recording is now available on YouTube, but we also wanted to capture the key messages from our speakers.

    Anna Powell-Smith, from the Centre for Public Data, highlighted the key recommendations from the Unlocking Fragmented Data report, published jointly with mySociety earlier this year. These are:

    1. A collaborative (but required) data standard to agree the data and format that is expected. 
    2. An online central repository of the location of the published data, so that data users can find it easily.
    3. Support from the data convener to make publication simple and effective.

    Alex Parsons, mySociety’s Senior Researcher, gave the example of trying to build a comprehensive database of council home EPC standards. This data is already published by all local authorities, but because it is published in a variety of formats and locations, it can’t be easily joined up. This data was compiled by volunteers through FOI requests (in order to get standard formats) for the 2023 Council Climate Action Scorecards, and the results were covered in the Financial Times. It was not ‘new’ data, it was just the first time it had been collated and compared.

    Eoin Devane from the Climate Change Committee stressed that data is essential for their work, and that their recent reports highlight the many data gaps that still exist in assessing the UK’s progress towards our 2050 net zero target. Contextualising the need for this data, Eoin also pointed to the CCC’s calls for more clarity on the role of local government, and on bodies like the Local Net Zero Forum.

    Julia Cushion. This then led onto my section, highlighting the types of climate data we need, which we have covered in a previous blog post. I also spoke about the supporting factors for these:

    • Echoing Eoin, more clarity from on the powers of local government for net zero delivery. This is also a key ask of the Blueprint Coalition
    • More transparency around the Local Net Zero Forum and how this acts as a connection between national and local government 
    • Greater coherence around the role of Oflog, especially how they prioritise their metrics
    • More involvement from the Central Digital and Data Office, who could play an important convening role 

    Next, we had our first councillor – Joe Porter, District Councillor for Brown Edge and Endon – who emphasised the importance of local councils as key players in climate action. Reflecting on Staffordshire Moorlands’ efforts, he discussed their annual Climate Change Report, emphasising the significance of monitoring progress, engaging with communities, and setting ambitious targets for carbon neutrality and nature restoration.

    Minesh Parekh, a Labour and Cooperative councillor from Sheffield, echoed the sentiments on the imperative need for councils to lead in addressing the climate crisis. He emphasised the criticality of data in guiding decision-making at the local level. Minesh pointed out the disparity in information available to local councils compared to Members of Parliament, stressing the need for more localised data and resources to support informed decision-making on climate initiatives.

    We rounded off the hour with a quick Q&A, which brought out the importance of sharing best practices, expertise, and data among councils through platforms like the Environmental Data Network. The councillors highlighted the significance of collaboration and the exchange of information to address challenges, bridge data gaps, and achieve more substantial climate action goals.

    Thanks to those who joined us, and we hope to see you at a future event soon. To stay updated on our climate programme, you can sign up to our newsletter.

    Photo by Benjamin Elliott on Unsplash

  6. Providing tech for effective crowdsourcing

    An important idea in mySociety’s history and work is that combining people-power and technology is a powerful method to make change.

    Recently we’ve had the chance to work with Climate Emergency UK on the council climate scorecards. As part of this we’ve been learning from how CE UK works with volunteers, and thinking how best we can be a technical partner to crowdsourcing projects.

    But we’ve also been thinking how we can better incorporate crowdsourcing into our core services. This has led to improving our approach to crowdsourcing within WhatDoTheyKnow, and developing a new platform (GRACE) to help volunteers crowdsource information for the scorecards.

    Developing our technical approach

    In the first round of the Council Climate Scorecards,back in 2021, the process was managed on a big Google Sheet. Each answer to a question has a line, which tracks the answer from a dropdown and keeps track of notes, and we use macros to populate and move information around. This worked and let us get started at the pace we wanted, but introduced a lot of room for error, and meant more work was needed at the end of the process to validate the data.

    For the second round of scorecards, with a more complicated set of questions, we wanted to move away from the spreadsheet approach, into something that could validate the data as we went, and integrated the right of reply process into the same platform.

    Working with CE UK, we built a crowdsourcing platform GRACE (named after CE UK’s first volunteer) to replace the old spreadsheet approach.

    The overall process worked like this:

    • Working with their advisory group, CE UK created questions across a range of different areas, with different sections for different kinds of councils.
    • These questions and data from FOI responses were uploaded into the platform.
    • Volunteers were given access to the platform, and could answer questions in the topic areas assigned to them. This would include an answer from a dropdown, a public “evidence” field, and a private notes field to share between volunteers.
    • Overall progress across all areas could be seen by the volunteer coordinators.
    • Local authorities were given a right of reply per question. Emails were sent to all local authorities, giving them access to the platform to review the scores and evidence for their section. This gave them the opportunity to submit new evidence where they felt the score did not reflect their actions. 74% submitted responses to the right of reply.
    • Responses from the right of reply were reconciled with the original answers by a group of volunteers and CE UK.
    • This data could then be reviewed and loaded into councilclimatescorecards.uk

    How did CE UK find the tool? Isaac Beevor, Co-director of CE UK:

    GRACE – our online data collection system was incredible. We would not have been able to collect the thousands of data points on local authorities’ climate action without it. It was simple to use and effective so all of our volunteers were able to use it, with minimal training. Furthermore, it allowed us to coordinate and track progress. This meant we were able to encourage volunteers who needed it and track progress in sections allowing us to plan ahead for staff and volunteer capacity. We then used it to allow access for councils in the Right of Reply and all of the feedback from officers, particularly its ease of use, was positive. The best thing is we can now use this for every Scorecards, which saves us so much time and energy.

    Effective crowdsourcing

    As we wrote in our report about public fragmented data the issue with the idea of ‘armchair auditors’ is not that they do not exist, but that they were thought about in the wrong way. People can and do use their time to support civic accountability through looking at spreadsheets. But they need to be given support and structure to work effectively together.

    Reflecting on the CE UK process and other crowdsourcing approaches we admire, like Research for Action and Democracy Club, what these projects have in common is that they involve knowledge sharing and collaboration across the country, with volunteers themselves contributing a local or specialist focus. “Armchair auditors” aren’t atomised individuals, but work together as a community.

    The Effective Crowdsourcing diamond (below) describes the aspects we think are key to successful projects, and helps us shape our thinking about future crowdsourcing/citizen science projects.

    A blue diamond with the following text "Effective crowdsouricng: working together to answer big questions". The four points of the diamond have: Volunteers: A pool of motivated supporters break the problem up Expertise: Experts and advisors to frame the questions to ask Technology: specialist tools streamline information gathering and quality assurance steps Impact: clear paths form gathering data to making change.

    In this framework, an effective process will have four features that all interrelate and reinforce each other:

    • Expertise – Helping move from high level principles to concrete questions and approaches that inform how work can be split up for volunteers, and the technical tools needed.
    • Volunteers – People who care about an issue, with the skills to contribute to answering questions.
    • Technology – Technology makes it easier for people to work together, and validation and cross checking improves the accuracy of the process, enabling more complex approaches.
    • Impact – What is the path from the result of a project to change the world? Clear routes to impact means clearer benefits of engaging to experts and volunteers.

    For Scorecards and other projects, we have provided the technical side of the diamond. But there is lots of potential for us to run crowdsourcing projects that improve our core Democracy and Transparency services. Starting with the tools we now have, we’re thinking about how we can strengthen our skills and approaches across the diamond – and how we might partner on projects to bring in other expertise and skills.

    GRACE screenshots

    Initial volunteer screen showing sections assigned:

    Screenshot of a page showing the assignments and completion rates assigned to a specific indivudal. A series of progress bars next to sections.

    Screen showing councils and progress

    Screenshot of a pages showing the overall progress for a section. It shows by council the number of questiosn completed for this group of questions

    Marking screen

    This shows the marking screen - which is a form capture the answer to a question, with boxes and links for evidence

    Right of Reply screen

    This is the right of reply screen that will be visible by councils - it shows the answers to the question in uneditable boxes -while providing options to agree/disagree with response and submit further evidence

    Admin screen so you can check that how much has been assigned to volunteers

    Volunteer management screen - showing how many sections are currently assigned to volutneerws

    Admin screen to show progress on an authority level

    Progress screen, showing the number of questions completed by council, and the overall progress

    Or on a section level

    Progress screen by question section, showing the number of councils with a complete set of questions for that section.
  7. mySociety recommendations in new ‘The Future Is Local’ report

    Over the summer, we were invited to be a part of the Local Mission Zero Network consultation, and we’re thrilled that our key fragmented data policy recommendations have been included in the new report, as well as recognition for some of our wider work on climate. 

    Rt Hon Chris Skidmore OBE MP, former Net Zero Review Chair and one of the co-authors of the report, said:

    “The Local Mission Zero Network’s first report, The Future Is Local, sets out over thirty recommendations to further the Net Zero Review’s local delivery mission. It’s clear that if central government won’t step up, it should get out of the way and allow local and regional leaders to forge ahead with their positive vision to achieve local Net Zero in partnership with communities up and down the country. Unleashing their ambition is the most effective way to harness the economic and regional growth opportunities that Net Zero can unlock.

    I’d like to thank MySociety for their involvement in the network and also for their input in making key recommendations on the need for better data and information to achieve Net zero.”

    The report, co-authored with Lord Ben Houchen, released today, is “intended not only to highlight the continued challenges facing the local delivery of net zero, it also seeks to frame these challenges into a new framework for ensuring local authorities and regions have the certainty to achieve their net zero ambitions”. It is a much needed intervention, and makes clear that “in the current policy environment, and ahead of the next General Election, greater certainty over local net zero is essential”.

    Within Recommendation 1, Introduce a Local Net Zero Charter to agree responsibilities and enhance partnership between the UK government, devolved governments and regional, city and local authorities, there are three specific recommendations relating to our fragmented data work:

    1f) A Local Net Zero Data and Reporting Framework should be established, in order to provide consistency and increase integrity for reporting across local authorities.

    1g) The Net Zero Review recommended that ONS should collect more forms of net zero related data, and this network maintains that net zero will be better delivered the more we know, and where we know action needs to take place.

    1h) The need for open source and operable data is also important, if we are to encourage better uses of AI and future systems thinking. This data to be held in a central repository, supported by a central government data convenor.

    In the Unlocking the value of fragmented public data report we published last year, we stress the importance of local climate data being published in a way that is useful, ultimately creating positive feedback loops across the economy. It’s great to see the report emphasise this:

    “The challenge of fragmented and inoperable data standards is not merely a matter for more effective local authority performance. The future of energy system planning could be better forecast if several datasets were better aligned.” 

    The body of the report also highlights our conclusions about the kinds of climate data we need:

    more about how  local authorities reflect on their own progress. In these instances, free text which we can semantically search, is often most helpful. We need data around:

    • Personnel, systems & processes to manage climate monitoring and reporting. This helps us to understand who is doing the work, and how resource allocation happens. 
    • Progress since the last reporting period, and key areas of focus for the period ahead. This gives a vital sense of context and perspective from inside the reporting body, and helps situate the scale of work undertaken against work yet to be done.

    Finally, our CAPE project was mentioned as “effective monitor[ing]”, and we were so pleased to see the work we do with Climate Emergency UK to create the Climate Scorecards recognised: “By simplifying complex data, it allowed stakeholders to identify gaps and progress in climate initiatives, empowering communities to advocate for change”.

    If you’d like to read the report in full, you can find it here. You may even want to share some of the recommendations from the report with your MP, which you could do using our service WriteToThem

    Any questions for our policy team? Get in touch: policy@mysociety.org 

    Image: Minku Kang on Unsplash

  8. Empowering local change together: mySociety joins the Blueprint Coalition

    We’re delighted to announce that mySociety has joined the Blueprint Coalition – an influential group of local government organisations, environmental groups, and research institutions working together to deliver local climate action with a joined-up approach. 

    The Coalition works across sectoral, geographical and party boundaries to make change happen. We’re excited to join the other members in calling upon the government to provide the crucial support local authorities need to deliver on tackling the climate crisis.

    About mySociety

    Becoming a part of the Blueprint Coalition isn’t just a milestone; it’s a commitment to a cause larger than ourselves. As mySociety joins hands with like-minded organisations, we are poised to make significant progress in our aim to make climate-related data more accessible. We believe that more information makes for better-informed action, so everything we do puts richer, more usable data into the open, where everyone can use it. 

    Our Climate, Transparency and Democracy streams consist of a number of services (such as CAPE, Climate Scorecards, TheyWorkForYou, WriteToThem, and WhatDoTheyKnow) which we bring to the Coalition alongside our research, policy and advocacy work. Our policy work has been focusing on the issue of fragmented data, and we’re excited to be planning a webinar on this topic with the Coalition – watch this space!

    About the Blueprint Coalition

    In December 2020, the Blueprint Coalition published a comprehensive manifesto that serves as a roadmap to expedite climate action and usher in a green recovery at the local level. It outlines the national leadership, policies, powers, and funding required to empower local authorities in making impactful changes on a substantial scale. Drawing on the first-hand experiences of local authorities that have declared climate emergencies, this blueprint serves as a guiding light for collective action towards a sustainable future.

    A defining feature of the Blueprint Coalition is its central ethos of fostering partnerships between civil society, national and local governments. Recognising that achieving net zero carbon emissions requires the collaboration of all levels of governance, the Coalition’s work serves as a testament to the power of collaboration.

    The Coalition partners include: 

    • Ashden
    • Association of Directors of Environment, Economy, Transport and Planning (ADEPT)
    • Centre for Alternative Technology
    • Climate Emergency UK
    • Friends of the Earth
    • Grantham Institute – Climate Change and the Environment (Imperial College London)
    • London Environment Directors’ Network (LEDNet)
    • Place-based Climate Action Network (PCAN) at LSE
    • Solace
    • in addition to support from London Councils and Green Alliance.

    If you’d like to show your support for the Coalition, you can sign up here. And to stay updated on our Climate programme, you can sign up to our newsletter.

    Any other questions or comments? Get in touch with Julia, our Policy & Advocacy Manager.

    Image: Public domain, via Wikimedia Commons. See page for author.

  9. Machine learning can make climate action plans more explorable

    We’ve used machine learning to make practical improvements in the search on CAPE – our local government climate information portal.

    The site contains hundreds of documents and climate action plans from different councils, and they’re all searchable.

    One aim of this project is to make it easier for everyone to find the climate information they need: so councils, for example, can learn from each other’s work; and people can easily pull together a picture on what is planned across the country.

    The problem is that these documents often use different terms to talk about the same basic ideas – meaning that using the search function requires an expert understanding of which different keywords to search for in combination.

    Using machine learning, we’ve now made it so the search will automatically include related terms. We’ve also improved the accessibility of individual documents by highlighting which key concepts are discussed in the document.

    Screenshot of CAPE: Search results for "flooding" , including 41 related terms such as drainage, river, and flooding. Screenshot of CAPE action plan listing. Shows a climate emergency action plan, and some of 65 topics extracted from the document: e.g. air quality, biodiversity, carbon budgets

    How machine learning helps

    We’re already using machine learning techniques as part of our work clustering similar councils based on emissions profile, but we hadn’t previously looked at how machine learning approaches could be applied to big databases of text like CAPE.

    As part of our funding from Quadrature Climate Foundation, we were supported to take part in the Faculty Fellowship – where people transitioning from academic to industrial data science jobs are partnered with organisations looking to explore how machine learning can benefit their work.

    Louis Davidson joined us for six weeks as part of this programme. After a bit of exploration of the data, we decided on a project looking at this problem of improving the search, as there was a clear way a machine learning solution could be applied: using a language model to identify key concepts that were present across all the documents. You can watch Louis’ end of project presentation on YouTube.

    Moving from similar words to similar concepts

    Louis took the documents we had and used a language model (in this case, BERT) to produce ‘embeddings’ for all the phrases they contained.

    When language models are trained on large amounts of text, this changes the internal shape of the model so that text with similar meanings ends up being ‘closer’ to each other inside the model. An ‘embedding’ is a series of numbers that represent this location. By looking at the distance between embeddings, we can identify groups of similar terms with similar meanings. While a more basic text similarity approach would say that ‘bat’ and ‘bag’ are very similar, a model that sorts based on meaning would identify that ‘bat’ and ‘owl’ are more similar.

    This means that without needing to re-train the model (because you’re not really concerned with what the model was originally trained to do), you can explore the similarities between concepts.

    There are approaches to this that store a “vector database” of these embeddings which can be directly searched – but we’ve gone for a simpler approach that doesn’t require a big change to how CAPE was already working.

    Using the documents we have, we automatically identified (and manually selected a group of) common concepts that are found across a range of documents – and the original groups of words that relate to those concepts.

    When a search is made we now consult this list of similar phrases, and search for these at the same time. This gives us a practical way of improving our existing processes without adding new technical requirements when adding new documents or searching the database.

    Because we now have this list of common concepts, we are also pre-searching for these concepts to provide, for each document, links to where that concept is discussed within it. With this change, the contents of individual documents are more visible, with it easier to quickly identify interesting contents depending on what you are interested in.

    Potential of machine learning for mySociety

    Our other websites, like TheyWorkForYou and WhatDoTheyKnow, similarly have a large amount of text that this kind of semantic search can make more accessible — and we can already see how they might be useful to those relying on data around climate and the environment WhatDoTheyKnow in particular has huge amounts of environmental information fragmented across replies to hundreds of different authorities.

    Generative AI and machine learning have huge potential to help us make the information we hold more accessible. At the same time, we need to understand how to incorporate new techniques into our services in a way that is sustainable over time.

    Through experiments like this with CAPE, we are learning how to think about machine learning, which problems we have that it applies to, and understand new skills we need to work with it. Thanks to Louis, and his Faculty advisors for his work and their support on this project.



    Image: Ravaly

  10. Learning from the way people use TheyWorkForYou

    This blog post is part of our Repowering Democracy series. This year we will be publishing a series of short pieces of writing from our staff, and external contributors who are thinking about how our democracy works and are at the frontlines of trying to improve it. Learn more about this series.


    We’re thinking about the future of TheyWorkForYou, and we want to ground our plans in an understanding of how it is currently used and the impact it has had.

    At a very practical level, it is much easier to make small changes than big ones — but small changes don’t have to have small effects. By leaning into how people are using the site, we can find ways of better supporting what people are already trying to do.

    TheyWorkForYou has been politically (and culturally) influential. The way in which MPs and parties conduct themselves has changed in reaction to the service: next week we’ll have a guest post covering this in more detail. A throwaway shot in 2018’s TV thriller The Bodyguard, where a character quickly scans a politician’s voting record, shows how the idea of TheyWorkForYou has become a part of the UK’s political shorthand (with many arguments about whether this is a good or a bad thing, that we’ll come back to in this series).

    That said, until recently we had no solid data as to how widely known the service is. In late 2021, Opinium gave us a set of free questions for a nationally representative poll. We used one of these to understand more about people’s awareness and usage of mySociety websites. We found that, out of all mySociety’s services, TheyWorkForYou was the one most people knew about. The poll found that one in three UK adults have heard of the site, and one in five have visited the site.

    One of the biggest obstacles to successful civic tech isn’t having a good idea for a digital service, but successfully getting more than a handful of people to use it. TheyWorkForYou, with 20 years of history behind it, has crossed that hurdle. Improving and refining TheyWorkForYou is potentially a much more impactful thing to do than launching new services, but the funding environment for civic tech means there is far more money available for new projects than for steady payoffs from established work. This is a key issue we need to navigate, balancing short-term survival with a commitment to doing the things that will have the biggest positive impact.

    Never heard of TheyWorkForYou 69%, have heard of it 32%, have visited/used 20%

     



    Parliamentary monitoring: a slow-burn success story

    A key unique feature of TheyWorkForYou is the email alerts service. We send daily emails to subscribers about the activity of their chosen MPs or Lords, or when phrases that they are interested in are used in debates or written questions/answers. On average, this means we send around 400,000 emails a month. People mostly use alerts to keep up with their own MP’s parliamentary activity, but the keyword search also means that this service  is a powerful free parliamentary monitoring tool. Alerts are used by a range of public, private and charitable sector organisations to track specific issues and keywords in Parliament.

    In 2021, we ran a survey on users of alerts and found that while the majority (84%) were citizen users, a sizable proportion (16%) were using it in a professional context. Focusing on this group for a follow-up survey in 2022, we got more details of the value that charities and campaigners get from TheyWorkForYou, but we also found that the alerts are in use by people working in Parliament and government departments, improving the flow of information inside these central institutions.

    These professional users were also interested in different elements of the site than citizen users, being slightly more focused on written answers and statements (as these can be the best statements of current policy), and having much less interest in voting records.   A 2016 GovLab report estimated an economic benefit of TheyWorkForYou, on time saved alone, of up to £70 million a  year to the third sector.  As a free service, it provides an important alternative to political intelligence organisations and helps level the playing field for civil society to engage with parliamentarians and decision-makers.

    There is real potential here to build on something that’s going well. We can better reflect, in the way the site works, and in the work we do around it, that a key way we have impact is via intermediaries, and making Parliament far more accessible to a range of charities and organisations. The technical side works well, but we could help organisations make the most use of this feature.

    Professional users are mostly in public or charitable sector jobs

     

    VOtes are seen as more useful by citizens than professionals


    During elections, people want different things from TheyWorkForYou

    For the last 10 years, TheyWorkForYou has generally had over six million page views each year. In 2021 there were 7.8 million page views, and there were 13 million in the last election year.  In each of the last five years, there have been a million views of either the summary or voting record pages of MPs. Information on these pages also travels far further than to these direct users, as it is amplified by journalists or social media.

    For the 2019 election, there was a clear increase in both overall traffic, but also in the proportion of people looking at the voting records pages. There were many views of the profiles of a small number of MPs (mostly party leaders) rather than views being evenly distributed among people looking at local MPs.

    TheyWorkForYou Page views by year, showing a spike in 2019 for the election Shows that generally there were more views of the summayr apge, until 2010 - when voting records almost reached parity

    This shift towards more and more views of just the voting records pages reflected a change in the way people arrived at the site. The pattern of people entering a postcode at theyworkforyou.com, arriving at a summary page (with party comparisons), before maybe moving onto more detailed individual voting policies was becoming less common. More users were coming to the site via search engines or social media, and missing parts of the information we presented. As a result, we changed the way we displayed information, moving more of the important context from the party comparison to the voting records page.

    But when the reason people are using the site changes, it’s a good time to consider how the information can best be presented. TheyWorkForYou in its design, is very focused on what happens in Parliament between elections, but the information it holds is obviously very relevant during an election. Steering into that, we could follow hints about what people want to know about (party leaders, and more widely, parties) and create new views on the information we hold, that reflect actions taken by a party over a parliamentary term. Here an existing usage suggests a different approach that could be useful to voters, that we are uniquely well placed to deliver, but which would be a substantial change in how we think about and present information.



    Purposeful incremental change

    TheyWorkForYou needs more than just code and servers to keep on serving the people of the UK well. Playing the biggest role we can in informing people over the next twenty years will require careful stewardship to support what works, while adapting to new problems and opportunities.

    As a long-running service, the site has picked up features that are sometimes useful, but sometimes outlive their purpose or the resources that are available to maintain them. Sometimes we have turned parts off. The core challenge of project-based funding is that it can only indirectly support “doing what’s already working”, and each additional project and feature adds long term maintenance costs. This is why it’s important that, while looking for opportunities to make improvements to the site, we need to make sure that our plans still fit into a coherent idea of what TheyWorkForYou is for – so all the parts of the site are still working together in a way that makes sense in the long run.



    If you'd like to see us extending our work in democracy further, please consider making a contribution.
    Donate now

    Image: Maksim Shutov