An important idea in mySociety’s history and work is that combining people-power and technology is a powerful method to make change.

Recently we’ve had the chance to work with Climate Emergency UK on the council climate scorecards. As part of this we’ve been learning from how CE UK works with volunteers, and thinking how best we can be a technical partner to crowdsourcing projects.

But we’ve also been thinking how we can better incorporate crowdsourcing into our core services. This has led to improving our approach to crowdsourcing within WhatDoTheyKnow, and developing a new platform (GRACE) to help volunteers crowdsource information for the scorecards.

Developing our technical approach

In the first round of the Council Climate Scorecards,back in 2021, the process was managed on a big Google Sheet. Each answer to a question has a line, which tracks the answer from a dropdown and keeps track of notes, and we use macros to populate and move information around. This worked and let us get started at the pace we wanted, but introduced a lot of room for error, and meant more work was needed at the end of the process to validate the data.

For the second round of scorecards, with a more complicated set of questions, we wanted to move away from the spreadsheet approach, into something that could validate the data as we went, and integrated the right of reply process into the same platform.

Working with CE UK, we built a crowdsourcing platform GRACE (named after CE UK’s first volunteer) to replace the old spreadsheet approach.

The overall process worked like this:

• Working with their advisory group, CE UK created questions across a range of different areas, with different sections for different kinds of councils.
• These questions and data from FOI responses were uploaded into the platform.
• Volunteers were given access to the platform, and could answer questions in the topic areas assigned to them. This would include an answer from a dropdown, a public “evidence” field, and a private notes field to share between volunteers.
• Overall progress across all areas could be seen by the volunteer coordinators.
• Local authorities were given a right of reply per question. Emails were sent to all local authorities, giving them access to the platform to review the scores and evidence for their section. This gave them the opportunity to submit new evidence where they felt the score did not reflect their actions. 74% submitted responses to the right of reply.
• Responses from the right of reply were reconciled with the original answers by a group of volunteers and CE UK.
• This data could then be reviewed and loaded into councilclimatescorecards.uk

How did CE UK find the tool? Isaac Beevor, Co-director of CE UK:

GRACE – our online data collection system was incredible. We would not have been able to collect the thousands of data points on local authorities’ climate action without it. It was simple to use and effective so all of our volunteers were able to use it, with minimal training. Furthermore, it allowed us to coordinate and track progress. This meant we were able to encourage volunteers who needed it and track progress in sections allowing us to plan ahead for staff and volunteer capacity. We then used it to allow access for councils in the Right of Reply and all of the feedback from officers, particularly its ease of use, was positive. The best thing is we can now use this for every Scorecards, which saves us so much time and energy.

Effective crowdsourcing

As we wrote in our report about public fragmented data the issue with the idea of ‘armchair auditors’ is not that they do not exist, but that they were thought about in the wrong way. People can and do use their time to support civic accountability through looking at spreadsheets. But they need to be given support and structure to work effectively together.

Reflecting on the CE UK process and other crowdsourcing approaches we admire, like Research for Action and Democracy Club, what these projects have in common is that they involve knowledge sharing and collaboration across the country, with volunteers themselves contributing a local or specialist focus. “Armchair auditors” aren’t atomised individuals, but work together as a community.

The Effective Crowdsourcing diamond (below) describes the aspects we think are key to successful projects, and helps us shape our thinking about future crowdsourcing/citizen science projects.

In this framework, an effective process will have four features that all interrelate and reinforce each other:

• Expertise – Helping move from high level principles to concrete questions and approaches that inform how work can be split up for volunteers, and the technical tools needed.
• Volunteers – People who care about an issue, with the skills to contribute to answering questions.
• Technology – Technology makes it easier for people to work together, and validation and cross checking improves the accuracy of the process, enabling more complex approaches.
• Impact – What is the path from the result of a project to change the world? Clear routes to impact means clearer benefits of engaging to experts and volunteers.

For Scorecards and other projects, we have provided the technical side of the diamond. But there is lots of potential for us to run crowdsourcing projects that improve our core Democracy and Transparency services. Starting with the tools we now have, we’re thinking about how we can strengthen our skills and approaches across the diamond – and how we might partner on projects to bring in other expertise and skills.

GRACE screenshots

Initial volunteer screen showing sections assigned:

Screen showing councils and progress

Marking screen

Right of Reply screen

Admin screen so you can check that how much has been assigned to volunteers

Admin screen to show progress on an authority level

Or on a section level

Image: Clay Banks on Unsplash.

Over the summer, we were invited to be a part of the Local Mission Zero Network consultation, and we’re thrilled that our key fragmented data policy recommendations have been included in the new report, as well as recognition for some of our wider work on climate. 

Rt Hon Chris Skidmore OBE MP, former Net Zero Review Chair and one of the co-authors of the report, said:

“The Local Mission Zero Network’s first report, The Future Is Local, sets out over thirty recommendations to further the Net Zero Review’s local delivery mission. It’s clear that if central government won’t step up, it should get out of the way and allow local and regional leaders to forge ahead with their positive vision to achieve local Net Zero in partnership with communities up and down the country. Unleashing their ambition is the most effective way to harness the economic and regional growth opportunities that Net Zero can unlock.

I’d like to thank MySociety for their involvement in the network and also for their input in making key recommendations on the need for better data and information to achieve Net zero.”

The report, co-authored with Lord Ben Houchen, released today, is “intended not only to highlight the continued challenges facing the local delivery of net zero, it also seeks to frame these challenges into a new framework for ensuring local authorities and regions have the certainty to achieve their net zero ambitions”. It is a much needed intervention, and makes clear that “in the current policy environment, and ahead of the next General Election, greater certainty over local net zero is essential”.

Within Recommendation 1, Introduce a Local Net Zero Charter to agree responsibilities and enhance partnership between the UK government, devolved governments and regional, city and local authorities, there are three specific recommendations relating to our fragmented data work:

1f) A Local Net Zero Data and Reporting Framework should be established, in order to provide consistency and increase integrity for reporting across local authorities.

1g) The Net Zero Review recommended that ONS should collect more forms of net zero related data, and this network maintains that net zero will be better delivered the more we know, and where we know action needs to take place.

1h) The need for open source and operable data is also important, if we are to encourage better uses of AI and future systems thinking. This data to be held in a central repository, supported by a central government data convenor.

In the Unlocking the value of fragmented public data report we published last year, we stress the importance of local climate data being published in a way that is useful, ultimately creating positive feedback loops across the economy. It’s great to see the report emphasise this:

“The challenge of fragmented and inoperable data standards is not merely a matter for more effective local authority performance. The future of energy system planning could be better forecast if several datasets were better aligned.” 

The body of the report also highlights our conclusions about the kinds of climate data we need:

more about how  local authorities reflect on their own progress. In these instances, free text which we can semantically search, is often most helpful. We need data around:

• Personnel, systems & processes to manage climate monitoring and reporting. This helps us to understand who is doing the work, and how resource allocation happens. 
• Progress since the last reporting period, and key areas of focus for the period ahead. This gives a vital sense of context and perspective from inside the reporting body, and helps situate the scale of work undertaken against work yet to be done.

Finally, our CAPE project was mentioned as “effective monitor[ing]”, and we were so pleased to see the work we do with Climate Emergency UK to create the Climate Scorecards recognised: “By simplifying complex data, it allowed stakeholders to identify gaps and progress in climate initiatives, empowering communities to advocate for change”.

If you’d like to read the report in full, you can find it here. You may even want to share some of the recommendations from the report with your MP, which you could do using our service WriteToThem. 

Any questions for our policy team? Get in touch: policy@mysociety.org 

Image: Minku Kang on Unsplash

We’re delighted to announce that mySociety has joined the Blueprint Coalition – an influential group of local government organisations, environmental groups, and research institutions working together to deliver local climate action with a joined-up approach. 

The Coalition works across sectoral, geographical and party boundaries to make change happen. We’re excited to join the other members in calling upon the government to provide the crucial support local authorities need to deliver on tackling the climate crisis.

About mySociety

Becoming a part of the Blueprint Coalition isn’t just a milestone; it’s a commitment to a cause larger than ourselves. As mySociety joins hands with like-minded organisations, we are poised to make significant progress in our aim to make climate-related data more accessible. We believe that more information makes for better-informed action, so everything we do puts richer, more usable data into the open, where everyone can use it. 

Our Climate, Transparency and Democracy streams consist of a number of services (such as CAPE, Climate Scorecards, TheyWorkForYou, WriteToThem, and WhatDoTheyKnow) which we bring to the Coalition alongside our research, policy and advocacy work. Our policy work has been focusing on the issue of fragmented data, and we’re excited to be planning a webinar on this topic with the Coalition – watch this space!

About the Blueprint Coalition

In December 2020, the Blueprint Coalition published a comprehensive manifesto that serves as a roadmap to expedite climate action and usher in a green recovery at the local level. It outlines the national leadership, policies, powers, and funding required to empower local authorities in making impactful changes on a substantial scale. Drawing on the first-hand experiences of local authorities that have declared climate emergencies, this blueprint serves as a guiding light for collective action towards a sustainable future.

A defining feature of the Blueprint Coalition is its central ethos of fostering partnerships between civil society, national and local governments. Recognising that achieving net zero carbon emissions requires the collaboration of all levels of governance, the Coalition’s work serves as a testament to the power of collaboration.

The Coalition partners include: 

• Ashden
• Association of Directors of Environment, Economy, Transport and Planning (ADEPT)
• Centre for Alternative Technology
• Climate Emergency UK
• Friends of the Earth
• Grantham Institute – Climate Change and the Environment (Imperial College London)
• London Environment Directors’ Network (LEDNet)
• Place-based Climate Action Network (PCAN) at LSE
• Solace

• in addition to support from London Councils and Green Alliance.

If you’d like to show your support for the Coalition, you can sign up here. And to stay updated on our Climate programme, you can sign up to our newsletter.

Any other questions or comments? Get in touch with Julia, our Policy & Advocacy Manager.

—

Image: Public domain, via Wikimedia Commons. See page for author.

We’ve used machine learning to make practical improvements in the search on CAPE – our local government climate information portal.

The site contains hundreds of documents and climate action plans from different councils, and they’re all searchable.

One aim of this project is to make it easier for everyone to find the climate information they need: so councils, for example, can learn from each other’s work; and people can easily pull together a picture on what is planned across the country.

The problem is that these documents often use different terms to talk about the same basic ideas – meaning that using the search function requires an expert understanding of which different keywords to search for in combination.

Using machine learning, we’ve now made it so the search will automatically include related terms. We’ve also improved the accessibility of individual documents by highlighting which key concepts are discussed in the document.

How machine learning helps

We’re already using machine learning techniques as part of our work clustering similar councils based on emissions profile, but we hadn’t previously looked at how machine learning approaches could be applied to big databases of text like CAPE.

As part of our funding from Quadrature Climate Foundation, we were supported to take part in the Faculty Fellowship – where people transitioning from academic to industrial data science jobs are partnered with organisations looking to explore how machine learning can benefit their work.

Louis Davidson joined us for six weeks as part of this programme. After a bit of exploration of the data, we decided on a project looking at this problem of improving the search, as there was a clear way a machine learning solution could be applied: using a language model to identify key concepts that were present across all the documents. You can watch Louis’ end of project presentation on YouTube.

Moving from similar words to similar concepts

Louis took the documents we had and used a language model (in this case, BERT) to produce ‘embeddings’ for all the phrases they contained.

When language models are trained on large amounts of text, this changes the internal shape of the model so that text with similar meanings ends up being ‘closer’ to each other inside the model. An ‘embedding’ is a series of numbers that represent this location. By looking at the distance between embeddings, we can identify groups of similar terms with similar meanings. While a more basic text similarity approach would say that ‘bat’ and ‘bag’ are very similar, a model that sorts based on meaning would identify that ‘bat’ and ‘owl’ are more similar.

This means that without needing to re-train the model (because you’re not really concerned with what the model was originally trained to do), you can explore the similarities between concepts.

There are approaches to this that store a “vector database” of these embeddings which can be directly searched – but we’ve gone for a simpler approach that doesn’t require a big change to how CAPE was already working.

Using the documents we have, we automatically identified (and manually selected a group of) common concepts that are found across a range of documents – and the original groups of words that relate to those concepts.

When a search is made we now consult this list of similar phrases, and search for these at the same time. This gives us a practical way of improving our existing processes without adding new technical requirements when adding new documents or searching the database.

Because we now have this list of common concepts, we are also pre-searching for these concepts to provide, for each document, links to where that concept is discussed within it. With this change, the contents of individual documents are more visible, with it easier to quickly identify interesting contents depending on what you are interested in.

Potential of machine learning for mySociety

Our other websites, like TheyWorkForYou and WhatDoTheyKnow, similarly have a large amount of text that this kind of semantic search can make more accessible — and we can already see how they might be useful to those relying on data around climate and the environment WhatDoTheyKnow in particular has huge amounts of environmental information fragmented across replies to hundreds of different authorities.

Generative AI and machine learning have huge potential to help us make the information we hold more accessible. At the same time, we need to understand how to incorporate new techniques into our services in a way that is sustainable over time.

Through experiments like this with CAPE, we are learning how to think about machine learning, which problems we have that it applies to, and understand new skills we need to work with it. Thanks to Louis, and his Faculty advisors for his work and their support on this project.

Image: Ravaly on Unsplash.

We’re thinking about the future of TheyWorkForYou, and we want to ground our plans in an understanding of how it is currently used and the impact it has had.

At a very practical level, it is much easier to make small changes than big ones — but small changes don’t have to have small effects. By leaning into how people are using the site, we can find ways of better supporting what people are already trying to do.

TheyWorkForYou has been politically (and culturally) influential. The way in which MPs and parties conduct themselves has changed in reaction to the service: next week we’ll have a guest post covering this in more detail. A throwaway shot in 2018’s TV thriller The Bodyguard, where a character quickly scans a politician’s voting record, shows how the idea of TheyWorkForYou has become a part of the UK’s political shorthand (with many arguments about whether this is a good or a bad thing, that we’ll come back to in this series).

That said, until recently we had no solid data as to how widely known the service is. In late 2021, Opinium gave us a set of free questions for a nationally representative poll. We used one of these to understand more about people’s awareness and usage of mySociety websites. We found that, out of all mySociety’s services, TheyWorkForYou was the one most people knew about. The poll found that one in three UK adults have heard of the site, and one in five have visited the site.

One of the biggest obstacles to successful civic tech isn’t having a good idea for a digital service, but successfully getting more than a handful of people to use it. TheyWorkForYou, with 20 years of history behind it, has crossed that hurdle. Improving and refining TheyWorkForYou is potentially a much more impactful thing to do than launching new services, but the funding environment for civic tech means there is far more money available for new projects than for steady payoffs from established work. This is a key issue we need to navigate, balancing short-term survival with a commitment to doing the things that will have the biggest positive impact.

 

Parliamentary monitoring: a slow-burn success story

A key unique feature of TheyWorkForYou is the email alerts service. We send daily emails to subscribers about the activity of their chosen MPs or Lords, or when phrases that they are interested in are used in debates or written questions/answers. On average, this means we send around 400,000 emails a month. People mostly use alerts to keep up with their own MP’s parliamentary activity, but the keyword search also means that this service  is a powerful free parliamentary monitoring tool. Alerts are used by a range of public, private and charitable sector organisations to track specific issues and keywords in Parliament.

In 2021, we ran a survey on users of alerts and found that while the majority (84%) were citizen users, a sizable proportion (16%) were using it in a professional context. Focusing on this group for a follow-up survey in 2022, we got more details of the value that charities and campaigners get from TheyWorkForYou, but we also found that the alerts are in use by people working in Parliament and government departments, improving the flow of information inside these central institutions.

These professional users were also interested in different elements of the site than citizen users, being slightly more focused on written answers and statements (as these can be the best statements of current policy), and having much less interest in voting records.   A 2016 GovLab report estimated an economic benefit of TheyWorkForYou, on time saved alone, of up to £70 million a  year to the third sector.  As a free service, it provides an important alternative to political intelligence organisations and helps level the playing field for civil society to engage with parliamentarians and decision-makers.

There is real potential here to build on something that’s going well. We can better reflect, in the way the site works, and in the work we do around it, that a key way we have impact is via intermediaries, and making Parliament far more accessible to a range of charities and organisations. The technical side works well, but we could help organisations make the most use of this feature.

 

During elections, people want different things from TheyWorkForYou

For the last 10 years, TheyWorkForYou has generally had over six million page views each year. In 2021 there were 7.8 million page views, and there were 13 million in the last election year.  In each of the last five years, there have been a million views of either the summary or voting record pages of MPs. Information on these pages also travels far further than to these direct users, as it is amplified by journalists or social media.

For the 2019 election, there was a clear increase in both overall traffic, but also in the proportion of people looking at the voting records pages. There were many views of the profiles of a small number of MPs (mostly party leaders) rather than views being evenly distributed among people looking at local MPs.

This shift towards more and more views of just the voting records pages reflected a change in the way people arrived at the site. The pattern of people entering a postcode at theyworkforyou.com, arriving at a summary page (with party comparisons), before maybe moving onto more detailed individual voting policies was becoming less common. More users were coming to the site via search engines or social media, and missing parts of the information we presented. As a result, we changed the way we displayed information, moving more of the important context from the party comparison to the voting records page.

But when the reason people are using the site changes, it’s a good time to consider how the information can best be presented. TheyWorkForYou in its design, is very focused on what happens in Parliament between elections, but the information it holds is obviously very relevant during an election. Steering into that, we could follow hints about what people want to know about (party leaders, and more widely, parties) and create new views on the information we hold, that reflect actions taken by a party over a parliamentary term. Here an existing usage suggests a different approach that could be useful to voters, that we are uniquely well placed to deliver, but which would be a substantial change in how we think about and present information.

Purposeful incremental change

TheyWorkForYou needs more than just code and servers to keep on serving the people of the UK well. Playing the biggest role we can in informing people over the next twenty years will require careful stewardship to support what works, while adapting to new problems and opportunities.

As a long-running service, the site has picked up features that are sometimes useful, but sometimes outlive their purpose or the resources that are available to maintain them. Sometimes we have turned parts off. The core challenge of project-based funding is that it can only indirectly support “doing what’s already working”, and each additional project and feature adds long term maintenance costs. This is why it’s important that, while looking for opportunities to make improvements to the site, we need to make sure that our plans still fit into a coherent idea of what TheyWorkForYou is for – so all the parts of the site are still working together in a way that makes sense in the long run.

—

Image: Maksim Shutov

 

We’ve recently published a report on fragmented data and local councils’ climate action. Download it here.

At all levels of government, and across the UK, there is growing recognition of the importance of local government in achieving the UK’s climate commitments. From this, there is a growing need to understand the impact of the interventions taking place at a local authority level, and as such, there are growing calls from central government and civil society for more climate data publishing. 

We recognise that these calls for more data do not always take into account the resources needed from within authorities to prepare this data, nor how to make the data useful to the authorities that published it in the first place.

More data publishing makes the climate data ecosystem richer, but smarter data publishing makes it more useful. If we replicate the history of previous central mandates to publish information, we will repeat mistakes that found local authorities using limited resources to put out data in ways that are far too costly to bring together and build upon.

We call this problem fragmented public data, and believe that a little bit more coordination and central support can supercharge the value of the data that local government produces. We need better tools and a better understanding of the skills and resources available to council staff. A realistic analysis of resource limitations of local government, and working with council staff who produce the data, will create more useful results, than a ‘best practice’ that requires obstructively high levels of technical skill. 

Central government has a role in providing more than an edict to publish: it must offer the support and resources to facilitate cooperation and publication of data spread over hundreds of local authorities. Net zero data publication does not have to be a burden. Together, civil society, central and local governments can come together to create a data ecosystem that is greater than the sum of its parts.  To build that ecosystem, we propose the following key principles:

• A collaborative (but compulsory) data standard to agree the data and format that is expected.
• A central repository of the location of the published data, which is kept up to date with new releases of data.
• Support from a data convener to make publication simple – such as, through validation and publication tools, coordinating data submissions, and technical support.

Take action

• Read the full report written by mySociety and the Centre for Public Data
• Sign up for our climate newsletter
• Contact Julia, our Policy and Advocacy Manager, to see how to implement our recommendations in your local area

---

Around one third of the UK’s emissions are within the power or influence of local authorities. Every local authority across the UK must be taking action to tackle climate change, in order for the UK to reach its 2050 net zero goal, and for local authorities themselves to reach their own net zero targets. 

mySociety builds services that support local authorities and their communities in their climate action planning. Through tools like the Climate Action Plan Explorer (CAPE) and our work on Climate Scorecards, we bring together local authority climate plans from across the country to make knowledge sharing easier, and the information more accessible. We know that council officers already use CAPE when seeking out best practice in different policy areas, but in order to improve these tools we need more data, and for that data to be published in an open format. That way, we can build and combine the data to make it more accessible, and more useful, for everyone. 

The data we need

1. To understand local authority emissions: structured data of council’s scope 1-3  emissions
2. To understand the effect on the area: Broader data about the local authority’s influence and activities in their wider local area
3. To understand development, setbacks and changes: Context and reflections from the public body on their own progress and future plans

Running through this, this information needs to be published in a way that avoids the problem of public fragmented data, to unlock the most value from publishing this data. 

Understanding local authority emissions

84% of councils have a commitment to bring their own emissions, or those of their area, to net zero by a certain date. To this, it’s essential for public bodies to publish structured data of their scope 1-3 emissions.

These scope 1 (direct operational), scope 2 (indirect from supplied electricity & heat) and scope 3 (supply chain & other indirect) emissions should be broken down by source and by year, from a baseline year. Reporting templates for this are already in use for local authorities in Scotland and Wales.

Understanding and sharing local authorities impact on their area

In addition to scope 1-3 emissions, the ‘one third’ figure above draws on the fact that local authorities have considerable influence over their wider local area. To help build a more detailed picture of the work taking place across local authority areas, we need structured data covering:

• Organisational plans and targets relevant to climate change, progress against these, and plans for future progress. These plans and targets are generated internally by the local authority or public body, but should capture the work they are doing internally and externally to support their wider community. For example, Aberdeen City Council’s recent submission using the Scottish Framework highlighted their Housing Strategy which included energy efficiency measures for privately owned, privately rented, and social housing.
• Details of carbon saving projects across the local area. These will be unique to the reporting body, but will again illustrate what is taking place across that authority’s local area, and may provide inspiration and opportunity for other public bodies. In the Scottish template, authorities are asked to provide structured data about all carbon savings from projects, and  the top 10 carbon reduction projects to be carried out by the body in the report year.
• Risk assessments and action plans for climate adaptation. These help to build a picture of the planning across the local area, and sharing this will help councils facing similar challenges to enhance their own planning. 

Progress tracking 

In order to provide the most useful data and tools, we also need to know more about how  local authorities reflect on their own progress. In these instances, free text which we can semantically search, is often most helpful. We need data around:

• Personnel, systems & processes to manage climate monitoring and reporting. This helps us to understand who is doing the work, and how resource allocation happens. 
• Progress since the last reporting period, and key areas of focus for the period ahead. This gives a vital sense of context and perspective from inside the reporting body, and helps situate the scale of work undertaken against work yet to be done.

Publishing data usefully

Requirements to publish data put extra costs on public authorities. As such, we need to make sure that this is done in a way that the data is most useful and accessible.

Fragmented public data is a problem that happens when many organisations are required to publish the same data, but not to a common standard or in a common location. With The Centre for Public Data, mySociety has published recommendations on unlocking the value of fragmented public data.  We recommend: 

1. A collaborative (but required) data standard to agree the data and format that is expected. 
2. An online central repository of the location of the published data, so that data users can find it easily.
3. Support from the data convener to make publication simple and effective.

This applies to information directly about climate data, but is also a useful requirement for any new requirement to publish. For instance, while both EPC ratings and datasets of council assets are required to be published, in practice the lack of a coordinated publishing approach for assets data means this data cannot be combined to understand energy efficiency of council properties across the country. 

Take action

This is an evolving document and we want your feedback! Get in touch.

• Read the Fragmented Data report (written by mySociety and the Centre for Public Data) for more detail, examples and case studies.
• Sign up for our climate newsletter
• Contact Julia, our Policy and Advocacy Manager, to see how to implement our recommendations in your local area.

---

Header image: Jason Blackeye on Unsplash

---

TheyWorkForYou’s goal is to make the UK’s Parliaments more transparent and accessible. We believe that fast access information about our elected representatives shouldn’t only be available to insiders, or those who can pay. We work to make information about Parliament accessible to citizens and to civil society.

One way we do this is through email alerts. Users of the website can sign up to receive an email when specific people speak, or specific keywords are spoken in any of the Parliaments we cover (now including the Senedd). On average, this means we send around 400,000 emails a month. While the main users of alerts are people subscribing to updates from their MP, one of our goals is that TheyWorkForYou’s alerts should lower the bar for small, often underfunded organisations to engage with Parliament. 

Last year, we ran a survey of subscribers to TheyWorkForYou’s alerts system to understand more about how people were using this feature. Through this we found more details on how the site helps small organisations stay engaged with Parliament. It is also helping those who work within both government and Parliament to access the data they need to perform their roles.

---

---

Charitable and service organisations

“We are too small to do any lobbying or to afford a paid-for service so this helps keep us in touch”. 

People working in charities told us that they used keyword alerts to track all mentions of themes relevant to their work, such as words around domestic violence; asylum and immigration; religious persecution; accessibility; nature conservation, and many more. One charity uses the site to provide briefings to colleagues before meeting MPs or looking up committee members when writing a consultation response.

“Without the site we might have to pay for a service, or give up trying to make our voice heard”.

Tracking which representatives mention keywords can help charities in identifying potentially interested parliamentarians to connect with, but can also be directly useful in organisations that deliver services, like advising people on their rights.  

“The alerts are invaluable as we don’t have the capacity to follow what’s happening in Parliament other than when we are working intensively on a bill or other activity.”

Our email alert system helps distribute the latest policy via subscriptions to written questions and answers. For instance, a child poverty group uses a subscription to written answers from Department of Work and Pensions (DWP) ministers to get clearer details of policy and policy changes. This helps them convey “up to date information to clients & even get benefit decisions changed!” 

Better flows of information can help positive feedback loops between concerned MPs and local civil society.  One respondent from a local social care reform campaign, said they “wrote [an] email to my local MP to congratulate her on her PQ and sought to update her on the govt response received so she would pursue”.

In the other direction, civil society organisations and campaigners can amplify the impact of questions MPs ask – TheyWorkForYou “enables us to ensure questions from elected members do not pass unnoticed”. Where relationships are more established, making written questions more visible helps civil society groups suggest written questions to MPs, because they can better match the language and style.

“We find your service very easy to navigate [and] a critical time-saver. It is invaluable in terms of alerting us to new developments and detailed responses we may otherwise have missed.”

---

---

Inside Parliament and government

TheyWorkForYou (and especially the alerts) continue to be part of the flow of information between and inside Parliament and Government departments.

“I rely on the alerts to stay up to date with any written questions or debates relating to the interests of the MP I work for.” 

MPs’ offices use the service to check if people live in the constituency, and for notifications of recent speeches by their or nearby MPs.

“It’s the quickest way to keep up with any questions or votes that my boss has participated in.” 

Information from TheyWorkForYou is also used as part of preparation of reports, media releases, and to support correspondence with constituents.

Devolved and local government

“As I’m an unpaid elected member your service effectively provides me with free parliamentary services which I value, especially the alert function so I can see what our MP acts on.”

Local and devolved elected officials said they use the site to keep track of developments in Westminster – making parliamentary activity more transparent helps visibility between different democratic bodies in the UK.

Civil servants

Civil servants similarly have an interest in understanding the history and views of their ministers. Respondents to our survey included civil servants from the Department for Environment, Food & Rural Affairs, Ministry of Justice, Cabinet Office, Foreign Office, Department for Work and Pensions and the Department for Education. 

They use the service to keep track of Parliamentary mentions of their department and work. Inside the DWP (one of the larger departments), one response came from a civil servant who used the alerts to shape service delivery by subscribing to questions answered by the minister. Because these answers might reflect recent policy changes, alerts through TheyWorkForYou can be a fast way for information to move around the department.

While charities highlighted that examples of existing written questions helped them draft new ones, they are also useful to civil servants when crafting responses as they can see how similar questions have been answered previously.

---

---

Other uses

Another notable group of users were academics and researchers. This includes those who study Parliament and government directly, but more broadly is useful to academics to help keep an up to date view of how MPs talk about their area of work in research and teaching. 

TheyWorkForYou is used by large and small private sector organisations to be better informed on policy changes. In some cases this includes companies who may be able to afford access to a closed, paid-for monitoring system – but lowering the barrier to entry means making it easier for everyone. Providing a service good enough for those who could afford to pay is encouraging about the quality of service being provided to those who could not.

In one private sector example, an accountancy firm uses TheyWorkForYou as part of due diligence checks on politically exposed persons. Improving the ease and quality of accessing official information about MPs’ activities (in particular given concerns about written questions and second jobs) enhances wider legal regimes around money laundering and anti-corruption. 

---

---

TheyWorkForYou and the Parliament website

Our survey did not specifically ask about this, but some respondents gave us some information about why they used TheyWorkForYou rather than the official Parliament website. While the official website has much improved, the search feature was highlighted as a reason why some respondents used TheyWorkForYou.

“Primary use is a better Hansard than Hansard (still, though Hansard has caught up a lot)” – Public sector organisation

There were several specific complaints about the search function of the official site. 

“Its [the Parliament site’s] search function barely works at all.”  – Business consultancy firm

“he search function is also better than Parliaments so when we are looking for quotes/references we will also use it to support our research.” – Researcher

“Easier to use than other sources such as Hansard’s website. Search function is much more precise and reliable” – respondent who works for an MP or Lord

In some of these cases the official site may improve in future, but in other cases there has been backsliding, such as availability of the register of interests. TheyWorkForYou has value as a backstop on the official service where it has flaws, but also in providing services like the email alerts that go above and beyond what the official service is ever likely to offer.

While our main focus as a service (and most of our visitors and alert subscribers) are individual citizens, supporting and amplifying the power of small civil society groups helps ensure a more level playing field of access to decision makers. In future, we’d like to be able to explore this path more, and provide better advice and guidance on how to make the best use of our tools to groups that would otherwise struggle to access the Parliamentary process. 

—

Image: Monisha Selvakumar

 

Links

• Online version of this document
• Application form
• Potential collaborators form
• Join the research commissioning mailing list/Research pool
• Download PDF

In one sentence

mySociety is looking for an individual, organisation or joint team to explore requests made through WhatDoTheyKnow.com to understand how requests for environmental information are being used in practice, highlighting success stories in releasing information at the local level, problems requesters encountered, and suggestions on how mySociety could better make use of the environmental information requests. 

About mySociety

mySociety is the charity behind UK civic services like TheyWorkForYou, WhatDoTheyKnow, FixMyStreet, and WriteToThem. We build open, digital solutions to help repower democracy, in the UK and around the world.

mySociety’s climate programme is funded by Quadrature Climate Foundation and the National Lottery Community Fund.

About mySociety’s existing work in this area 

mySociety’s Climate Programme is exploring how digital tools and approaches can work to reduce the third of UK emissions that local authorities have influence over. A key focus of this work is improving the quality and quantity of information that exists around local climate action. We make local authority plans more accessible and visible, campaign for better official data, and support pooling and crowdsourcing of data to improve knowledge and accountability for climate action. You can read more about mySociety’s Climate programme here.

WhatDoTheyKnow.com is a website built by mySociety, and administered by mySociety in partnership with volunteers, that helps people make Freedom of Information (FOI) requests and publishes the results in public. Over 900,000 FOI requests have been made through this platform. We have published research looking at the scale of FOI in local government, and in improving FOI through the UK, and Europe. 

About this project

Our ultimate goal in our climate work is to decrease UK carbon emissions that are either directly controlled or influenced by UK local government. Our main approach is by improving the information environment so that a variety of local institutions and actors are better able to understand the current situation, share knowledge on their approaches, and take more informed action. We have done this by making local authorities own plans and documents more discoverable and searchable, supporting crowdsourcing of information about plans and actions, pooling information held by different groups of campaigners, and arguing for improved publishing of official information. 

Another method open to us is to use (or facilitate the use of) Freedom of Information laws to get more official data and information into the public domain. Freedom of Information laws give a wide range of public access to information held by authorities. This is useful in releasing information about authorities’ own plans and actions, but is also useful in releasing underlying information which local authorities, so that other organisations (including companies, charities, and other public sector bodies) can use to inform their own approach and actions. 

The Environmental Information Regulations (EIR) substantially overlap with Freedom of Information. EIR requests make it easier to ask for and receive information specifically related to the environment, and covers a wider range of organisations than the general Freedom of Information law. We believe there is the potential to use improved support and guidance for Environmental Information Requests to facilitate more information relevant to local climate action being released, and being made discoverable by local authority actors, civil society and communities. This might take the form of modifications to WhatDoTheyKnow, or light-weight specialist services that sit on top of WhatDoTheyKnow’s FOI archive and infrastructure (similar to FragDenStaat’s Climate Helpdesk). 

But before we can do that, we want to understand more about Environmental Information Requests. WhatDoTheyKnow’s archive gives us access to a huge number of requests made for environmental information, but we understand very little about the contents of these requests. 

Based on a quick text search, we have a maximum of 70,000 requests that in some way mention environmental information regulations. As part of our wider pool of 1 million FOI requests, there may be other requests with environmental interests that have not explicitly mentioned EIR. We are looking for a contractor to explore the large WhatDoTheyKnow dataset, and highlight themes, patterns, and individual examples that show how EIRs are being used, obstacles users are encountering, and the implications of this for mySociety’s future work in this area. 

The available budget for this work is up to £8000-10,000 (inclusive of VAT), and the project would need to be completed by the end of July. There is a small amount of flexibility on this end date – but we are looking to make decisions in August about best ways to follow up the project in September. 

What we want to know more about

The broad goal of this project is to refine our understanding of how EIR works in practice. We’ve broken this into six areas we’d like to understand more about. 

1. EIRs have fewer exemptions, and a higher threshold for withholding material on public interest grounds, than FOI and. In principle, relevant information should be easier to get through EIR, but it is unclear how much this is significant in practice. We want to know more about the kinds of information that are successfully being released through EIR.
2. EIRs apply to more types of organisation – and organisations that have an important local footprint may be accessible to FOI. We want to know about the kinds of institutions and authorities who are or could be releasing information under EIR that is relevant to our local emissions goals.
3. Authorities should consider which regime a request falls under – and default to the more permissive EIR regime when a request is for environmental information. We want to know if there are cases where information is being withheld under FOI but should have been considered under EIR.
4. EIR might be more poorly understood by both requesters and authorities – information may be being incorrectly withheld that should be released. We want to understand any patterns of refusal that would inform advice tools. 
5. At the moment we don’t have an automatic way of identifying EIR responses (when authorities have judged a request under these rules), but it should be possible to create classification rules that separate them from FOI requests. We want to refine this approach so we can more consistently identify and analyse EIR responses, so they can be in current or future local climate services.
6. The big way in which EIRs are less accessible is that authorities can charge fees without a minimum amount of time taken, which is the case for FOI requests. Generally we think this doesn’t happen much (the ICO’s guidance is not to charge for a reasonable request), but we don’t know. We want a clearer sense of how often authorities ask that requesters pay a fee for environmental information.

What we are looking for in and from a partner

Expertise / skill set

We think there are a few different approaches to this project and are open to a range of approaches. In general we anticipate two main kinds of applicants: one leaning more on technical ability, and the other on specialist knowledge of FOI and EIR, or environmental data more broadly. For either kind of applicant, we can provide basic support in the other skillset. The ideal candidate would cover all of these bases (where a subject matter expert makes the technical search process sharper) – and we can help facilitate partnership applications between technical and specialist partners who might otherwise submit separately. If you are interested in this route, please fill out this collaboration form and we can help put you in contact. 

The key thing an applicant needs to have is an approach to searching the large quantity of information on WhatDoTheyKnow. At a minimum this requires some technical skills and ability to navigate the site – but beyond that might be accomplished by large scale text analysis, or a sampling and search method. WhatDoTheyKnow’s search allows searching for specific search terms – but the search count is not always accurate for larger queries. 

For technical strategies, we can provide data exports to enable searches off-platform. But for subject matter experts, we can also provide an Excel sheet of links to URLs that have triggered particular keywords, and authorities. 

Something that is important to us is being able to point to individual examples – and so approaches based on aggregate analysis (which might be the better approach to identify the scale of fees being charged, for instance, or automatically analyse themes of requests) need to also be able to drill down to individual requests. 

With this in mind, the following is a list of skills we would expect the candidate to have a couple but not necessarily all of the following:

• Technical skills
  • Experience with search and processing large amounts of text delivered in a mostly unstructured way.
  • Experience with Natural Language Processing (NLP) or automated topic extraction. 
• Subject matter skills
  • Knowledge and experience of Freedom of Information and/or Environmental Information Requests
    • Especially practical understanding of refusal and appeal grounds in both FOI and EIR.
  • Less important, but understanding of the wider European context of Environmental Information Regulations might provide additional understanding of potential approaches mySociety could take in the UK. 

Alignment with values and aims

Our Repowering Democracy strategy puts a special emphasis on embedding equity and inclusion in our work practices and services, and our work aims in general to fulfil values of equity/justice, openness and collaboration. 

Applicants should consider if this presents any obstacles to a working relationship, and think about how these values should be reflected in the project plan, either in terms of subject matter to investigate, or research approach.

Working practices

mySociety works flexibly and remotely, and there is no requirement to work from or visit an office. Applicants can distribute their work as appropriate over the time available, but we would expect regular check-ins on progress to be arranged over that period. A shared Slack channel and a specific contact person will be used to help coordinate and quickly share questions and information between mySociety and the researcher. 

Successful applicants would be expected to abide by the mySociety Code of Conduct in mySociety communications channels and events. 

Outputs and deliverables

The purpose of this project is to inform mySociety’s future projects, especially forming the base of knowledge around the use of EIRs for a prototyping week.

We are open to the form of outputs – this may take the form of one large report, briefings around our individual question areas, proposed amendments to guidance, etc. For technical submissions, analysis code under an appropriate licence would form part of the output.

The outputs should also work as a general contribution to knowledge of the current use of EIRs, and we might either publish or edit down and publish these briefings/reports.

Q&A and contact details

The application timeline includes a Q&A event, to which you can sign up at the link at the top of this document. The Q&A session will include an element to help individual researchers coordinate to form a joint submission (applications are also welcome from individual researchers). Answers will be made available in a video on this page for applicants who cannot take part. Questions can be emailed to the contact address below. 

Please send any queries or questions to research-commissioning+eir2023@mysociety.org and mention which project it is in regard to. Questions in advance are preferred and will be prioritised in the session. 

Your application

Applications can be submitted by individuals, organisations, or joint teams of individuals/organisations. These should be sent to research-commissioning+eir2023@mysociety.org by the closing date. 

You should submit a short application, of up to 4 pages of A4. A template for the response can be downloaded at the link at the top of this page, and covers: 

• Who you are (whether an individual, organisation, or joint team).
• A description of your previous experience/previous work and why you want to take on this project.
  • To the extent that this is possible, this should be anonymous and not include names of the org or members of the team (to help with anonymous stages of the recruitment process)
• How you would approach and deliver this project – a short project plan with approximate timings. 
  • This could include discussion of appropriate outputs for the project, and balancing technical and subject matter requirements. 
• The total value (£) of your proposal (including VAT), and high-level breakdown of costs (perhaps an indication of days per person, any other expenses). This does not need to include production costs of the report. 
  • Given the cost of the project, we will not be giving a great deal of weight to budget plans so please keep this short and high-level – we can dig into further details during interviews, if necessary.
• A short description of the individuals or team who will do the work, including biographies

There is a separate equalities monitoring form to fill out, which is processed separately from the main application (there is a link to the form in the application form). This is for understanding the reach of our method of distributing the call for proposals. 

If you are interested in joining a ‘researcher pool’ mailing list that we will contact with details of future projects, please see the link at the top of this document. 

Application timeline

If there are changes during this timeline, the table on the website version of this form will be updated. 

Stage	Date	Description
Call for proposals published	7 March 2023
Q&A Webinar	21 March 2023	An open, online public event for interested bidders to learn more about the project and ask questions. This will be recorded and available afterwards. You can submit questions in advance to research-commissioning@mysociety.org. Questions in advance are preferred.
Questions answered	23 March 2023	Video of the webinar to be made available to all potential bidders, in addition to answers to any other questions submitted via email
Deadline for applications	31 March 2023 (end of day)
Initial decisions	7th April 2023	Applicants to be informed whether they have made it through to a short panel interview (and may be asked for a sample of existing work). Applicants not progressing past this stage to be offered written feedback
Interviews	w/c 10 April 2023	Format to be decided, but this will likely be a one-hour panel interview with several people involved in the climate programme, towards the end of the week (14th, 15th April)
Final decision	w/c 18 April 2023	Remaining applicants to be informed of the final decision. Applicants not progressing to be offered feedback
Project briefing/kick-off meeting	End of w/c 18 April 2023	To include a brief introduction to mySociety, discussion of any onboarding required and approach to project management, communication and catch-ups
Project deadline	End of July 2023	End of project (Time range of project is a little flexible – we want it to inform decisions in August about any follow up work in September)

What happens after the project 

We intend to publish the report or briefs you produce, credited to you, on the mySociety website, licensed under a Creative Commons licence (see recent publications on research.mysociety.org for details). We may make some light edits (beyond proofreading) before we publish. You will be free to make publicly available your own version should you wish to, and any other material based on the research you conducted. 

We will convene a short ‘lessons learned’ session for the contractor and mySociety to discuss how the project went – what went well and anything that could have been improved. We will also discuss any future work based on the delivered project (e.g. if you are an academic and might want to co-author an article) and our ongoing relationship. We would also like to arrange a presentation on the project to mySociety staff, and there may also be an opportunity to promote the work in a public event held by mySociety (budgeting for this would be separate to the project above). 

Terms and conditions

Interested parties must be UK-based individuals or organisations.

 

Questions and Answers

 

For data analysis, what format is the data available in?

We create a regular research export of the database in CSV format. This doesn’t include the last few months of data and removes requester names where possible (this is generally available on the website, but we are trying to future proof against information being too available if redactions are needed in future).

There is a a data table of requests made by authority, and then a dataset of the individual messages (with full text) for each request. 

We currently don’t make the downloaded files themselves available in bulk (but are accessible through the site) – but the bulk export does include any cached conversion of a word document or pdf attached.

We can produce reduced data sets limited to specific authorities or keywords. For instance, to those flagged as potentially EIR projects at the start.

WhatDoTheyKnow data sharing policy for bulk data

What is the time scale for the research?

Broadly we’re looking for work to be completed by the End of July to help us inform how we spend our time in September. Depending on the nature of outputs, there may be some flexibility around this.

The retained EU law will ‘sunset’ EIR at the end of the year, unless explicitly retained. How does this work relate to that?

Our current default assumption is “everything will be fine” and this project can proceed as if EIR will continue past 2023. In the event it is looking like everything is not fine, this research helps us understand more about the impact of EIRs for campaigning purposes.

How have you handled projects like this before and how have they worked?

We’ve previously commissioned two pieces of research: one was about how we should commission research, the other was the role of local government in climate change. In both case we worked with a sole researcher, with regular check-in meetings and a shared slack channel. That said, we’re open to group applications (and this project may be appropriate for that). Neither of those projects was particularly data heavy, but we have worked to provide external researchers with data before. 

What support will you give the project?

The main support and contact for the project will be Alex Parsons (Senior researcher) from aa research and data perspective. There is limited available of the WhatDoTheyKnow on a day to day basis team – but I can either answer questions or get answers to questions we need to know.

—

If you have any other questions about this project or the application, please email : research-commissioning+eir2023@mysociety.org

As a joint project between mySociety and the Centre for Public Data, we have written a set of simple principles for how to get the most impact out of publishing public data.  You can read the report online, or download it as a PDF. 

Fragmented public data is a problem that happens when many organisations are required to publish the same data, but not to a common standard or in a common location. Data is published, but without work to join up the results, it rarely has the intended impacts. 

The results of this are frustrating for everyone. Data users cannot easily use the data, policy makers do not see the impact they want, and publishers in public authorities are required to produce data without seeing clear results from their work. 

Better and more consistent publication of data by local authorities helps enable understanding and action at scale across a range of areas. At the same time, we recognise that the technical advice given has assumed higher levels of technical capacity that in practice is possible for many data publishing tasks. Our goal has been to make sure our advice makes data more accessible, while having a realistic idea of technical capacities and support needed for data publishing. 

This report recommends three minimum features for a data publishing requirement to be successful: 

1. A collaborative (but compulsory) data standard to agree the data and format that is expected.
2. A central repository of the location of the published data, which is kept up to date with new releases of data.
3. Support from the data convener to make publication simple and effective – e.g. through validation and publication tools, coordinating returns, and technical support.

We recommend that:

• Whenever government imposes duties on multiple public authorities to publish datasets in future, it should also provide the staff and budget to enable these features.
• The Central Data and Digital Office should publish official guidance covering the above.

You can read the report online, or download it as a PDF. 

Better data publishing helps climate action

This project is informed by recurring frustrations we have run into in our work. Projects such as KeepItIntheCommunity, which mapped registered Assets of Community Value, were much more complicated than they needed to be because while transparency was required of authorities, coordination was not – meaning the task of keeping the site comprehensive and updated was enormously difficult. In principle, we could build a tool that empowered communities in line with the intentions of the original policy makers. In practice, a lack of support for basic data publishing made the project much harder than it needed to be.

This problem also affects our work around local government and reducing emissions. Local government has influence over one third of emissions, but much of that is indirect rather than from the corporate emissions of the authority directly.  As such, many activities (and datasets) of local government have climate implications, even if the work or data is not understood as climate data. For instance, the difficulty in accessing the asset data of local authorities makes it harder for civil society to cross-reference this information with the energy rating of properties, and produce tools to help councils understand the wider picture. 

In future we will be publishing in more detail the kind of data we think is needed to support local authorities in emission reduction – but emissions reduction cannot be isolated from the general work of local authorities. Improving the consistency of the data that is published helps everyone better understand the work that is happening, and makes local government more efficient. 

Sign up to hear more about our climate work, or to the monthly mySociety newsletter. 

Photo credit: Photo by Olav Ahrens Røtne on Unsplash