This blog post is part of our Repowering Democracy series. We are publishing a series of short pieces of writing from mySociety staff and guest writers who are thinking about how our democracy works and are at the frontlines of trying to improve it.

This week, we’re re-publishing a blog post from Anna Powell-Smith at the Centre for Public Data, which is a new, non-partisan non-profit working for stronger public data. We’re previously worked together on recommendations to avoid fragmented public data. This blog post touches on several issues close to our hearts: Parliamentary written questions, and where there isn’t enough data to understand what’s going on.

—

Data gaps are under-reported, because it’s hard to write about data that doesn’t exist.

As we’ve written about before, newspapers publish endless stories on house prices, where there’s lots of data – but few on rental costs, even though millions of people rent. That’s partly because the Office for National Statistics doesn’t collect much data on rentals.

To tackle this problem, I’ve been thinking about how to map data gaps, and make them more visible.

And I think the best way is actually to think about questions, instead of data. What are the important questions that the government can’t answer?

Obviously, ‘important’ is subjective! But one source of clearly important questions is Parliamentary written questions, which are the formal questions that MPs and peers ask the government. Where the government doesn’t have the data to answer them, it has to say so.

So this post introduces new research: a data analysis of 200,000 Parliamentary written questions, and what they tell us about the UK’s missing numbers.

Our modest goal: to find the UK’s biggest data gaps.

What we did

Building on some previous research of ours, we strapped on our coding hats 🪖, and did the following:

• First, we scraped all the written questions in Parliament from December 2019 to February 2023, from TheyWorkForYou, which gaves us about 200,000 questions.
• Next, we flagged questions asking for quantitative information, with phrases like “how many” or “how much” – which showed that about a fifth of questions wanted data, just under 40,000.
• Then we flagged questions where the government apparently said the data was “not held”, “not collected”, etc. About a quarter of quantitative questions were answered like this.

And we ended up with a dataset of around 10,000 questions where MPs apparently both (i) asked for data, and (ii) were told it was not available. So: missing numbers.

Then we spot-checked the questions to check our method. It wasn’t perfect, but it was very decent. (It helps that Parliament uses formal, consistent language.) You can download the full dataset here.

Sometimes, MPs ask about strange things, like jobs for clowns. But most are extremely serious, covering the issues that affect MP’s constituents. And overall, they tell us what MPs need to know.

Data gaps by department

Firstly, we looked at how often each government department said that data wasn’t available. (See the code.) And there were were huge differences:

• At the Department of Health & Social Care, around 40% of quantitative requests were unanswered (though we can cut them some slack, as this was during the Covid pandemic).
• At the Home Office and the Department for Work & Pensions, around a third were; at the Ministry of Justice the proportion of unanswered quantitative requests was 30%, and the Department for Education 27%.
• But the proportion was much lower at other big departments – almost all others were below 20%.

Of course, we need to be cautious here, as the numbers are approximate. Without reading each question, we can’t be sure that we’ve tagged it correctly, or if the MP was asking something impossible. It’s probably most useful to consider the differences between departments.

Given that, it’s not surprising that the health, benefits, justice and education departments would get requests for data, since they run massive operational services that affect people’s lives. (The Foreign Office, by contrast, largely seems to get asked about wine.) It’s more surprising that they seem to struggle to answer them more than other departments.

Now let’s dive into what these unanswered questions were about.

The topics with the biggest data gaps

Each question scraped has a title. We can use this to see which topics were least likely to get an answer.

Other than Covid-related topics, the major topics with the highest proportion of unanswered questions were:

1. Benefits – grouping together benefits like Universal Credit and PIP
2. Asylum, refugees and migrants
3. Child maintenance
4. Energy meters
5. Armed forces housing

This seems plausible. The DWP Select Committee has repeatedly criticised the government for the lack of visibility over the benefits system; the statistics regulator has expressed concerns about the use of asylum statistics, while the National Audit Office has noted gaps in the data available on smart meters.

We also used GPT-4 to try tagging questions, which worked quite well. We used it to tag questions to the Department of Health & Social Care. This helped us identify major clusters of unanswered questions in these areas.

In healthcare, MPs often struggled to get basic prevalence information, whether:

• the number of people diagnosed with particular conditions, like silicosis
• diagnoses broken down by characteristics, like the number of women with meningitis, or region, like autism in the East of England
• or diagnoses for particular (important) groups, like prisoners with mental health conditions.

Also, funding is a topic it’s surprisingly difficult to get information about, e.g.

• funding for particular conditions, like endometriosis, or
• funding on social care by local authority
• funding at a local level, especially per hospital, which MPs often care about.

Following on from this, hospital-level information in general often seems to be poor, e.g.:

• how many A&E visits there are per hospital
• what waiting times are per hospital.

And finally, workforce is a huge one, with topics like:

• vacancies – how many current GP vacancies are there?
• retention – how many dentists are still working 5 years after qualifying?
• skills – how many specialists are there with particular skills, like Parkinson’s nurses or motor neurone nurses?

You can see the tagged questions here – there are many more examples under each topic.

This gets really worrying when you look at the dataset over time. It’s immediately clear that MPs often ask the same thing over and over again – yet the information doesn’t seem to improve.

What next?

We think statistics producers should be monitoring Parliamentary questions, to tell them where data needs to be better. After all, MPs deserve answers to their questions, and so do we all.

If you can help us make this happen, we’d love to talk.

If you’re interested in this research – or even better, if you can fund us to do more of it! – please do get in touch.

Image: Tom Chen on Unsplash.

This blog post is part of our Repowering Democracy series. This year we will be publishing a series of short pieces of writing from mySociety staff and guest writers who are thinking about how our democracy works and are at the frontlines of trying to improve it.

This week Dr Kathryn Rix writes about the opening up of parliamentary information in the 19th century. TheyWorkForYou is twenty years old, but in many respects is a digital continuation of similar projects and arguments about parliamentary transparency that go back centuries.

Learning more about this history helps situate our work in the longer context – and this period (with league tables of MPs, arguments about league tables of MPs, and a clear illustration of the link between changes to the physical building and transparency), is one with obvious application to our work today.

—

On 22 February 1836 a landmark vote took place in the House of Commons. It was significant not because of the issue involved – a local railway bill – but because for the first time, MPs left the chamber to vote in two separate division lobbies, following which an official division list was published, recording the names of every MP voting in the majority and minority.

The publication of these official lists from 1836 made it far easier for those outside Westminster to scrutinise and assess the activities of their representatives. One Radical MP described this change as “perhaps one of the most important measures ever sanctioned, as a check upon the conduct of members”.

Before this ground-breaking division, the official records of the Commons gave only the names of the tellers who counted the votes in each division, and the total number of MPs voting on each side. The Commons had just one lobby, so when divisions took place, the presumed majority stayed in the chamber to be counted there, and the presumed minority went out into the lobby.

The informal record

Although there was no official record of how each MP voted, some MPs – notably the Radical backbencher Joseph Hume – compiled their own lists of divisions and supplied these for publication in the newspapers. However, these unofficial lists were usually produced only for more important divisions and sometimes gave the names of those in the minority, but not the majority. They were notoriously inaccurate, and MPs regularly wrote to the press correcting errors.

Pressure to improve this system and publish a full official record of every division increased after the 1832 Reform Act, the first major reform of the British electoral system. One of the Act’s key aims was to restore public confidence in government by making the Commons more responsive and accountable to public opinion. In this spirit, the Radical MP Daniel Whittle Harvey put the case for official division lists, arguing that:

“every person now acknowledged that responsibility, and not secresy [sic] and concealment, was the basis of the trust reposed in the hands of Representatives by their constituents … In a Reformed Parliament he believed, that all hon. Members would be desirous that their constituents should know how they voted”.

A select committee in 1834 felt the best option for compiling accurate official division lists would be to construct a second division lobby, but the cramped conditions of the Commons chamber made this impractical. However, when MPs moved into temporary accommodation following the catastrophic fire at Westminster in October 1834, this obstacle was removed. The additional second lobby, which facilitated the publication of official division lists, was built during the 1835-6 recess. It became an integral part of the Commons and was replicated in the new Palace of Westminster designed by Charles Barry.

Use by newspapers and journalists

Information extracted from the official division lists was widely reproduced and analysed in the newspaper press, as well as in guides to MPs’ voting behaviour, such as An Atlas of the Divisions of the House of Commons (1836), which listed every MP’s vote in the 1836 parliamentary session in a tabular form. Such analysis of MPs’ votes was not entirely new. Richard Gooch’s The Book of the Reformed Parliament (1834) had tabulated MPs’ votes in selected divisions in 1833 and 1834, and had been used at the 1835 general election to challenge MPs in several constituencies about their attendance levels at Westminster. MPs had, however, disputed the accuracy of Gooch’s publication, based as it was on unofficial records. At Droitwich, where his opponent calculated that he had voted in just 11 of the 116 divisions listed by Gooch, John Foley insisted that his votes “had been given much nearer ninety-nine times than nine”.

Extract from Atlas of Divisions (1836)

The publication of official division lists meant that MPs could no longer try to shirk responsibility for particular votes by claiming that they had been misreported. Another highly significant development was that, with all divisions fully recorded, accurate calculations could be made of how many times MPs voted each session. The Atlas displayed these totals against each MP’s name, a novel feature which one commentator tellingly described as “a scale of diligence” by which voters could measure “the conduct of representatives”.

Rankings and league tables

National and local newspapers compiled and dissected figures on MPs’ attendance levels on a weekly and annual basis, making it much easier for voters and the wider public to access this information and use it to call MPs to account. While annual attendance statistics sometimes listed their names alphabetically, it became common for MPs to be ranked alongside their colleagues – either nationally or regionally – on the basis of how often they had voted, producing what were effectively ‘league tables’ of MPs. The Gateshead Observer referred to its annual attendance tables of north-eastern MPs as a ‘parliamentary audit’ or ‘reckoning day’. MPs had been accustomed to explaining to their constituents how they had voted, but increasingly also found themselves having to justify how often they did so.

For those MPs who appeared towards the top of these tables, these statistics provided welcome proof of their diligence. After being ranked as the fifth most attentive MP during the 1840 session, Henry Salwey, MP for Ludlow, was praised by his supporters as “most assiduous, and most constant – ever vigilant in promoting … the local interests placed in his care, and the general welfare of the community”. In contrast, other MPs and their supporters rejected attempts to reduce their Commons contribution to mere numbers. At the 1841 Hertfordshire election, the voting record of the Conservative MP Abel Smith was compared unfavourably with his Liberal opponent, who had voted three times more often. One of Smith’s backers argued, however, that “we don’t count the number of divisions – we look to the importance of them: we don’t wish our member to sit through every paltry discussion – as though nailed to the benches”.

The reaction of the quantified MP

The question of how useful these attendance figures were as a measure of MPs’ commitment to representing their constituents was widely debated during the nineteenth century. The Morning Chronicle was not alone in mocking the idea that an MP’s “whole duty … consists in walking in and out of the lobbies”. It was pointed out that notable figures such as Lord John Russell (a former prime minister) – who voted in 28 of 198 divisions in 1856 – and William Gladstone (a future prime minister) – with 58 votes that session – would be found lacking if judged only by this measure. The ‘Division-list Test’, as one MP labelled it, failed to take account of MPs’ contributions in other areas of the work of the Commons, particularly in serving on committees. Although one Worcester newspaper noted the relatively poor attendance of the local MP, Joseph Bailey, in divisions, it observed “in justice” to Bailey that “his labours on Committees have been incessant”.

There were other reasons why voting in numerous divisions was not necessarily seen as demonstrating dedication to parliamentary business. The influx of MPs from the smoking or refreshment rooms when the division bell rang, without having heard the preceding debate, was often commented upon.

Such behaviour prompted the Conservative MP Charles Adderley to argue that there could be “no worse test of a man being a useful member of Parliament”, since “a man might attend every division… and be the idlest dog in the House”. Another argument against testing MPs’ diligence in this way was that the overall totals did not distinguish between critical issues and less significant matters, such as local or private bills which did not affect the MP’s constituency. William Scholefield, generally seen as a hard-working representative, told his Birmingham constituents that he had deliberately abstained in many such cases, since “I will never vote on a bill unless I distinctly know what I am going to vote about”. Yet while the use of the “Division-list Test” could be challenged in various ways, it continued to be seen as a useful indicator for constituents in deciding whether their representatives had been attentive or neglectful in carrying out their parliamentary duties.

Images: ‘Division barrier and lobby’ and it is taken from pg. 409 of J. Ewing Ritchie, The life and times of William Ewart Gladstone. The pictorial edition Volume I.

Dr Kathryn Rix is Assistant Editor of the House of Commons, 1832-1945 project at the History of Parliament Trust, which is currently researching electoral and parliamentary history between 1832 and 1868.

This blog post is part of our Repowering Democracy series. This year we will be publishing a series of short pieces of writing from mySociety staff and guest writers who are thinking about how our democracy works and are at the frontlines of trying to improve it.

This week Professor Kate Dommett, writes about different ideas about how MPs represent us, and the implications of that for how sites like TheyWorkForYou create and display information about representatives.

Key points:

• People want different things from representatives…
• …but it’s not as clean as that: people see the value of different representation styles at different times.
• Sites like TheyWorkForYou shouldn’t assume a one-size-fits-all approach, but cater to different possible lenses.

—

Why do we want information about how our politicians behave? This answer to this question initially appears simple – we want to hold our elected representatives to account. And yet it’s not at all clear that people want the same information, or that they use it to evaluate representatives in the same way.

Some people might want to know whether their representative delivered on a promise to defend the local hospital; another might want to know if their MP has stuck to the party line when casting votes. Another may wish to know not about voting, but about whether their MP has taken on a second job, whilst a different person might be interested in knowing how much time their representative spent contributing to debate in the chamber.

The information that we want can vary, but currently citizens tend to be given a relatively uniform set of insights. In this blog post, I want to think about the different types of information that citizens may desire, and the alternative ways that a site like TheyWorkForYou could present information.

What do we know about elected representatives?

Given you’re reading this post, you’re likely to be familiar with the format of TheyWorkForYou and have previously looked at the information the site provides about recent votes, news and upcoming debates in your chosen Parliament. You may even have used the ‘Find out more about your MP’ search function and looked at your representative’s voting record and register of interests. Yet what you may not have considered before is why you are being shown those particular pieces of information.

Sites like TheyWorkForYou are making a series of assumptions about what people may want or need to know about their representatives. They have decided that people should know how MPs vote on key issues, whether they tend to vote with their party or not, what debates they are contributing to and what their interests are. These are all important insights, but the choices made raise a series of questions: why are we being presented with this specific information? What else could we have been shown? And why might we want to know different things about what MPs do?

Styles of representation

Representatives might be judged on the data presented, and academic scholarship has suggested that people can make a range of different types of calculations when they determine how to vote. Some people vote in accordance with party ties, others focus on future promises, whilst others make choices based on how a representative performs. It is this latter, evaluative style of judgement that is of interest here, and academic research has shown that evaluations can be based on many different types of evidence.

One way of thinking about the information citizens might use to make an evaluation about their representatives is a framework originally offered by the 18th century MP and political thinker Edmund Burke. Burke’s ideas have been developed to distinguish three different styles of representation:

• Delegate – Representatives should act as a conduit for citizens’ desires
• Trustee – Representatives should not seek to simply reflect and respond to citizens’ whims, but should instead exercise their independent judgement and act as ‘trustees’ of citizens’ interests, which might involve taking decisions which, though in constituents’ and/or the national interest, would be opposed by citizens themselves
• Partisan – Representatives should act in accordance with the goals and objectives of a particular party agenda or ideology

These different styles of representation are interesting because they provide alternative metrics on which representatives can be evaluated.

Previous polling on these styles of representation suggests that when you ask people which style they prefer, the ‘delegate’ style of representation comes out on top. Indeed, in a poll by YouGov which asked whether an MP should vote in accordance to his or her judgement, or according to the majority view of his or her local electorate, the latter (delegate style of representation) was favoured by 58% of respondents, while 29% supported the trustee model (with 13% ‘Don’t know’). This kind of representation can be overtly offered, indeed Andrew Grey – a recent candidate at the Selby and Ainsty by-election offers an interesting case in point.

From this, people appear to primarily want information about the degree to which MPs act in line with their own (or their local area’s) preferences. And yet, when people are not asked to choose just one representative style, it becomes clear that people can value all three forms of representation. Indeed, my own research has shown that equal numbers of survey respondents think that representatives should act as delegates and trustees (72% of respondents), whilst partisan representation is favoured only slightly less (66%). People value multiple styles of representation and hence are likely to want information on representatives’ performance in each category.

Evaluating representatives

Thinking back to what’s covered on TheyWorkForYou, we can see some useful information about MP’s representative styles. The pages provide insight on partisan voting behaviour, revealing whether or not the MP votes the same way as other MPs from the same party. And yet, there are not clear indicators of the degree to which the MP acts as a delegate or trustee.

Whilst finding indicators of these representative styles is challenging, there are some possibilities. It would be possible, for example, to look at the degree to which MPs made pledges to their local constituency and monitor the degree to which they then honour those pledges in office. One MP who has done this previously is Gisela Stewart, who provided a personal manifesto alongside the party manifesto.

It’s also something done by local councillors who often provide local place-based manifestos that outline pledges to their local area. These provide useful metrics of the degree to which representatives are acting as delegates in line with their pledges, and sites such as TheyWorkForYou could capture such pledges and then trace the degree to which an MP’s voting record matches these promises.

To monitor trustee behaviour, it would also be possible to highlight the number of times particular votes were introduced or justified as important for advancing the national interest, looking at the justifications given for particular votes and the degree to which these claims were contested in debate.

Taking multiple approaches

Whilst the precise metrics require refinement, the principle of measuring and reporting different styles of representation chimes with the idea that people can draw on different types of information to make assessments about their representative’s performance. Recognising this, sites like TheyWorkForYou could create different filters when presenting information, asking users which type of information is most important for them, and presenting data accordingly. An individual concerned with partisan loyalty could therefore receive detailed breakdowns about party voting and where and why representatives depart from the party line. Alternatively, someone placing equal value on trustee and partisan representation could be shown performance information on both metrics.

The value of thinking about what information citizens want and need when evaluating their representatives opens the door to an important debate about the different metrics that can be used to judge MPs. It allows closer reflection on how and why different information can be presented to users, suggesting that a uniform strategy may not always be suitable.

Yet, it also encourages recognition of the very different standards that can be used to evaluate representatives and, in so doing, it highlights the challenge of being an MP. Far from having to comply with one standard of behaviour, MPs have to balance a range of different imperatives and are hence unable to satisfy everyone. But at the moment MPs rarely shape expectations about how they plan to act in office and what representative style they intend to follow.

There is a case for MPs themselves to take some additional responsibility in shaping expectations, making it clearer to citizens how they intend to behave and which metrics are therefore most appropriate for evaluating their behaviour in office.

From this perspective, websites such as TheyWorkForYou can make it easier for citizens to evaluate representatives, both by presenting a range of different metrics and clearly signalling to citizens how and why they might want to look at different pieces of information. But they can also encourage MPs and other representatives to be clearer about how they choose to work, helping to establish clearer expectations of representative politics.

Image: Atanas Chankov on Unsplash.

This blog post is part of our Repowering Democracy series. This year we will be publishing a series of short pieces of writing from mySociety staff and guest writers who are thinking about how our democracy works and are at the frontlines of trying to improve it.

This week’s guest post comes from Dr Ben Worthy. He’s been looking at the ways which sites like TheyWorkForYou impact politics. We’ll be incorporating things we learn from guest writers into our future thinking around our work.

—

Our world is awash with information but does more data make for a better democracy? Between 2019 and 2022, our Leverhulme Trust-funded study looked at the impact of monitoring sites like TheyWorkForYou on the UK Parliament.

While our research focused on TheyWorkForYou, there are now a whole range of others, from the new Westminster Accounts, which allows you to see MPs’ interests and donations, to (my personal favourite) this Twitter bot that tells you every time someone with a Westminster IP address changes a Wikipedia page (of which there were 5000 edits between 2003 and 2014). Taken together, these sites and platforms are now a powerful source of accountability and a useful short-cut, used to question MPs, and even predict the positions of new Prime Ministers, as you can see with Liz Truss and Rishi Sunak.

We take all this data for granted in 2023 but it’s important to remember what a difference it makes. If you wanted to find out about a vote in Parliament in 1895, or even 1995, you needed to either procure a copy of Hansard, ransack a local library or hope the results had been published in a newspaper. Do all these new sources have any effect on our politicians?

Potentially, all sorts of things could happen now all the data is so easy to find. The hope is that all this data means MPs are less likely to misbehave, for fear of being caught. The more data, the logic runs, the more people are looking, the more likely you’ll get found out. However, the cynical among you could argue the opposite: misbehaving MPs might get better at hiding. There’s been speculation about unintended consequences, too. Could all this data mean MPs to “try to look busy” by turning up very briefly in debates to speak or asking lots of Parliamentary Questions?

Perceptions and effects on politicians

To find out, we commissioned a YouGov poll of 100 MPs and asked them the following: ‘A number of websites now exist that enable the public to easily monitor Parliamentary activity, such as TheyWorkForYou. What effect, if any, do you think websites such as these have on (a) your work and role as an MP (b) MPs in general?’

As a whole, MPs had a roughly equal response between feeling positive about the sites (26%), negative (35%), or believing they had no effect (35%). However, there is good news. MPs do feel they are ‘being watched’. One MP described how data from monitoring sites’ keep MPs on their toes and accountable’.

This is one major finding from our project. MPs are more accountable. After votes, they now share explanations and justifications in Hansard, on Twitter or in the local press. In 2021, for example, Conservative MPs who voted against the government’s COVID-19 lockdown measures and tier system took to Twitter to explain their decisions – making memes themselves both before and after key votes.

MPs also, in some cases, change their behaviour. In 2021, after journalists and campaigners crunched data on MPs and their second jobs, some MPs quietly dropped theirs.

Challenges and concerns in data monitoring

So far, so positive. There is, however, a but. Several, in fact.

First, our poll found monitoring was often targeted at those MPs on the government benches. There was a clear cost of being an MP in the government party. 51% of Conservative MPs saw sites having a negative effect on their own work and 61% on MPs in general. This contrasted with just 19% and 27% respectively for Labour.

Second, it mattered how long you’ve been an MP (and perhaps how safe you are in your seat). Mark Harper argued back in 2006 that:

“TheyWorkForYou.com […] puts Members under incredible pressure. If they do not undertake a volume of work, their performance is criticised—that applies more to new Members than experienced colleagues, who are more relaxed because they have more experience in the House.”

MPs who had been in the Commons for decades, especially from 1997-2009, were far more likely to claim monitoring had “no effect”. This could be because they were safer in their seats or because MPs from 2009 onwards, or 2015, lived through the expenses scandal and Brexit.

Third, gender also plays a role. Female MPs believed far more strongly than their male counterparts that monitoring had a negative impact. This matches what we know about female MPs being frequently subject to greater scrutiny and, for example, less willing to submit expenses claims. The data from TheyWorkForYou was used to famously create a ‘lazy list’ of MPs by the Sun, which then retracted it when it emerged many of those had caring responsibilities. TheyWorkForYou itself has over time published less comparative information, in part due to concerns about gender differences found in the data.

Party and wider contexts

Are there any downsides? One major concern from MPs is how data could be taken out of context. TheyWorkForYou explains the context carefully. Nevertheless, it is true, as the journalist Marie Le Conte put it, ‘sharing screenshots of an MP’s voting history misses out vital pieces of context’. Party and party loyalty are the key to (most) of what happens in the House of Commons. TheyWorkForYou does contain comparisons between MPs and their parties, but doesn’t clearly present the mixed agency of the MP and the party leadership in making voting decisions – something that would be made much easier if the parties published the voting instructions they give to their MPs.

To take a famous example, the Marcus Rashford-inspired vote on low income children and school holiday meals, which led some Conservative MPs to be banned for life from shops in their constituency, was an Opposition Day vote, where Labour forced Conservative MPs to choose between partisan loyalty and hungry children. On climate change legislation, a full 50 Conservative MPs complained in a letter to the chief executive of mySociety in 2021 that ‘misleading’ data ‘misrepresented’ their positions.

However, we need to remember there is substantial usage of TheyWorkForYou from people working in the Houses of Parliament, and they are happy to use voting data themselves. Keir Starmer accused Johnson of ducking a Heathrow vote, whereas Johnson made up entire Parliamentary votes that didn’t exist.

So, overall data sites do make politicians behave better. As ever in politics, it depends, but more data does actually make for better politics.

—

To find out more, read the final report and take a look at our project site at https://whoswatchingwestminster.wordpress.com/

Image: Akira on Unsplash.