-
Have you visited sewagemap.co.uk? If you care about your local rivers, lakes and seas, you might want to take a look at this very graphic display of where sewage is being discharged across England.
If you’re wondering which sense of ‘graphic’ we mean, well, it’s both: first, it’s easy to see the information at a glance. And second, if there’s an active discharge right now, you’ll see an unapologetic poo emoji hovering above your postcode, alongside a brown river to show how the sewage will spread.
So this is a very useful site that makes it easy to understand the current picture around sewage discharges in your area. But if you’re hoping to understand how the situation is changing, you’re out of luck. You can’t see historic data, and that’s because — with the exception of Thames Water — the water companies don’t make it publicly available.
Infuriating, right? Good! Because, thanks to incorporation of our WriteToThem service into the site’s workflow, you can email your MP and let them know why you think this data is important.
And it’s working — we became aware of the site because of the number of messages it refers through to WriteToThem. Curious to know more, we spoke to Alex Lipp, one of the site’s two creators, to find out how it had come about, and what it has achieved.
“It’s kind of a pet project,” Alex explains. “One day I was was playing around and I noticed there was live sewage data being published by water companies.
“Now, the data itself might be quite dull, but I realised that if you could make the link between sewage stats and the rivers that people know and love, they would be more likely to take an interest.
“I work in fresh water science so I could handle the technical side of things; and Johnny, a friend from uni, works in geodata visualisation: he did the design. The set-up isn’t exactly high-tech: it’s literally running off my old laptop, plugged into an Ethernet port at work.
“But it works — it performs well in search engines, and it had 350,000 views last year. We just want to present what would normally be very boring environmental data in a way that engages and makes sense to regular people. Existing sites only showed the points of discharge. It just seemed really obvious to me that what people wanted was information about impacted rivers, which was not included in the other visualisations at all. So I added it in!”
And that historic data would be useful too, right, so people could see whether things were improving or declining?
“Yes! Perhaps surprisingly, Thames Water are much better than other providers, when it comes to data transparency. They published data before all the others, and they have a live API you can query.
“The other water companies make it difficult to query historical data – you generally have to send them an EIR request, because the law technically only requires them to share live data. It’s an oversight of the 2021 Environment Act that water companies aren’t required to publish historical data. So we note on the site where a company doesn’t make it available, and that’s the prompt for people to email their MP and explain why it’s needed.”
Why does the site only cover England? “The Environment Act doesn’t apply to Scotland or Wales – so that’s even more complicated. In Scotland and Wales, water companies are nationalised. They voluntarily publish similar data, but aren’t required to, and it’s in a different format.”
And so – with the site encouraging users to email their MPs, what makes for an effective message? Alex reckons it’s two things:
“Personal stories, and local stories. Just at a fundamental level it feels wrong for sewage to be going into rivers”. So, if you can explain what it means to you — maybe you’d like to take your kids swimming in that river, or you remember it being a clean place to play when you were a kid yourself? “Yes. It’s actually an issue most people can agree on, and there’s strong cross-party support.
“And I think this is a mutable thing that could be fixed…well, lets see if that’s true!”
What impacts has Alex seen so far? “We have a lot of users, and I get a lot of contact from members of the public — anglers, swimmers, campaign groups — who use the data we present and our visualisations to help gauge whether the rivers are safe.
“I’m also fairly confident (but can’t directly prove) that MPs are using the site to get data for discussions that have taken place in Parliament, and are recorded in Hansard. And, most recently, the press team for the recent Dirty Business documentary used the site to get information for their campaign.
“So, we have in general had a distributed impact via widespread use. I confidently think we have contributed to the wider discussion, applying pressure to fix the issue of sewage spills.”
We very much hope so! If you agree that data is the key to understanding the sewage spills issue, and being able to do something about it, head over to sewagemap.co.uk, check out your own postcode and then drop a line to your MP (you’ll find the link in the ‘discharge history’ tab when you click on an icon) to let them know.
-
/RSS FeedTheyWorkForYou aims to improve the quality of UK democracy by making more and better information available to everyone. In previous updates, we’ve expanded coverage to all the UK’s parliaments and brought all the registers of interests together.
Now we’re pulling in data from beyond Parliament to provide richer insights into your representatives. Alex and Julia share our new features:
- Committees and APPG memberships
- Signatures (Early Day Motions and open letters)
- Vote annotations
- Adding context to parliamentary debates
- Improved email alerts for political monitoring
No intro this time: we’re plunging you straight into the audio from the event!
Useful links:
- If you’d prefer to watch the video of this session, it’s on our YouTube channel.
- TheyWorkForYou is here, and TheyWorkForYou Votes is here.
- Donate! It helps us do more of this sort of work! Thank you!
- Subscribe to our updates here (make sure ‘Democracy & Parliaments’ is ticked if that’s what you’re interested in).
- Julia mentions Local Intelligence Hub, which you can play with here.
- Alex mentions a video about how other parliamentary websites get audiences in an age when search engines have become less useful: it’s this one – also available as a podcast here.
Transcript
00:00 Julia Cushion Thanks so much for coming along to this little update. We try and do these every so often.
00:03 You might have come to our one a few months ago, we were talking about TheyWorkForYou Votes, whereas today, we’re telling you about some of the broader stuff we’ve been up to on TheyWorkForYou. We’re calling it “A richer view of Parliament.” (more…)
-
Often, responses published on our Freedom of Information site WhatDoTheyKnow result in newspaper stories, or feed into campaigns or research.
When this happens with one of your own requests, you can add a link to the page. These then appear in the side column, like this:
It’s a great way for other users of the site to see the direct results that come from the simple act of making an FOI request — and now we’ve also added an ‘FOI in Action’ page, where you can see all of them in one place.
Here are five stories that have caught our eye from that page:
- A request for all communications around Eric Trump’s March 2025 visit to Edinburgh allowed the public to see the briefings made to the First Minister of Scotland ahead of their meeting — and resulted in this national news story.
- Minutes from the Ministry of Justice’s Working Group on Unregistered Marriages, acquired via this request, fed into a chapter of research on many aspects of modern marriage, this one being on unregistered Muslim marriages.
- All evidence points to this response being the basis for the New York Times piece [paywalled] that broke the massive story of the government’s £2.4 million expenditure to hide a life-or-death data breach, concerning Afghans who worked with the British forces.
- A 2022 report into misogyny in the British Army was not released until requested and then pursued via the user’s right to an internal review. The user knew of its existence thanks to previous news stories referring to it. The Byline Times reveals the report’s shocking findings in this news story.
- This 2019 report from The Bureau of Investigation looked into public sector adoption of algorithmic and data-driven systems, presciently foreseeing the explosive adoption of AI in our public services. This was based on several requests from a single user.
We’re not far off listing 3,000 citations on WhatDoTheyKnow — and these are just the ones users have added. If your request resulted in a piece of journalism, informed a campaign or fed into research, do add it in. As well as helping to show others what FOI can do, it provides a significant link back to the external site, helping bring it more readers.
—
Image: Peter Lawrence
-
TheyWorkForYou Votes is mySociety’s new platform that provides more information than ever about how MPs (and other elected representatives) have voted.
It’s launching on Monday 19th May, and we’re running an event where you can learn all about it.
Votes in the UK’s Parliaments determine the laws that we all live by, and we want the information about who voted for what to be as accurate, easy to use and easy to understand as possible.
Whether you’re a data whizz who wants to get into the details, or a citizen who wants to know whether your MP has been paying attention to your emails, we think this new service will be helpful to you. Thanks to TheyWorkForYou Votes, we’ve been able to make improvements to our own websites (like TheyWorkForYou and the Local Intelligence Hub), and also, true to our open source principles, we’re making more data available in more formats that you can use and re-use for your own clever tools!
Join us for our launch event at 12pm Monday 19th May to cover both why we publish votes, and what you can get out of the new platform.
Register on Eventbrite now to hear from:
- Louise Crow, mySociety’s Chief Exec
- Dr Ben Worthy, Birkbeck University
- Alex Parsons & Julia Cushion, mySociety’s democracy team
See you in a couple of weeks!
—
Image: UK Parliament (CC BY 2.0)
-
/RSS Feed
It’s March 4 2025, and we’re releasing a bunch of new data on TheyWorkForYou, around each MPs’ financial interests: that’s whether they have second jobs, what donations helped them campaign ahead of the general election, and whether they’ve received gifts such as Taylor Swift tickets.
In the course of assembling this data — with the help of our brilliant team of volunteers — we’ve come to understand exactly what the problems with the current system of reporting are.
If you’re seeing this on the morning of release, we’ll also be launching a report at 1pm today, and you’re welcome to join us. (Don’t worry if you’re too late; we’ll be sharing the video afterwards. Just make sure you’re signed up for our newsletter to be alerted when it’s available).
Don’t forget to check out your own MP, to see who funds them, on TheyWorkForYou.com. And if you have any questions about this project, the data, or MPs’ financial interests in general, send them to us at whofundsthem@mysociety.org.
If you appreciate this type of work, please help us do more of it by making a one-off (or even better, a regular) donation. Thank you!
Transcript
[0:00] Julia: If you’ve ever wondered if your MP has a second job, what donations they received, or if they were one of the ones that got a free Taylor Swift ticket, we’ve got the answers for you. (more…)
-
/RSS Feed
We’ve got updates from Julia on this Parliament’s first Register of Financial interests, showing what second jobs and gifts, etc, MPs have declared; and on the startlingly diminished list of All Party Parliamentary Groups (APPGs).
Meanwhile, Gareth tells us how to get a discount on WhatDoTheyKnow Pro, and we hear from AccessInfo about a new award – the winner will be invited to Madrid to present their work.
Alongside all of that, Myf explains how a WhatDoTheyKnow user harnessed the power of Reddit to verify the responses they were receiving to their FOI requests.
Enjoy!
Links
- Blog post on the Register of Financial Interests spreadsheet; and more details on what it contains
- Blog post about Reddit, WhatDoTheyKnow, and Physician Associates
- Blog post on the diminishing number of APPGs
- AccessInfo Impact Awards
- Full details on how to get a discount on WhatDoTheyKnow Pro by linking to your outcomes
- Our TikTok account
- Our Bluesky account
Music: Chafftop by Blue Dot Sessions.
Transcript
[0:04] Myf: Hello. Thank you very much for tuning in.
[0:07] This is our second monthly collection of news and updates from mySociety, and my name is Myf Nixon. I’m mySociety’s Communications Manager.
[0:15] This month, I’m going to share with you five pieces of news — two from our democracy work, and three from our transparency side. (more…)
-
/RSS FeedIt’s our first ever podcast at mySociety! Heeey how about that?
Myf, our Communications Manager, runs you through all the stuff we’ve been doing at mySociety over the last month. It’s amazing what we manage to fit into just 30 days: you’ll hear about a meeting of Freedom of Information practitioners from around Europe; our new (and evolving) policy on the use of AI; a chat with someone who used the Climate Scorecards tool to springboard into further climate action… oh, and there’s just the small matter of the General Election here in the UK, which involved some crafty tweaking behind the scenes of our sites TheyWorkForYou and WriteToThem.
Links
- TICTeC videos on YouTube
- TICTeC photos on Flickr
- Browse the TICTeC 2024 schedule, find slides etc
- Matthew’s post on updating TheyWorkForYou on election night
- Sign up to get an email whenever your MP speaks or votes
- Democracy resources and our future plans in Alex’s post
- Local Intelligence Hub lets you access and play with data around your constituency
- Matt Stempeck’s summary of the Access to Information meetup
- Our summary of Matt’s summary of the meetup
- Updates from all those ATI projects around Europe
- New in Alaveteli: importing & presenting blog posts; request categories and exploring csvs in Datasette
- Fiona Dyer on how volunteering for Scorecards upped her climate action
- Where to sign up if you fancy volunteering as well
- mySociety’s approach to AI
- Contact us on hello [at] mysociety.org if you have any questions or feedback.
Music: Chafftop by Blue Dot Sessions.
Transcript
0:00
Well, hello and welcome to mySociety’s monthly round-up.
My name is Myf Nixon, Communications Manager at mySociety.
0:11
This is part of an experiment that we’re currently running where we’re trying to talk about our work in new formats, to see if that makes it easier for you to keep up with our news. (more…)
-
CSV is a great format for releasing sets of structured data in response to Freedom of Information requests. Indeed, on WhatDoTheyKnow we’ve seen several thousands of CSVs released.
We’ve recently added the ability to explore CSV files via a Datasette instance. Here’s an example.
Opening the CSV in Datasette makes it easy to explore and analyse it in an interactive website.
If you’re not familiar, Datasette converts the CSV to an sqlite database, which means you can then query the data using SQL.
Alaveteli uses the publicly available lite.datasette.io instance by default, but you can host your own instance and configure it at theme level like we’ve done for WhatDoTheyKnow.
You can see the implementation details at mysociety/alaveteli/#7961.
—
Banner image: Joshua Fuller
-
My colleague Alex has already written about looking forward from this election, so here I am going to look back at the technical work that was involved for the election, and in getting all the new MPs into TheyWorkForYou.
Boundary changes
This election was the first UK Parliament election with boundary changes since 2010. Due to the long-running nature of TheyWorkForYou, which has been around now for over 20 years, this can throw up some interesting challenges. In this particular case, it turned out we were using two different JSON data lists of constituencies – both containing the same data, but one also included the other Parliaments and Assemblies, whilst the other included alternative names for some constituencies. I took the opportunity presented to merge these together and update the bits of code to use the one consolidated dataset, and then added in the 650 new constituencies to the JSON data.
Loading the new constituency data into TheyWorkForYou then threw up another historical problem – the constituency table was still using the very old Latin-1 character set encoding, rather than a more modern encoding such as UTF-8, that almost everything we have uses. This had been fine until now, with even Ynys Môn covered by that encoding, but the new constituency of Montgomeryshire and Glyndŵr contained a letter that Latin-1 could not cope with, leading to a quick emergency upgrade of the table to UTF-8 (thankfully this is a backwards compatible encoding, so worked without issue).
We had already generated data of the new constituencies and loaded these into our lookup service MapIt before Christmas. Ordnance Survey more recently published the official dataset of the boundaries, which we could then import via our usual processes, though even this raised a small issue to be resolved. It turned out in the last data release OS had given the parts of two county council electoral divisions with detached parts (Lightwater, West End and Bisley and Thorpe St Andrew) different identifiers, which they had reverted in their new release, causing our import script to get a bit confused – resolved with a small manual script.
Displaying on TheyWorkForYou
In the period before the election, we knew people would be using our site as a postcode lookup, perhaps to look up their previous MP but perhaps also expecting something useful for the upcoming election, which we wanted to provide, and so we used Democracy Club’s API to show election candidates and link to their WhoCanIVoteFor and WhereDoIVote services. We also displayed your boundary changes using the new constituency data mentioned above.
TheyWorkForYou isn’t just the UK Parliament, though, it also covers the Scottish and Welsh Parliaments, and the Northern Ireland Assembly, so we also had to maintain the provision of that information to people – email alerts for those bodies continued throughout as usual, and the postcode lookup kept showing people their representatives in the devolved nations.
Once the election closed, we automatically updated our messaging, and the next day switched back to our normal behaviour of taking you directly to your MP page in England, and showing you your MP and other representatives elsewhere.
We had a fun issue where some people were getting their new MP, whereas some were getting the old MP – during the period of dissolution, when there are no MPs, we have a configuration flag to enable the site to know it should return the latest result even if it’s not current (you don’t want this all the time, when e.g. an MP has resigned or died), but once new data was being loaded in, one database query was returning results in a random order; fixed by adding some sorting by descending end date.
Election result data
At the last election in 2019, we took a live feed of election results from Democracy Club, who have collected all the candidate information for their Who Can I Vote For service – which all began as the result of a mySociety project back in 2010.
Democracy Club were performing the same service this time, and gratifyingly it was quite a small change to have our 2019 code work with any 2024 changes to the source information (incidentally, there aren’t a lot of narrative doctests in our codebase, but I quite like the one in use there!).
This script would do half the job, of taking in some source data (who has been elected, and including their TheyWorkForYou identifier if they already had one due to being a previous representative of some sort) and amending our source JSON data to add the newly elected representative.
The other half is loading that source data into the TheyWorkForYou database for display on the site. Our normal loading script works fine, but looks through all the source data to see if there have been any changes to take account of. For the election, we don’t need it to do all that, so I tweaked the script to only do the minimal necessary to load in newly created information.
These two scripts were then added to a cron on our server, running every few minutes through the night. I did stay up long enough to check that the first few worked okay, before leaving it to itself from then on. I also set it up to pipe its output to our Slack channel, so people could see it operating:
This also meant as the final few trickle through, it’s popping up reminding us it’s still doing its job:
All the results (bar the one we’re still waiting for) are now committed to the repository, joining all our other open data.
Support TheyWorkForYou and our work
TheyWorkForYou and WriteToThem are run by mySociety, a small UK charity. We’re a very efficient operation and do a lot with a small team; if we had bit more money, we could achieve a lot more.
We want to see a transparent, resilient democracy, with equal access to information, representation and voice for citizens. If you believe in this vision please donate today to enable greater transparency and accountability of the next government.
—
Image: Moritz Kindler
-
The ICO have today announced that they intend to fine the Police Service of Northern Ireland (PSNI) for their accidental release of staff’s personal information in August 2023. This data was released in response to a Freedom of Information request made using WhatDoTheyKnow.
mySociety is a charity; we run WhatDoTheyKnow as a vital tool to help anyone exercise their right to information held by public authorities. We understand the repercussions of a breach like this, which serves to demonstrate that public authorities must be good at dealing with personal information. We welcome the ICO’s emphasis on the importance of robust release processes to ensuring that information that is important to the public interest can be released safely.
We take the responsibilities that come with operating a large platform extremely seriously, especially around the personal data breaches that can occur when authorities’ release processes fail. Following this breach, we’ve undertaken a significant programme of technical and process work to play our part in reducing the risks of this kind of incident.
We’ve developed a new piece of code which analyses spreadsheets as they come in as responses to FOI requests on WhatDoTheyKnow, and holds them for review if they are detected to contain hidden data. The deployment of this code has proven successful and we will be continuing to improve it. In its first three months, this spreadsheet analyser has screened 3,064 files and prevented the release of 21 spreadsheets that have been confirmed to contain data breaches, and 53 which were likely to contain data breaches (around 2% of the files screened in total).
In an ideal world, such measures would not be necessary; we continue to work with authorities making such releases to help them understand the reasons for data breaches, the potential severity of their impact, and how to avoid them.
This blog post was updated at 10:04 on 23 May to correct the figures around the number of spreadsheets screened.
—
Image: Pietro Jeng
