As of this year, mySociety has been working for a better democracy for 20 years.
mySociety was formed in 2003 to explore the ways in which the internet could help people to discover, discuss and participate directly in politics, and whether it could empower them to make changes in society and to the political process itself.
Over the last 20 years, this amazing community of designers, coders, volunteers, partners and funders has created and run digital services that have served millions people each year in the UK and around the world; campaigned for transparent, responsive institutions fit for the 21st century; and supported the incredible persistence, dedication and commitment of people who want to understand and participate in decisions that affect their lives and communities.
Through the power of open source software, mySociety’s technical work has gone on to deliver services around the world, and our international outreach and TICTeC programme has brought together a community of people dedicated to the principled use of technology to improve civic life.
The ideas that the organisation was founded on have spread too: mySociety’s principles and approach inspired a culture of user centred design in the use of the web to deliver government services, with the creation of the Government Digital Service and the Parliamentary Digital Service — input that is still benefitting millions of people as they get something done or find information on government websites every day.
A lot has changed since 2003, but some things haven’t changed. Let’s start with our conviction that the quality of our democratic and political life matters deeply, and that digital services can and should be used to improve it.
In an era of misinformation and mistrust, extending the reach of clear and impartial information about the workings of our democracy is vital. As we face the climate crisis, the decisions we have to make as societies need to be open to participation from all kinds of people, not just the well-connected and well-resourced. Our institutions need to evolve to meet the demands of the moment, find new ways of listening to those they represent, and show that they’re worthy of the trust we place in them.
The big questions of how we will live together through the transitions of the next decades, the questions that politics and democracy ultimately decide, are those that deserve the very best tools and data – made with the people who need them to be beautifully simple, tested and improved, and run responsibly.
The other thing that hasn’t changed is that it takes a lot of commitment to build services that help people at scale – from hugely dedicated and expert staff and volunteers who have given up a significant portion of their lives to making mySociety work, to the thousands of people who’ve responded to one of our calls to do a small task, like gathering a single piece of information.
Throughout the year, we’ll be inviting you to join us in recognising those contributions, reflecting on what’s worked, what’s changed, and looking to the future, and what we’ll need to do to rise to the challenges ahead. Please stay with us as we go through what promises to be a fascinating process.
We're committed to this work — but your support is also much needed as we start to shape our future. Please consider contributing, if you can.Donate now
Join us on Thursday 16th March 2023 for our online event Unlocking civic tech impact: reflections on TICTeC Labs.
Over the last 18 months, we’ve run a programme of Civic Tech activities and events: TICTeC Labs. With thanks to financial support from the National Endowment for Democracy, TICTeC Labs has discussed some of the biggest challenges facing the global civic tech/digital democracy sector. Across six themes, we’ve:
- hosted Civic Tech Surgeries – discussing challenges, existing research and experience, and identifying gaps and needs
- set up Action Lab working groups to take forward ideas generated in the Surgeries and commission work to meet some of these needs
- and funded subgrant projects to produce work to contribute to meeting these challenges.
Hear about the work produced by the subgrant projects and how these have met the needs we identified. Members of the mySociety team, Steering Group and Action Labs will also reflect on how this experimental format worked – the successes and the challenges – and which aspects of it we’ll take forward into future TICTeC activities. There will be opportunities to ask questions about the outputs and the programme as a whole. The session will run from 14:00 to 16:00 GMT and we’d love you to join us live if you can (a recording will also be available shortly afterwards)!
At our Freedom of Information service WhatDoTheyKnow, when faced with requests to remove material, we operate on the principle of removing the minimum amount possible.
Alaveteli, the codebase which underlies WhatDoTheyKnow and a number of other FOI sites around the world, gives moderators a range of options for removing content – with the ability to surgically remove text ranging from individual words and phrases, to individual messages, or even entire request threads. This is useful when we spot misuse of our service, for example.
What we’ve been lacking, up until now, was a way to apply these types of removals to attachments.
Back in the early days of WhatDoTheyKnow, attachments were less common, but now we see many more: there can often be several attachments to one individual message.
Over the last few years, there have been occasions where we’ve had to remove an entire message, which may contain several useful attachments, just because of a small issue with one of them.
We’d then go through an annoying manual process to download the publishable ones, upload them to our file server, and then annotate the request with the links – here’s an example.
Back in 2013, when the original suggestion for enabling finer grained control was raised, the site contained around 400,000 attachments. There are now more than 3,500,000! We don’t remove content often, but at this scale it’s inevitable that we need to intervene now and then.
After a little code cleanup we were able to make individual attachment removal a reality. This allows us much more control over how we balance preserving a historic archive of information released under Access to Information laws, and running the site responsibly and meeting our legal obligations under GDPR.
As an example, let’s imagine that the FOI officer replying to our request inadvertently makes a data breach when releasing some organisation charts in `organisation chart b.pdf`.
Previously we’d have had to have hidden the whole response. Now, we can go into the admin interface and inspect each individual attachment.
We can then set our usual “prominence” value – offering a few options from fully visible to completely hidden – and include a reason for why the content has been hidden. We always seek to run the site transparently and explain any actions taken.
On saving the form, you can see that only the problematic attachment has been removed, with the remainder of the response intact. This saves us considerable time when reviewing and handling material with potential data issues, and keeps as much information published as possible while we do so.
As an extra bonus, since the main body text of emails is also treated as an “attachment” in Alaveteli, we’re now able to hide potentially problematic material there without affecting the attachments we present.
We’ve already used this feature several times to republish material where we’d previously had to hide the entire message due to the technical limitations at the time.
Image: Kenny Eliason
Alaveteli is our platform for running Freedom of Information websites — it underpins WhatDoTheyKnow as well as many other sites around the world. It’s made up of many interconnecting elements, but a key part is the database of public bodies.
On WhatDoTheyKnow, we list over 42,000 public bodies that are subject to FOI and EIR – possibly one of the largest databases of public bodies in the country. Along with their name, we record information like their FOI request email address, publication scheme and disclosure log URLs, and we categorise them so that they’re easier to browse and make requests to.
We often want to add additional insight to help citizens understand more about the public body. This can vary from body to body, from describing the information they hold, what they don’t hold, guidance around how to challenge their poor FOI-handling practices or why we list them when they’re not currently subject to FOI.
We do this via what we call “notes” – a free-text field per authority that admins can update with whatever information is useful for the particular body.
We often find we need to add or update the same note for a group of authorities — for example, we added the same note to all Business Improvement Districts to explain our reasons for listing them.
This can be a major challenge. We list nearly 300 BIDs, and updating each manually, one at a time, would be a several-hour ordeal. An alternative is for our developers to write a quick one-off script to update the text, but that comes with a coordination cost, and can be tricky to work around other text that may be present in the free text field that we want to preserve.
One of the behind-the-scenes ways we manage the categorisation of authorities is through tags. Many of our other records have the ability to add tags too.
This led to the thought that it would be great to be able to apply a note based on a tag, rather than only having one note field per authority. This would allow us to more quickly add useful information to groups of authorities.
Another issue we wanted to solve was to be able to add notes to content other than public bodies, like FOI requests for specific types of information and our category lists. In particular we wanted to be able to celebrate requests that led to particularly useful public interest information being released or having wider impact, and also to add clarification around requests that may be misinterpreted.
We’ve received funding from the Joseph Rowntree Charitable Trust to help support marginalised communities to make more well-formed requests, and to more effectively use the information they obtain to engage with and influence authorities in a way that contributes to fairer decision-making. Thanks to this, we were able to add this underlying capability, which we can use to continue to help users more easily understand how to navigate the complexities of public authorities and what information they hold.
We then needed an interface to allow admins to list and search tags for each type of record that has them, and browse all records that are tagged with a particular tag.
As well as it being generally useful to be able to browse our database through our tags, this gave us a place to add our new functionality for adding a note based on a tag.
While Rails’ polymorphic associations make creating a link between one record and another really easy, they don’t cover creating the link through a free-text tag. Fortunately, it took nothing more than a few lines of code to link up notes that are directly associated with a single record, and notes that are applied via a tag.
Now our public bodies can have notes applied to them and only them, and also notes applied to several authorities based on their tag.
In this example, the Environment Agency has some specific notes about it, but since it’s also a local lighthouse authority, we can apply the relevant notes simply by tagging the authority record and the notes for local lighthouse authorities will automatically get applied. Magic!
We’ve also applied this pattern to information requests. Now, we can add notes to individual requests, like this example that points to another source for obtaining the information…
So far we’ve created 46 notes that get applied via a tag. These notes are applicable to 7,998 requests and 15,283 authorities. Using a rough guess of 30 seconds to manually apply a note to a record, it would take 11,640.5 minutes – 8 full days, or 25 working days – to do so for each of these requests and authorities. This just wouldn’t have been possible before.
This new feature unlocks a whole new avenue for us to support citizens and users that we just wouldn’t have had capacity for otherwise.
Image: Keila Hötzel
A broad range of organisations and individuals are active on climate — and our services can help them to be more effective, from grassroots movements right up to institutional authorities.
Here’s an example of the latter: the Council Climate Plan Scorecards site, for which mySociety provides technical support, was cited in oral evidence to the Scottish Parliament Committee by the Accounts Commission for Scotland.
Commission Member Andrew Burns used data from the site as evidence of inconsistencies across councils in the UK, supporting the Commission’s view that Scottish local councils need to work together more effectively – as reported in the committee transcript (page 9).
The Accounts Commission holds councils and other local government bodies in Scotland to account, and helps them improve, by reporting to the public on their performance.
As the need for cutting emissions becomes ever more pressing, it’s vital that the public can keep an eye on how resources are being allocated and whether authorities are fulfilling their pledges. In November 2021, the Net Zero, Energy and Transport Committee of the Scottish Parliament launched an inquiry into the role of local government and its partners in financing and delivering a net zero Scotland.
The inquiry aims to seek out the main barriers at a local level to Scotland reaching its target of being net zero in emissions by 2045. It will consider what practical steps councils are taking to break them down, in partnership with business, the voluntary sector, and local communities.
It is also considering what role the Scottish Government and its agencies can play in both supporting and, where necessary, challenging local government to work well with its partners to deliver net zero; and how local government can play its part in ensuring a ‘just transition’ to net zero, ie one that is economically and socially fair.
A source of climate data
And that’s how the Scorecards came in useful for the Accounts Commission. They first discovered the website when collating evidence for their publication Scotland’s councils’ approach to addressing climate change.
“The Scorecards Project gave us a specific comparison across many UK local authorities, including some councils in Scotland, as regards their approach to climate action and achieving Net Zero”, said Andrew. “The variation seen in the scorecards confirmed the need for Scotland’s councils’ targets and plans to be scrutinised further.
“Our interest in this area is ongoing, as is the work of the Scottish Parliamentary Committee”.
The Scorecards site and its sister site CAPE show at a glance that there are big differences in the targets that councils have set and their timescales for reaching net zero. With further scrutiny, the Accounts Commission arrived at the conclusion that increased collaboration across councils and with key partners and local communities is needed.
Across Scotland, the Accounts Commission found that 28 councils had declared a climate emergency at the time of the report, with 81% setting a target for the council’s own emissions and 53% a more ambitious target to cover emissions for the whole area. The Accounts Commission report also clearly sets out which years the different councils are aiming to reach net zero by.
And will the next version of the Scorecards, which aims to measure concrete action from councils, be useful as they progress?
Andrew has no doubt: “Absolutely yes, it will be”.
We thought so too! After all, this is an ongoing process for councils everywhere, and the bodies that keep them accountable. We’ll go on putting out the data and we hope to hear many more instances of its use like this.
Image: Mike Newbry
It’s a new year, but let’s not overlook the work we did at the end of the last one. Here’s a quick run-down.
Preparing for a storm of activity
Our partners Climate Emergency UK have been busy recruiting volunteers to mark councils on their climate action for the next phase of the Scorecards project. To help with this ambitious data-gathering exercise, Alex and Struan have been developing a basic web interface for CE UK’s volunteers to use while doing that marking — thus saving them the pain of filling data in across seven different spreadsheets.
If you’d like to experience the joy of not filling in seven spreadsheets, there’s still time to join this year’s volunteers.
Helping people understand how to use our stuff
We’re rolling out some changes to CAPE that will make it easier for non-experts to understand what it is and how to use it — lots more to come on this front. Basically we’d like climate-concerned citizens to be able to land on the site and immediately understand what they can find there, and how it could be useful to them.
And while we’re doing so, we’ve decided it’s called CAPE. No trotting out ‘Climate Action Plan Explorer’ every time we mention it (although of course we’re happy to explain the derivation if people ask). It just feels easier this way.
And making sure they can find it in the first place
Meanwhile we’ve been briefing an agency to buff up our Adwords, which are provided to us free by Google, to be more effective in bringing in the kind of user that those changes will benefit. They’ll be looking at the keywords people type in when they’re interested in the sort of service CAPE provides; and the messaging that can persuade them to click through to the site and learn more.
It’s a relief to be able to hand this over, as keeping up with Google’s various changes in best practice and new features is definitely a full time job in itself, and one we haven’t had much time for in recent years.
Ads can’t do everything though — sometimes you need to talk to your potential audiences! In last month’s notes we mentioned taking part in the What’s The Power In Data event from Trust For London, showing our tools to small London charities. All the sessions, including ours, can now be viewed on YouTube — hopefully that means our insights can have an even more lasting effect.
Embracing the future
Alex met with Faculty AI to put in motion plans for welcoming a data scientist into our fold on a six-week fellowship: we’re keen to ensure the experience is useful for both sides so he’s been putting quite a bit of thought into what machine learning could usefully do for our Climate programme, without taking us on too much of a diversion from our core work.
Proving our impact
We’re continuing to write up nice usages of our services as we find out about them (if they’re feeding into your work in any way, please do drop us a line and let us know). In our latest case study we chat to The Commitment to discover how they’re hoping to secure climate pledges from MPs, with the help of their constituents, ie you!
It’s nice that we’ve been continuing to gather proof of impact, since we met with our new grant manager from the National Lottery Community Fund (NLCF) last month, too. Obviously it’s desirable to be able to show what effect our work is having — and of course we hope that case studies also inspire new directions in others as well.
Mince pies all round
Before we all went off for our holidays, we found out that mySociety had won an award — for Outstanding Contribution to Democratic Change, no less. What a nice way to mark the end of the year! Not only that, but CE UK were also shortlisted in the Democratic Innovation category, for their Scorecards work.
That certainly put a smile on our faces as we set up the out of office reminders and went off to crank up the Crimbo music and put the brussels sprouts on to boil. A good week or two’s holiday was enjoyed by all — and now here we are, refreshed and back for another year of Climate innovation.
Image: Yang Shuo
Last year, mySociety provided technical support to Climate Emergency UK (CE UK) for their Council Climate Scorecards project, which marked every UK local authority’s climate action plan across 78 different areas. The resulting data made clear where plans were adequate, and where there was still work to do. It has informed campaigns, researchers, news stories and councils themselves, as well as feeding into government-level policy.
But plans are one thing, and putting them into action is quite another — not to mention, rather more crucial. So this year, CE UK have set themselves the task of scoring councils on the progress they’ve made on climate action.
To do so, they’ll be using many of the same methods they put to such good effect in the Action Plan Scorecards: they’re currently assembling teams of volunteers (want to get involved? See the end of this post) that they’ll train up with the research skills needed to scrutinise such a huge body of data accurately and with a good understanding of the issues at hand.
Scoring the plans may have seemed like a big task, but at least they are documents which were — to a greater or lesser extent — possible to find online. Action, of course, happens in the real world, so some different methods are required.
CE UK’s methodology for the Action Scorecards can be seen in detail here; it relies not just on the councils’ own reporting, but on a number of different documents and news reports. And where the information can’t easily be found in the public arena, they’ll be submitting Freedom of Information requests.
Of course, this is an area in which we at mySociety have long experience, so our Transparency team is helping out. CE UK will be using our WhatDoTheyKnow Pro service to send the large batches of FOI requests and manage the responses; once the Action Scorecards are launched, the data will, of course, be made public for everyone to access.
With our help, the requests have been refined to provide minimum disruption to busy council officers; at the same time, we hope that these requests, which are all for information that really should be available — energy standards for council-operated housing, for example, or numbers of staff members in climate-related roles — will encourage more proactive publication of data, so that it won’t need to be requested in future years.
We’ve also been able to advise CE UK on forming good FOI requests that will surface the required information.
Because of CE UK’s training strategy, we’re delighted that this knowledge will be passed on to their cohorts of volunteers, effectively informing a new tranche of citizens on how and why to use FOI responsibly. They’ll be helping to classify the responses and compile useful datasets through our early-stage FOI collaboration tool.
We’re proud to be supporting this important work from a climate perspective, too: councils have a crucial role to play in cutting emissions, and there’s an obvious public interest in how they go about doing so — how they allocate public funds, how effective their interventions are, and whether they are on track to reach carbon zero by their self-set deadlines.
All in all, the small team at CE UK have embarked on a massive but vital task. Can it be done? Their approach, as always is: there’s only one way to find out, and that is to try it!
If you’re interested in helping out, there’s still time to apply to be a volunteer — closing date is this Thursday though, so hurry! You’ll be working from home, trained up via online webinars and then helping to collect data as part of this huge effort. Sounds good? More details are here.
On 21 December, we added the National Centre for Circus Arts as a public authority on WhatDoTheyKnow, making it easy for anyone to send them a Freedom of Information request. We could have left it off the site and let our users find the email address themselves, but we didn’t want them to have to jump through hoops.⭕
Joking aside, this is a valid addition to the site — see below for more details about why this college is subject to the FOI Act — and one where we can imagine some interesting requests being submitted.
For example, one could ask for copies of the risk assessments used for the various circus skills taught to students — we imagine these would contain quite a bit of detail.
A procurement list might throw up some items you wouldn’t see from other education institutions, like juggling equipment, unicycles or clown shoes.
Prospective students who want to understand more about their chosen course’s application procedure and admission statistics often submit requests to UCAS, but applications for the National Centre for Circus Arts are handled directly (which makes sense: it would be hard to evidence circus skills on an UCAS form!).
Instead, anyone looking for more information about how the application process works could submit an FOI request directly to the college.
In addition, there is a wide range of requests that can be sent to almost any public authority on matters such as the use of public money, minutes of meetings, policies, letters, emails and contracts. All of these might have uniquely circusy angles when applied to this particular institution!
Some decisions are in-tents
It’s not always easy to work out whether a body is subject to FOI. For some bodies it’s really simple: they’re listed by name in Schedule 1 of the Freedom of Information Act 2000 (or the Scottish equivalent).
The National Centre for Circus Arts isn’t named here, but Schedule 1 does include references to definitions contained in other laws.
In the course of recent work to find more bodies to add to WhatDoTheyKnow, our volunteers have read and interpreted three pieces of legislation and an official register to work out which higher education institutions are subject to FOI.
The upshot of our research is that we consider that all bodies listed on the Office for Students register as “Approved (fee cap)” are subject to FOI. This includes the National Centre for Circus Arts (registry entry). Helpfully, the Centre acknowledges the fact that it is subject to FOI on its website which not all higher education institutions do – kudos for that.
UK higher education institutions walk a tightrope between the public and private sectors. They are subject to rules such as competition law that are normally relevant to private business, whilst at the same time being subject to laws that apply to the public sector such as the equality duty and FOI. It’s quite a lot to juggle. 🤹
Interestingly, the debate about the role of circuses in public life goes back to Roman times where the provision of food and entertainment to the poor were used to gain political power.
WhatDoTheyKnow lists a number of obscure public authorities on the site ranging from the Crown Estate Paving Commission to the Treasure Valuation Committee but the National Centre for Circus Arts is certainly one of the more unusual higher education institutions we list. There can’t be many places where people jump through fire for their graduation (page 13 of the 2023-24 prospectus).
We wish all our users a happy and healthy 2023.
Image: Jonny Gios
We’ve had 109,653 Freedom of Information and Environmental Information Regulations requests made on WhatDoTheyKnow this year. In the run up to the end of 2022 here’s a countdown of 12 of the more unusual ones that have caught our eye this year…
National Highways released 1.25 TB of bat survey data carried out for the Arundel bypass scheme. This was made up of over 115,000 files, that included 786 videos – that’s over 250 hours of footage – 54,570 audio files, 354 spreadsheets and 2,532 images.
We like this because we think it is the largest ever release of information, and as the climate crisis brings urgent challenges for our public institutions to address, access to environmental information will be increasingly valuable to businesses, campaign groups and the general public. You can read more about this release here.
The Medicines and Healthcare Products Regulatory Agency released the nucleotide sequences of the AstraZeneca & Pfizer/BioNTech Covid-19 vaccines used in the UK, after an initial refusal was overturned on review. The response says the companies involved consented to the release.
We liked this because it is a great use of FOI to get such important medical information released and available in the public domain.
Sheffield City Council released the location of every public bin in the city.
We liked this because not only is it really useful information; it is the sort of data that councils should be making freely available to citizens.
Edinburgh City Council released the sewer and cable plans for parts of the city centre.
We liked this because they are chaotically beautiful — not what you’d expect from an underground asset plan.
The Open University released a full textbook in response to a request for the information held on the Early modern Europe: society and culture c.1500-1780 module(s).
We liked this because it’s not not often that you see full textbooks being released.
7: Tower Bridge
The City of London Corporation released a list of the past Tower Bridge lift dates, times, and vessel names from the start of 2022 until now.
We liked this because it generated a long and sometimes amusing conversation on Reddit.
6: War memorabilia
The Ministry of Defence released a WW2 medal card.
We liked this because we’ve never seen something like this obtained by FOI before.
5: Honours board
The Charter Trustees of the Town of Margate released their Freedom of the Town list in a more unusual format.
We liked this because the information released was a photo of a painted wooden board. This is, after all, still a form of recorded information — and a nice permanent one.
4: Seaside nuisance
Brighton & Hove City Council released a copy of all of the bye-laws that apply to the seafront.
We liked this because of the phrase: “no annoying gramophones on the beach”, which may be a slightly outdated view of the worst possible noise nuisance.
3: Big cats
North Wales police released the details of big cat sightings in 2021.
We liked this because the information disclosed in the request was used for a number of news articles in Wales.
2: Library books
One of our users has been doing some research into the top 25 books borrowed from libraries in 2021; here’s an example of one of them.
We liked this because it’s fascinating to see what books people are choosing to read, and how this varies between different areas of the country.
1: Trains galore
And finally, here at WhatDoTheyKnow the team are all big fans of trains. This means we tend to notice the more interesting disclosures on train related topics. Here are some of our favourites from this year:
a. Network Rail released the engineering drawings that were produced during the construction of the London & South Western Railway’s station at Branksome, near Bournemouth in the mid to late 1800.
We liked this because the drawings are beautifully crafted and not something that we get to see very often.
b. Northern Trains Limited released the .wav file of the two jingles used for their station automated announcements system.
We liked this because it’s unusual for audio files to be released and we’ve found so many uses for this!
c. London North Eastern Railway Limited (LNER) released some information about the voiceover artists used for the automated announcements on the Class 800 and Class 801 fleet of trains
We like this because it shows that FOI responses don’t need to be formal or complicated; they just provide, where possible, the information that the request-maker has asked for. LNER is particularly good at this.
d. Transport for London released 3D station layout drawings for the Elizabeth line.
We liked this because it’s really interesting to see how the new stations on the Elizabeth line have been designed, and how the layout works with their surroundings.
We hope you enjoyed 12(ish) of our favourite FOI requests from this year.
Yes, once again the year is drawing to a close, and it’s time to look back on everything we’ve achieved over the past 12 months, in the 2022 annual review.
Our work this year has had impacts in all kinds of places, from kids’ school dinners to a prison newspaper; from access to information across Europe, to research that helped shape thinking around FOI and data that extended our knowledge of climate action at the local level. Our TICTeC Labs programme created solutions that feed back into the civic tech sector; and SocietyWorks was busy innovating through the year too, making sure that your council’s services work more smoothly.
That’s just a taste of what you’ll find in the annual review, but you can read it all for yourself here; or, if you prefer dip into the WhatDoTheyKnow transparency report or SocietyWorks’ own review here.
All the best of the season to our friends and followers — and we’ll see you in the New Year!