Last month, we sent some of our newsletter subscribers an email to say that we’d noticed they weren’t opening our newsletters and, unless we heard otherwise, we’d be removing them from our mailing list.
A number of people replied to that email to say, ‘Actually, I do open your emails — I just have image loading turned off so you don’t track my activity’. This was a welcome reminder to examine our own practices.
Such responses triggered a conversation within mySociety, resulting in a session at our team meeting last week, with the upshot being a decision to turn off personally identifiable tracking on all our newsletters.
We’ll be able to see how many people visit our blog posts from the newsletter, but that’s all — we won’t be able to associate visits with particular accounts, nor will we know how many of the views are from repeat visits rather than different users.
This decision reflects mySociety’s position as an organisation concerned about matters of privacy and also feels like it’s part of a wider movement within our sector: we’re not the only ones having this conversation.
But why did you track opens at all?
The obvious question, given this statement, is why mySociety were tracking activity in the first place.
Partly tracking was in place to help manage the financial cost of our newsletters. We use Mailchimp, whose costing structure is based on how many subscribers you have at any one time: you pay more for each subscriber even if they are not active.
Sending an email to people who hadn’t opened — or who appeared not to have opened — our emails for over a year was a way of cutting the costs of holding a large database on Mailchimp, and you can only do that if you know who they are.
For purposes of understanding the impact of our communications, too, it is helpful to be able to see how many people have opened a mailout, what stories have been clicked on, and which have not.
However, this can be done in a way that doesn’t intrude on the users’ privacy – and, as mySociety team members posited during our team meeting, a cost saving on our side doesn’t justify infringing on privacy.
Mailchimp’s defaults assume that you want as much information in as granular a form as possible, but they do give you the option to turn tracking off.
As far as we can see, one has to remember to do this each time a mailout is sent, but we’ll also be contacting Mailchimp to ask them if the relevant boxes could be unchecked by default, or managed at a global level.
So from now on
After our team discussion, we’ve made the immediate decision to turn all tracking off on our newsletters. Thank you if you were one of the people whose feedback spurred us to have this conversation and arrive at this decision.
And if you’d like to subscribe to our newsletters, you can do so here.
Image: Jehyun Sung
If you’ve had a look at our annual report for 2019 you’ll know that we’re a busy bunch at mySociety, keeping lots of useful civic services running and talking about our work on an almost daily basis.
In 2020 we’re going to be doing something a bit different.
You’ll still hear from us regularly through our blogs and research and conference, but we’re going to be talking about one thing above all else – the climate crisis.
We’ll still talk about democracy; but more than likely we’ll be considering how participatory and deliberative approaches can be useful in finding consensus on the difficult decisions we’ll all need to take to avoid the worst climate impacts. And thanks to your contributions towards the successful crowdfunder for TheyWorkForYou, we’ll be able — along with other much-needed improvements and updates — to help you hold the new parliament to account on how they respond to the climate emergency.
You’ll still hear from us on transparency; we’ll be helping people make the most of WhatDoTheyKnow to request information from public bodies on how they are responding to the crisis, and we’ll be looking at how we might apply our long experience of improving access to public information to similar private sector services in areas like pensions and investments – where divestment from fossil fuels is urgently needed.
When we refer to community, and especially our work with FixMyStreet, we’ll be underlining how important it will be to support local democracy and help create resilient flourishing communities if we’re to mitigate how our changing climate will hit the least well off in society.
One focus, one reason
We are doing this for one simple reason – there really is not a more important issue facing our society today.
We can’t address the climate crisis without also addressing the parallel democratic crisis we face in many countries around the world, where lies, deceit and fake news have become normal paths to power.
We can’t solve issues like climate change without also addressing the lack of equality and fairness in society, where those with the least power and influence will be affected the most.
And we can’t avoid the worst impacts without building and living with strong and resilient communities where every citizen can play their part.
So we’ll be exploring what small role we might be able to play at mySociety — both improving our environmental impacts internally, and examining how we align our current and future work with the need to tackle the climate crisis. And alongside this you’ll still be able to report a pothole on FixMyStreet, or follow your MP on TheyWorkForYou on every other topic beyond the climate as usual.
We’d encourage all our friends and colleagues in civil society, government and the private sector to consider what role they might play themselves both as individuals and through their organisations – and we hope you’ll also share your plans and we can learn more from each other in the year ahead.
Before the summer we began the search for new Trustees and Non-Executive Directors to join our charitable and commercial boards. As we enter Autumn I’m very pleased to be able to share the fruits of our search with a clutch of lovely and talented new board members.
For our mySociety Ltd board which oversees our commercial and product development work we are pleased to welcome three new Directors:
Steve is Director of Strategy and Innovation at the Government of Jersey, turning emerging theory of open policy making, participatory design and digital government into practice. Previously at Stockport Council, and as Chair of the GM Connect programme for the Greater Manchester Combined Authority he is a leading thinker on all things Local Government.
Ade is no stranger to mySociety and has been a supporter of our work since her time leading the Government Digital Service’s Data Infrastructure programme. Currently a data strategist at Cloudera Fast Forward Labs, she will help us consider the wider application and impact of ethical data use across our range of services.
Cam is a Director of Green Angel Syndicate, a network of angel investors funding early stage technology businesses which fight against the climate crisis, and for the sustainable use of resources. He’ll help advise on the further development of our SaaS products, with a particular interest in shaping our response to climate change.
We welcome two new Trustees to our charity board for UK Citizens Online Democracy, who are responsible for the strategic direction and sustainability of the whole organisation:
Julia is a Senior Transparency Advisor at the Westminster Foundation for Democracy, advising parliaments and political parties on accountability, transparency and the rule of law. She works closely with the Open Government Partnership (OGP) on enhancing the role of civil society in public policy making and government oversight.
Kate is currently Private Secretary to the Chief Executive of the Civil Service & Cabinet Office Permanent Secretary, where she works on EU Exit and Civil Service HR. Previously working in the Cities and Local Growth Unit at BEIS and before that at the Design Council and CABE, she brings invaluable insights around Government and public policy.
In addition to Owen Blacker who stepped down after an amazing 17 years of involvement with mySociety, we also say goodbye as a Trustee to Nanjira Sambuli. Both stepped down at our AGM in July. Immense thanks to both for their support and service.
Our search for a new Chair of UKCOD will continue over the next couple of months as we speak to candidates, and we expect to make additional board appointments during 2020.
Photo header image by Jakub Kapusnak
After over five years of active development we have decided to pause work on the EveryPolitician project for the foreseeable future.
In this post we’ll outline where we are leaving things, how you can make use of the data that does exist, and how you might be able to help migrate or transfer some of what we’ve collected over to services like Wikidata.
What’s in place today
The EveryPolitician project is, as its name suggests, based on the simple idea to gather accurate and up-to-date data on every politician in the world, collated and shared in a consistent format for free download and use by researchers, democracy projects, campaigners and individual citizens.
Over the course of the project we have gathered, structured and shared data on 78,382 politicians from 233 countries and territories presented on EveryPolitician.org via hundreds of scrapers run on morph.io and hosted on GitHub, producing the data on everypolitician-data.
Mostly the data covers the main chambers of recent parliaments around the world, but it also includes thousands of entries for previous parliaments, in some cases going back decades.
This has been a sizeable undertaking, involving a handful of very talented developers and colleagues within mySociety, as well as contributions from dozens of other organisations and individuals, many of whom make use of the data within their own projects.
The reality is that this work is hugely time consuming, complex and requires not just expert knowledge but a commitment to go deep into the intricacies of parliamentary data in order to make it comprehensible to a wider group of users. And looking to the next couple of years this task is only ever going to increase in complexity — too much for one underfunded organisation.
We therefore intend to freeze the current data as it currently stands, and it will continue to be available for download and reuse. We just can no longer commit to keeping this data up to date.
Always playing catch up
The challenge with data projects like EveryPolitician, beyond the complexity of understanding the structures and relationships within hundreds of individual parliaments (every parliament is an edge case in some way), is that the data is always steadily going out of date.
Across the world’s national parliaments there is an election somewhere roughly once a week, and that’s often when parliaments choose to update their websites, sometimes breaking our scrapers and changing the format of the data. Throughout the life of a parliament you might expect a few percent of MPs to change, sometimes more in different systems, so keeping on top of those individual changes is a sizeable task – especially where errors or duplications occur.
In addition to managing the hundreds of scrapers, we also included data from other sources — increasingly from Wikidata. Over the past 18 months we’ve been attempting to migrate more and more of what we’ve learned on EveryPolitician over to Wikidata via the WikiProject every politician.
Where the project goes next
EveryPolitician was built on the many years of work we had already delivered in this area, through PopIt, Poplus and working with Popolo. We knew what was needed, what worked and what didn’t.
We saw the potential to create an Open Corporates for political data, and hoped that EveryPolitician would be able to attract grant funding to grow, and potentially develop appropriate commercial services in support.
However, after five years of significant investment we just don’t have the funding to continue this work on our own.
In time we hope to be able to continue to contribute again to the wider availability of political data, and with hindsight it’s clear that Wikidata should be the natural global home for this type of data – benefitting from much greater reach, the contribution of motivated individuals in each country, and from the wider Wiki community.
As part of our contribution to Wikidata, we’ve created numerous tools to support the cross-referencing, verification, and supported update of data between EveryPolitician and the Wikiproject. This is still something of a work in progress, but we see it as a key way that others might contribute and take on aspects of the project in the future.
In the meantime we hope that many people continue to make use of the wealth of data that’s already been collected.
If you have a specific interest in a country, group of legislatures or some other combination, perhaps you can consider adding the kind of data that EveryPolitician has collected to Wikidata. We have no further resources to devote to this work; however if you do have an interest in taking some of this on then we will try to advise what options might best suit.
Image: Jelle van Leest
For a charity like mySociety the appointment of a new Chair and Trustees is an important opportunity to bring new leadership and energy to our boards and to the organisation as a whole.
Over the next three months we are looking to appoint a clutch of new board members for UK Citizens Online Democracy (our charity board) and for mySociety Ltd (our commercial board) as well as the uniquely important appointment of a new charity board Chair.
After 15 years of service, our Chairperson, James Cronin will be stepping down at the end of the year.
James was one of the original founding group of volunteers who established mySociety. He has been a Trustee and Chair of UK Citizens Online Democracy and a Non-Executive Director of mySociety Ltd for most of their existence. James has been critical to the success of mySociety over the years and continues to be a mentor for me and the team.
Before then another of our very long-standing Trustees, Owen Blacker, will be stepping down at our AGM in July. Owen was also one of the original cohort of mySociety volunteers and has similarly served for over 15 years. We’ll miss Owen’s technical expertise, passionate defence of privacy rights and support of LGBT rights, and our board will be diminished as a result.
Strengthening our boards
So with this changing of the guard we have the opportunity to appoint new leadership and advisors to our boards and we hope you might consider applying.
A strong and diverse board is essential to ensure that our organisation is well-run, solvent, operating within the law, and making the right strategic decisions.
Three years ago we went through a similar exercise, appointing new Trustees Rachel Rank, Tony Burton and Nanjira Sambuli. We also established our commercial board with Jonathan Flowers, Tim Hunt and Anno Mitchell all being appointed. We also very recently appointed Mandy Merron as a new Trustee following a number of months in which she acted as an independent advisor to our Finance Committee.
So over the past few years we’ve already brought in new voices and opinions, improved our gender balance, and moved to fixed term appointments with a maximum of two consecutive three year terms and we’re keen to continue this progress through the next phase of our development.
Applications and appointments
For our charity board we are looking for new Trustees who can bring additional legal and policy experience. Specifically we need help in understanding the impact of any legal challenges we might face, especially in relation to data protection and Freedom of Information. We also need advice on how best to shape and influence public policy and consultations, especially where they relate to Democratic Participation, Place and Communities or International Development.
Trustees should have experience of working with the charitable or not for profit sector and must have a passion for our goals and experience of pursuing them.
For our commercial board we would like to appoint new Non-Executive Directors with SaaS development experience and service provision to government and the public sector. We would especially welcome experience in developing and scaling software as a service products, and working on, or commissioning, service provision within Local and Central Government; especially in relation to Democratic Participation, or Place and Communities.
It would be beneficial for Directors to have experience of working with the charitable or not for profit sector, but this is not essential. They of course must have a passion for our goals and experience of pursuing them.
We’re keen to continue to diversify the make-up of our boards and are particularly keen to see applications from women and underrepresented groups.
A unique new Chair
Appointing a new Chair is an exceptional opportunity to join a pretty amazing and committed, impactful charity. You’ll work with your fellow Trustees and the senior management of mySociety to help shape and direct our growth and development over the next few years.
We want to recruit a Chair who represents the best of what mySociety has to offer, who can help extend our ambition, hold us to account and champion our cause. They will need to be comfortable representing UKCOD and mySociety publicly and be prepared to speak on our behalf from time to time.
They will help us continue to diversify our board and team across a range of occupational and personal backgrounds, ensuring we deliver on our support of equal opportunities in ways that reflect the range of our users and work programmes.
They will be able to use their experience and connections to contribute towards helping more people to be active citizens and improving access to and participation in democratic processes around the world.
The closing date for all applications is 10.00am, Monday 13th May.
If our mission motivates you in the way it motivates our staff and volunteers, we would love to hear from you and you can find full details for how to apply here.
Image: Xiang Hu
In our previous post, we identified WhatDoTheyKnow’s current need for sources of funding.
But WhatDoTheyKnow also needs more volunteers to join the team. Since the site’s launch, it’s always depended on a highly-motivated, active group of administrators who work to keep it running.
At mySociety, we’re very grateful for the work the volunteers do; for their part, they tell us that they find the work rewarding and interesting — but we’re always aware that we can’t, and shouldn’t, demand too much from them. The more volunteers we can recruit, of course, the less the workload will be for everyone.
We’ve identified three general areas in which volunteer help would be very welcome, and if you think you’d fit in to any of these, we’d love to hear from you.
- interested in FOI and transparency
- happy to work remotely but as part of a team, communicating mainly via email
- able to dedicate a minimum of a few hours per week to helping run the site
Each of our volunteer administrators give their time freely and are the only reason we can run the service day to day at all.
Being a volunteer is both rewarding but also challenging, as each juggles their day jobs and home lives. So the more volunteers we have, the more we can spread the workload between them.
If you have a specific interest in FOI or transparency, or indeed you’d just like to help support a well used civic tech service then we’d love to hear from you. There is always a diverse range of jobs and tasks needing to be done, even if you can only help a couple of hours a week. We all work from home and communicate via email and other online tools.
If you can help us a volunteer the first thing to do is to write to the team introducing yourself and letting us know about your relevant skills, experience and interests.
- a law student or professional who can offer expertise in the day-to-day running of the site; or
- a legal firm or chambers who could offer legal advice on an ad hoc, pro bono basis
Volunteers with legal backgrounds We take our legal and moral responsibilities in running WhatDoTheyKnow very seriously and we always welcome volunteers with experience of legal matters. Some of the legal aspects of running the site are handled routinely on a day to day basis by the admin team.
They may, for example, remove correspondence which could give rise to claims of defamation, or where personal data is disclosed by an authority mistakenly and they consider continued publication to be unwarranted.
The legal challenges thrown up by operating our service are varied and interesting. Joining us could be an opportunity for someone to get some hands on experience of modern media law, or for a more experienced individual, to provide some occasional advice and guidance on more challenging matters.
We often find ourselves balancing claims that material published on our site could aid criminals or terrorists, or could cause harm in other ways, and we do our best to weigh, and balance, such claims against the public interest in making the material available.
As material published on our website may have been used to support news articles, academic research, questions from elected representatives, and actions by campaign groups or individuals it’s important we don’t remove correspondence lightly and that we’re in a position to stand up, where necessary, to powerful people and institutions.
Legal firms that can offer advice As from time to time there are cases which are more complicated, we would like to build a relationship with a legal firm or chambers that can advise us on an ad hoc basis on defamation, privacy (misuse of private information) and data protection.
The ability to advise on copyright law and harassment law would also be an advantage. And we also on very rare occasions may need help as to how to respond to the threat of litigation.
Could you offer help in this area? Please do get in touch to discuss getting involved.
- a committed, organised, empathetic person who could volunteer a few hours (working from home) a week
In our previous post we mentioned that we’d ideally secure funding for an administrator who could handle our user support mail and deal with routine but potentially complex and time-sensitive tasks such as GDPR-based requests.
While we seek funding for this role, would you be willing to fill it on a voluntary basis? Please get in touch.
Lots to help with
So in summary, what we need to keep WhatDoTheyKnow running is money, volunteer help, and legal support. If you can help with any of these, or have some ideas of leads we might be able to follow, please do get in touch. It also helps to share this post with your networks!
Alternatively, you can help out with a donation large or small — every little helps.Donate now
Image: CC0 Public Domain
If you appreciate our Freedom of Information site WhatDoTheyKnow, then you’d probably like to know that it’ll be around for the foreseeable future.
That will only be a certainty if we can secure new volunteers across a broad range of areas; or new sources of funding for the site — or ideally, both! WhatDoTheyKnow is a free service, run on a charitable basis by a currently very thinly-stretched team of volunteers.
We’ve identified four areas in which we need help:
- Legal support
- Admin support
- Additional volunteers
In this post we’ll be looking at the first of those; and in our next post we’ll talk more about various volunteer roles and ways of helping the site to operate. If you think you might be able to assist in any of these categories, please do read on.
WhatDoTheyKnow.com is a Freedom of Information service used by millions of people each year, from journalists and campaigners to ordinary people trying to navigate bureaucracy.
We recently celebrated the 500,000th request made via WhatDoTheyKnow, and also the site’s tenth anniversary. Each month, it’s visited by over half a million people and over 2,500 requests are made via the site. It’s a success story — an example of civic tech that runs at scale, has lasted, and has had an impact to match.
One of the ways that mySociety has always tried to make change in the world is by building things on the web that show how the world could be better. In the case of WhatDoTheyKnow, we asked ‘What would it be like if everyone felt able to ask questions of those with power, and get answers?’.
Our position as a small digital charity allows us to be bold in the things we build, to act as critical friends to institutions of power, and to design for the citizen. In practical terms, it also allows us to ask forgiveness, not permission — without that freedom, many of our sites and ideas would never have seen the light of day. That we have had success with WhatDoTheyKnow is wonderful, but leads us to ask a new question: how can we, again as a small digital charity, ensure its future?
It’s always been a necessary engineering principle for us as software developers, trying to build sites that have impact, to require as little ongoing intervention as possible. However, technology isn’t and shouldn’t be everything — a site that runs on the scale of WhatDoTheyKnow can’t run without different kinds of support. In running WhatDoTheyKnow, we’ve learned that digital institutions, like other institutions, are shaped by people. The people who originally designed them, for certain, but also those who pick up the torch, who continue to make the day-to-day decisions that keep the institution relevant, humane, responsive and responsible. It’s this support that distinguishes brilliant technical ideas that flame out from those that grow and become so embedded in our culture that they start to fundamentally change the way the world works.
A vital part of that support for WhatDoTheyKnow comes from a handful of volunteers who run the service day to day. These volunteers handle everything from simple user support to advising on complex points of law and policy.
Now the success of the site means that they need help on the front line. We’re always on the lookout for new volunteers — but there are also other things we need to ensure that WhatDoTheyKnow is around for the next ten years and another half a million requests.
We need funding for admin
It’s becoming increasingly urgent that we recruit a part-time assistant, responding to our users’ queries via email. This person would help our amazing team of volunteers support people in all walks of life as they go through the process of requesting information from public authorities.
They’d help to deal with the diverse day to day user enquiries, make sure we meet important deadlines in handling time-sensitive issues like GDPR-based requests, and share feedback to improve our user and volunteer experience over time. The cost of a paid part-time support role would be at least £15k per year.
We don’t currently have any funding for this increasingly essential role, nor indeed any direct funding for WhatDoTheyKnow itself.
We need funding for development
Although WhatDoTheyKnow hasn’t changed fundamentally over the years, there are always ways in which we could improve it — a recent example is our work to start developing features for journalists and other professional users.
The site does also require a certain amount of ongoing development work in order to keep it running at the scale it does. That includes making sure it gets the latest security updates, and dealing with new problems that arise as it grows, such as the fact that the more popular it becomes, the more rewarding a target it becomes for spammers.
Work to maintain Alaveteli, the code that runs WhatDoTheyKnow, also supports the community of Freedom of Information campaigners, journalists and citizens around the world that use Alaveteli-based services to exercise their right to know in 26 countries.
We don’t currently have any financial support for developers to support and maintain WhatDoTheyKnow and it’s important we find at least project funding of £30,000 to £40,000 a year, if not general unrestricted financial support from new funders.
Funding to date
We should acknowledge the funding which has allowed us to run thus far, and for which we are of course very grateful. A grant from the Joseph Rowntree Trust originally got WhatDoTheyKnow off the ground; Google’s Digital News Initiative supported the development of Alaveteli Professional, and unrestricted support from a number of funders ensured that mySociety has been able to continue paying their developers to work on the project. It’s perhaps worth noting that this support has, to date, always sustained development rather than administration.
We do have a revenue stream through WhatDoTheyKnow Pro, our FOI service for professionals such as journalists, but as yet this is very modest. As the service develops, we hope that this may one day become part of the framework that helps sustain WhatDoTheyKnow, but we’re some way from that at this point in time.
Can you help identify a fund or donor who might be willing to cover the costs we’ve identified above for the next year or two? Please get in touch.
Or perhaps we can be more imaginative. One model we’ve seen used to good effect by other sites run on our FOI platform Alaveteli has inspired us to conceive of a similar (but not identical) set-up for WhatDoTheyKnow. This would involve sponsorship from one or more reputable media organisations who could make use of WhatDoTheyKnow for their own journalistic investigations, while also gaining the benefit of recognition across the site.
Of course, that’s just one idea — there must be many other possible models for supporting the site and we’d love to hear any ideas you have in the comments below.
Now you might like to read our second post, in which we’ll be talking about ways you might be able to help with time, rather than money.
You can help out, with a donation large or small — every little helps.Donate now
Image: CC0 Public Domain
mySociety has been a leading light in the Civic Tech movement since 2003, helping to shape and define the sector and building services used by over ten million people each year in over 40 countries worldwide.
During this time Civic Tech has grown and matured; delivering plenty of impact, but also hitting numerous stumbling blocks along the way. In mySociety’s fifteenth year we’re taking stock of the best way to achieve our long term goals and ambitions.
So today at the Code for All summit, Heroes of Tech in Bucharest, we announced our intention to become an affiliate member of the Code for All network.
mySociety and Code for All both recognise the power of working in partnership, of being honest and self-critical about the effects of our work, of working openly and transparently and seeking the best outcomes for citizens in their dealings with governments and the public sector.
Code for All is probably best known for Code for America, which set out the blueprint for a civic tech group working closely with government. Now that Code for All is growing beyond these early roots to become more than a collection of individual ‘Code For’ organisations it is broadening its own perspective to include more groups outside of government, we feel that this is a good time for mySociety to deepen our collaboration within this growing movement.
Every success we’ve had has come from working well with our partners. Each of our services internationally is run by a local partner with mySociety providing development help and support and the benefit of our service development and research experience.
In recent months through our Democratic Commons project we’ve worked with numerous Code for All partners, including CodeForPakistan, OpenUp, CodeForJapan, ePanstwo, G0v and others. Those of you who have attended our TICTeC conferences will know that they attract many members of the Code for All network as participants each year.
What mySociety can bring to the network is a unique international aspect, a commitment to collaborate and combine our efforts on cooperative democratic projects, a willingness to more widely share our research and evidence building experience and a desire to improve the positive impact of our work.
We would benefit from more of our work being seen as truly collaborative, and are no strangers to the challenges of seeking grant and project funding and the importance of working together to achieve this.
With all the challenges facing democracy — governments struggling under austerity; fake news and dark money distorting the truth; a slow burn environmental catastrophe playing out around us; hard won rights and the norms of a fair and just society under threat — now more than ever feels like an important time to be working more closely together.
So we’re excited by the opportunities that this timely partnership will deliver and keen to see where this takes us.
Keep it in the Community, or KIITC for short (pronounced ‘kitsy’), currently shows a snapshot of over 5,000 England-wide registered ACVs. We hope that researchers of all sorts will use it as a resource.
A nationwide picture
Since the introduction of the Localism Act, community groups up and down England have been taking advantage of the opportunities it affords to nominate places and spaces as Assets of Community Value.
And while the Act also requires local authorities to maintain and publish a register of such Assets, one thing has been missing: the ability to see a picture of how these rights are being used across the country as a whole.
Do some regions contain substantially more ACVs than the norm? Are more applications rejected in some places than others? And just how many Assets of Community Value have been identified to date?
We believe this sort of inquiry is essential if we’re to understand the efficacy of the Act and whether it’s achieving what it was designed to do, and now KIITC makes that possible.
Scaled back ambitions
As you may recall from our previous posts, our original ambition was not only to gather together and publish all the existing data from the ACV records of England’s many councils, but also to invite community groups to submit new applications to their local authorities, directly through the website. From long experience in similar projects, however, we knew that there would be challenges, and indeed this turned out to be the case.
While all councils are legally required to display this data, they’re not given any guidance as to the format in which it should be displayed, and the huge variety of different formats, together with the frequency with which the location of the files in which they are published change, make an automated approach almost impossible, especially within a resource- and time-constrained project.
These factors make it hard to sustain what is really the only practicable approach for a project with limited funding — the automated ‘scraping’ of websites. Scraping sends a small script out onto the web to regularly check whether new data has been added to a location; in this case, each council’s ACV register. A piece of code can retrieve the data and put it into the right format to be republished on your own site: it’s how we published Parliamentary debates each day on TheyWorkForYou.com for many years, for example.
But this is really only a practical option when everyone is publishing in one of a few standardised formats.
What would be a solution? Well, in an ideal world every authority would be putting their records out as lovely, consistent data. But we understand that this is rather an unrealistic expectation.
A data snapshot
Faced with these difficulties, we met with Power to Change, who originally funded the work. We were in agreement that, sad though it was to set aside the other features of the project, there was still great value in collecting a snapshot of all ACV data across England.
So, ambitions of scrapers accordingly readjusted, we manually entered all the relevant councils’ registered ACVs and uploaded them to the site. Please note that some data may not be 100% accurate; it all depends on what we were able to collect at the time.
The most comical result of this is that assets where we don’t have a precise postcode for location may appear to be floating in the middle of the ocean… but these are the minority. And it’s worth remembering that the dataset as a whole is the best available right now.
Of course, the data will quickly become out of date, but we believe that this unprecedented collection will have many uses, nonetheless.
For the moment, we won’t be updating the site with future data, unless further funding becomes available for us to do so. We’ve also put plans for community group submissions on the back burner for now, aware that if we are to provide this service, we need to better what is already out there on the councils’ own websites.
When time allows, we’d like to explore ways to encourage citizens to help keep the site up to date, allowing them to update data that has already been imported, and consider how they might suggest new ACVs to their relevant council via the website.
As part of that vision we’ll need to reach out to councils to demonstrate how we have visualised the data, and work with them to participate.
Open data projects such as these rely on identifying useful and practical ways for public sector organisations to more easily release data in a common and consistent format so that others can make best use of the information — a task that has much wider implications for all sorts of niche datasets such as this.
If you’d like to find out more please get in touch with us at email@example.com: we’d be keen to hear from you if you’d like to help us trial managing your own data on the service.
So we’re pleased to announce the (quiet) beta launch of our latest little site, Keep It In The Community, which we hope will become an England-wide register of Assets of Community Value (ACVs).
The Localism Act 2011 was introduced with a great hope. Its provision for giving groups the right to bid on buildings or land that contribute to community life would allow the protection these assets, potentially taking them into direct community control should they come up for sale.
Sadly, as currently implemented, the law hasn’t yet delivered on that promise.
In Scotland, the legislation comes with an actual right to buy, but in England, that’s not the case, and with developers finding ways around the legislation more often than not, often the best the Act can bring about is the delay of an inevitable change of control. For the moment we’re not expecting the legislation to be given any more teeth.
With Keep It In The Community we intend to at least help support a greater takeup of registration by local communities.
In yet another project built on the flexible FixMyStreet Platform, Keep It In The Community has three main roles:
1. We’re gathering together existing asset listings from the 300+ English councils who hold them, to provide a single synchronised and complete record of all listed and nominated ACVs.
2. We’re providing a straightforward route for established community groups to nominate new ACVs in their community.
3. We’ll allow community members to provide more details, photographs, and useful anecdotes about each registered asset, beyond that required by the legal listing process.
Plans for the summer
So far we have data from around 20 local authorities on the live service, with another 50 or so due to be added over the next few weeks. The remaining councils will be added over the summer. All the data is drawn directly from each local authority and as new assets are nominated or their status changes we’ll update their status on Keep It In The Community.
Whilst we complete final testing we’re restricting the ability of community groups to nominate assets, but hope to fully switch that on shortly, once more of the existing assets are displayed on the site.
The initial process for connecting each council listing is fairly low tech, relying on the scraping of a commonly formatted spreadsheet hosted on each council website.
So for the moment. there will still be a fair amount of manual tweaking to keep things in sync. This is one of those important elements that will be fine to manage when the service is starting out, but may start to creak further down the line if it becomes well used – a classic ‘known known’ issue we’ll need to keep on top of.
Over the summer we’ll be working with a representative set of community groups to extend the features of Keep It In The Community to improve how to submit assets for nomination, and how best to celebrate the listed assets by adding all sorts of local detail and background.
Have a look and let us know what you think so far.
In addition to the initial grant from Power To Change, this project has been implemented with the support of the Plunkett Foundation and the Ministry of Housing, Communities and Local Government.