Stuart Lawson is a librarian, one of the editors at the Journal of Radical Librarianship, and a part of the open access movement which advocates that research outputs should be distributed for free and online, with an open license — for the good of all.
So it should come as no surprise to learn that some of Stuart’s recent research, informed by Freedom of Information requests to the UK’s Higher Education Institutes, has focused on research journals — and specifically, how much institutes are paying for subscriptions. You can see the requests here, the data released in this spreadsheet, and the resulting report here.
The study collected details of payments made by HEIs for access to academic journals from 2017-2019, focusing on ten publishers. The research team discovered that the total expenditure was more than £330 million.
We spoke to Stuart and asked for some background to this FOI-based investigation, beginning with an explanation of the original motivation behind it:
“Open access publishing means that research is available to everyone, but there are debates around how that model can be paid for. And since there is currently a mixed system where some publications are open access and some require subscriptions to access, libraries are continuing to pay a lot of money for subscriptions while also trying to find ways to fund open access.
“I am a librarian who wants all research to be published open access rather than behind a paywall, so I felt that it was important to know the financial costs of the current system.
“Previously, the amounts were unknown. It’s impossible to have conversations about the appropriate cost of scholarly publishing if we don’t know what those costs are in the first place!”
The need for FOI
Freedom of Information is, of course, a practical way to obtain data from public authorities, and to build up a nationwide picture. But in this case, it was vital.
“Using the legal tool of FOI was the only way to get this data, as institutions were not voluntarily releasing it.
“One publisher, Elsevier, even had a clause in the contracts signed by libraries that forbade them from telling anyone how much they were spending, unless they were required to via an FOI request.
“These ‘non-disclosure’ clauses are common worldwide in publisher contracts, but thankfully not widely used in the UK (except by Elsevier) because Jisc — the higher education body that negotiates most deals — have worked to remove them”.
Despite the reluctance that one might assume that this signified, Stuart says getting the required information was pretty straightforward once they’d submitted the FOI requests. In fact, the hardest part was the admin:
“A majority of HEIs provided the data promptly, although some refused in the first instance which meant I need to push back and sometimes requested an internal review of the handling of the request.
“Eventually most institutions provided the data, but the hold-outs caused a lot more work for me”.
Making requests in public
Why was WhatDoTheyKnow particularly suitable for this project?
“It was the best way that I knew of to make bulk requests to organisations. But more importantly for me, I wanted to make sure there was a complete public record of all responses so that when I compiled the data, others could verify it”.
That’s one of the reasons that WhatDoTheyKnow is set up to publish FOI requests and responses online, so we were glad to hear this.
And what is Stuart’s desired outcome from this study?
“For people to realise the high cost of subscription charges, and for libraries to question how much they are spending. And perhaps even cancel some of the deals and spend their money on enabling open access instead.”
It’s possible that this piece of research will be enough of an eye-opener to start making a change in this area. But Stuart’s realistic:
“I hope I don’t need to send these requests again in future years, but the situation is still moving quite slowly, so it might be necessary to use WhatDoTheyKnow once again!”
Image: Bruno Mira
Thanks to the travel restrictions imposed due to Covid-19, TICTeC will not be going ahead in Reykjavik. Instead, we will be taking the TICTeC experience online. We very much hope you’ll join us from the comfort of your own homes.
Here’s everything you need to know to be a part of the first ever completely virtual version of our annual Impacts of Civic Technology conference.
On the afternoons of Tuesday 24 and Wednesday 25 March.
Tuesday: 13:00 to 17:00 GMT
Wednesday: 13:00 to 17:10 GMT
There’ll also be fringe events happening around these times.
Free of charge, of course.
What’s the agenda?
You can see the timings for each session on the agenda here. Join us for the whole lot, or dip in and out to the parts that interest you.
We’re delighted to say that we’ll still have barnstorming keynote sessions from Nanjala Nyabola and Hollie Russon Gilman, each giving profound insights into the extraordinary political times we are living in, from a global perspective.
There’ll also be sessions from representatives of Iceland‘s local and national government and civil society; research presented from Uganda, Taiwan, North and South America and beyond; and speakers from a range of organisations including Civic Hall, Citizens Foundation of Iceland, g0v.tw, Transparency International UK, The African Legal Information Institute, Frag Den Staat… and more.
So TICTeC will still be allowing you to gain insights from all around the world, sticking to our mission of bringing together practitioners, commentators, academics and funders to debate, network, and share research and knowledge in the civic tech field.
What else is going on?
There will be fringe events as well (details TBA) and the opportunity to chat online with other attendees.
What do I need to take part?
Presentations will be taking place via Zoom. Before TICTeC starts, please make sure you have downloaded the Zoom launcher, or read this page to learn more about Zoom, including how to use it in your browser. The link to TICTeC is https://zoom.us/j/528401903.
This link is also on the agenda page.
Slido allows the audience to submit and vote for audience questions, so the ones that the speaker answers are the ones most people want to hear. Once the event is live, just enter the code at the top of the agenda on slido.com in your browser. If you prefer to use the app, download it here.
Access to Google docs
There’ll be a collaborative document for each session where we can all work together to take notes for each session. Find them all here.
You won’t need a Google account to view or add to these documents.
You might want to do all this in good time before Tuesday, just to make sure everything works!
What if I can’t make those dates?
We’ll be putting the videos online afterwards, and you can check the notes as well. So the only thing you’ll miss out on is the real-time chatting and networking.
Please spread the word
We have room for literally thousands of participants, so this is the chance for anyone who has an interest in Civic Tech to come and enjoy some great presentations for free.
Let’s take this opportunity to widen our audience and put the word out through social media, newsletters, blog posts, wherever people will see it. Thanks!
We want to extend our thanks to the sponsors and supporters of TICTeC 2020:
Back in February 2012, we announced the launch of a new site for Mzalendo, a parliamentary monitoring website for Kenya.
This year, we handed the hosting, development and maintenance of the site over to the Mzalendo team on the ground. We’re delighted that they are in the position to no longer require our help.
Supported by the Indigo Foundation, this was one of mySociety’s first formal partnerships in which we developed a website for an existing organisation — in this case, building on the work of two activists Ory Okolloh and Conrad Akunga, who had been filling a gap in Kenya’s public provision of parliamentary information by blogging and publishing MPs’ data since 2005.
If it wasn’t for their work, Kenya would be a whole lot less informed about its own parliament: the official government website, for example, only had information about 50% of the nation’s MPs at the time, and the country’s Hansard could only be accessed by request to the Government’s Printer’s Office.
We were able to draw upon our experience with our UK parliamentary site TheyWorkForYou to avoid the common pitfalls in building such projects, and provide useful features such as an online searchable Hansard, responsive design, MP ‘scorecards’ and an easily-updated database for representatives’ details.
During the years of our partnership, Mzalendo kept the site maintained with data and news, while we worked on the development of new features they requested, fixing any bugs that arose, for example when the Kenyan parliament changed their data outputs, and hosting.
But there are plenty of willing and able developers in Kenya, and it became increasingly obvious that funding could be more effectively — and efficiently — routed directly to them rather than to us in the UK.
Like most mySociety code, the Pombola codebase on which Mzalendo was built is open source, so anyone is free to inspect, reuse or just take inspiration from it. The handover should, therefore, be reasonably painless for the new developers.
We wish Mzalendo all the best in their ongoing efforts to keep Kenya informed and politically engaged.
Image by Valentina Storti: a tawny eagle flying over Laikipia District, central Kenya (CC by/2.0)
If you’re looking for a quick and simple thing you can do from home to support meaningful action on climate change, help us make a list of councils’ Climate Action plans.
In the past 18 months, there’s been a spate of climate emergency declarations from local councils, in which they recognise the seriousness of the climate situation and commit to taking action. 65% of District, County, Unitary & Metropolitan Councils and eight Combined Authorities/City Regions have now declared a climate emergency. Many of these declarations commit to a date for getting to net zero, ranging from 2025 to 2050.
These declarations, and the commitment from central government to reach net zero by 2050, represent much needed progress. Commitments are good. But what we really need to address the climate emergency, both at a national and local level, are concrete plans.
As councils develop their plans for addressing the crisis, many individuals and groups need to be able to easily access, discuss and contribute to them to make sure they’re ambitious and high quality.
Councils can also learn and draw encouragement from each other’s efforts. At the moment, we think that’s harder than it should be.
Lots of people who want to take action on the climate locally are having to do the same work of finding their council’s plan, or finding out where they are in developing it, or finding other plans to compare it to. There’s no central place to find all the Climate Action plans that have been developed, or to track the process of developing them (or not!)
The climateemergency.uk site has been collecting those climate action plans they can find, but we think we can help them get a fuller picture, and create a resource that will help us all — and we’d like your help!
In the spirit of ‘start where you are’, we’ve made an open spreadsheet for collecting council climate action plans, and kicked it off with the ones from climateemergency.uk, to see if we can help improve what’s available. At the very least we’ll maintain this as a simple open resource, and share it wherever we think it might be useful. If you have thoughts about people who ought to know it exists, to use it or contribute to it, please do share them in the comments or drop us an email.
The key piece of information we want to collect at this point for each council is the URL where their Climate Action Plan can be found. But we’ve added some extra columns for anyone who wants to start looking at the details.
So, if you have five minutes, please have a look for a council’s Climate Action Plan and add it to the sheet.
If this works out well and seems useful, we’ve got some ideas about how to extend it and start to turn it into a more detailed and useful dataset or service. For example, tracking how the plans develop over time, how councils make progress against them, or breaking them down into a more detailed and comparative dataset — there seem to be key questions that would be useful to answer, for example around things like whether the plans only address emissions under councils’ direct control, or whether they’re focused on the area as a whole. So if you’d like to partner with us or support us to turn these ideas into reality, we’d love to hear from you! Email us at firstname.lastname@example.org.
Last month, we sent some of our newsletter subscribers an email to say that we’d noticed they weren’t opening our newsletters and, unless we heard otherwise, we’d be removing them from our mailing list.
A number of people replied to that email to say, ‘Actually, I do open your emails — I just have image loading turned off so you don’t track my activity’. This was a welcome reminder to examine our own practices.
Such responses triggered a conversation within mySociety, resulting in a session at our team meeting last week, with the upshot being a decision to turn off personally identifiable tracking on all our newsletters.
We’ll be able to see how many people visit our blog posts from the newsletter, but that’s all — we won’t be able to associate visits with particular accounts, nor will we know how many of the views are from repeat visits rather than different users.
This decision reflects mySociety’s position as an organisation concerned about matters of privacy and also feels like it’s part of a wider movement within our sector: we’re not the only ones having this conversation.
But why did you track opens at all?
The obvious question, given this statement, is why mySociety were tracking activity in the first place.
Partly tracking was in place to help manage the financial cost of our newsletters. We use Mailchimp, whose costing structure is based on how many subscribers you have at any one time: you pay more for each subscriber even if they are not active.
Sending an email to people who hadn’t opened — or who appeared not to have opened — our emails for over a year was a way of cutting the costs of holding a large database on Mailchimp, and you can only do that if you know who they are.
For purposes of understanding the impact of our communications, too, it is helpful to be able to see how many people have opened a mailout, what stories have been clicked on, and which have not.
However, this can be done in a way that doesn’t intrude on the users’ privacy – and, as mySociety team members posited during our team meeting, a cost saving on our side doesn’t justify infringing on privacy.
Mailchimp’s defaults assume that you want as much information in as granular a form as possible, but they do give you the option to turn tracking off.
As far as we can see, one has to remember to do this each time a mailout is sent, but we’ll also be contacting Mailchimp to ask them if the relevant boxes could be unchecked by default, or managed at a global level.
So from now on
After our team discussion, we’ve made the immediate decision to turn all tracking off on our newsletters. Thank you if you were one of the people whose feedback spurred us to have this conversation and arrive at this decision.
And if you’d like to subscribe to our newsletters, you can do so here.
Image: Jehyun Sung
What is your local authority doing about the climate emergency?
Of course, we all want to see action, and fast. Several authorities across the UK have declared a climate emergency, while others are bringing climate-friendly propositions to the table. But how do you know the actual concrete outcomes of these?
Fortunately, Friends of the Earth have put together a tool which helps you see just that — and we’re glad to say it makes use of our MapIt API.
We spoke to Joachim Farncombe, FoE’s Digital Delivery manager, to find out more about what they built, how it works, and how exactly MapIt fits in.
How climate-friendly is your area?
“The Climate Tool invites people to tap in their postcode, and then discover how their local authority is performing on a number of measures, including renewable energy, transport, housing, waste and tree cover.”
Joachim explains that in fact, they’ve produced two tools: “There’s one highly detailed version which we think our existing supporters will use, and another which provides a summary of the data for those newer to Friends of the Earth and the whole area of councils’ climate responsibility.
“Both tools reveal data from local authority areas, around key issues that are impacting our climate. The ultimate aim was to create an engagement opportunity that would drive new and existing supporters to take climate action locally.
“The whole project is designed to highlight that there are different ways of addressing the climate emergency. One of the key drivers of change is for communities to put pressure on their local authorities to make urgent changes to reduce emissions”.
So — once you’re all clued up on how your local area is doing, what then?
“Once you’ve absorbed the data, there’s the option to click on ‘What can I do to help?’.
“We’re asking people to add their name to support a climate action plan in their area. We’ll also be introducing those who sign up to our climate action groups, a network of community groups working to make our communities more climate friendly.”
Where does MapIt fit in?
The MapIt API allows developers to include a postcode input box anywhere on a web page. When a user enters their postcode, MapIt checks which administrative boundaries it sits within. The developer can choose what type of area they need — for example, if the site wants to encourage people to write to their MP, MapIt will return the constituency; or, in this case, as users will be contacting their local authorities, it returns the relevant council.
Joachim says that FoE already knew of MapIt as they’d used it in their campaign for more trees. “It was very straightforward. The JSON response was easy to parse and the API speed was impressive.”
Once the user has been matched to the right council, the climate tool dips into its store of data to show them the current climate performance in their area, across key topics.
“We developed an internal API called FactStore which indexes whatever sets of data you need. In this case, this was data collated from approximately fifty different external datasets. This data was all pulled from open data sources, mostly released by the authorities themselves.”
The tool was well received, and was shared across social media by supporters and new users alike. “Actually”, says Joachim, “it was a bit more popular than we’d anticipated, and we hit our initial quota on MapIt very early after launch, but there was a quick fix (we just upgraded our quota!)”.
In short? “MapIt has been invaluable. Without it, we’d be unable to connect the users location with the datasets we’d collated”.
We’re looking forward to working with Friends of the Earth more in the coming months — watch this space.
This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here.
I saw a comment on Twitter the other month along the lines of: “is civic tech too boring? It’s dominated by reporting potholes to councils”.
As someone working in civic tech I find this terribly unfair because civic tech is about so much more than that! For instance, we also report dog poo to councils.
But it’s certainly true that there are a lot of potholes involved. It’s the largest use of FixMyStreet, representing a quarter of all reports. People have submitted over 361,000 reports and over 54,000 photos of potholes. As a result, while the FixMyStreet database represents a fraction of all potholes, it represents one of the largest datasets of pothole reports covering the whole country.
And while it’s easy to think of potholes as the obsession of people pointing at roads in local papers, they are a serious problem. There are a lot of them, they appear everywhere, cause problems on roads when people try to avoid them, and damage when they don’t. For cyclists, potholes can be fatal.
Given that, what does FixMyStreet data tell us about potholes?
How many potholes are reported through FixMyStreet?
Up to the end of 2019 there have been 423,736 potholes or road surface defects reported through FixMyStreet (either .com or a cobrand), with 90,000 reported in 2019. Working from a rough figure of 675,000 actual pothole reports a year, this is around 13% of all potholes reported in the UK.
A feature of reports to FixMyStreet is that, while the majority of reports are made by men, there are different ratios in different kinds of reports and categories are often gendered in terms of reporters. Deriving the gender of the reporter from their name, potholes and road surface defects are mostly reported by men, and disproportionately more than the site in general.
As explored in a previous post, this isn’t an essential gender difference but is likely to result from men having far more cause to encounter potholes. In 2013, men in the UK were on average driving twice as many miles per year as women.
People who report potholes are more likely to have reported multiple problems than other reports. Most pothole reports are made by people who have reported multiple reports and represent a smaller proportion of single report users than other report types.
When are potholes reported?
Potholes tend to be reported during the day, but disproportionately compared to other requests around the evening commute. The chart below shows the distribution of reports by time of day, where green indicates the number of reports is higher than the general distribution of FixMyStreet data.
While potholes are associated most with the start of the year, they occur in smaller numbers all year long. The number of potholes reported through FixMyStreet peaks on the 28th February.
Where are potholes reported?
While reports in FixMyStreet are less likely to be made in less deprived areas in general, this effect is larger for potholes:
This effect is driven more by reporting being lower than usual in more deprived areas than especially high in less deprived areas:
This pattern was generally similar for the Income and Employment domains of deprivation. This does not necessarily mean there are more actual potholes in these areas, but possibly that people in areas with higher income and levels of employment are more likely to report them.
Examining reports using the deprivation subdomain that measures difficulty accessing services (GPs, supermarkets, etc) shows a different pattern, where a disproportionate amount of pothole reports are made in areas with the least access to services.
The area with the worst access to services (typically a measure of distance to services) has a disproportionate amount of total pothole reports on FixMyStreet. This doesn’t necessarily indicate this is where most of the potholes actually are, but more remote, less traffic-ed potholes will rank lower in risk-based calculation than those on busier roads, and hence may go longer without fix, and make a report on FixMyStreet more likely.
Fixing potholes is a never-ending task, as they are an inevitable result of erosion of roads over time. That said, poor repairs will make the return of a pothole more inevitable than it might be. The issue isn’t just that the same pothole returns: if a pothole initially formed because the road surface was poor, others are likely to form in the same area too.
Looking at reports on FixMyStreet up to the end of 2016, for 3% of potholes a new pothole was later reported within 10m between six months and two years after it was first reported (with an average time lag of 15 months). Expanding that ratio to a 20m radius, 7% of potholes had a new pothole reported in the same time range.
While FixMyStreet’s data on potholes is far from universal, the geographical range gives us better scope than any single local authority’s data to see how reporting of potholes relates to social factors. You can examine this data yourself, on our geographic export, which gives counts of different categories of report by LSOA.
Research Mailing List
Sign up to our mailing list to hear about future research.
This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here.
When people make their first Freedom of Information request using WhatDoTheyKnow they are sent an email two weeks later, asking them to complete a survey. This survey has been running from 2012 and in that time has received 6,861 replies. Because this is an optional survey and not a requirement of making a request, this is a small proportion of the number of first time requesters in that time (around 3-4%). This response rate reflects that the survey is currently quite long and asks questions that, while more useful when the service was new, are now less helpful in understanding its ongoing impact.
As it’s unclear how representative this sample is of requesters of WhatDoTheyKnow, the overall results shouldn’t be read as authoritative of the user base. What is more interesting is how different groups of respondents use the site in different ways. The data from the surveys has been added to the explorer minisite, a research tool that uses chi-square tests to examine if there is a statistically significant difference in the distribution of responses.
Looking at the overall picture, the average age of respondents is around 45-54.
There were more than double the number of male respondents as female respondents. 17% of respondents said they had a disability. Disability is a broad category where self-identification can vary, which makes comparison to national figures difficult. However in 2011, 8.5% of the population of England and Wales were ‘limited a lot’ in their daily life as a result of a health problem of disability, while 9.3% were ‘limited a little’. This suggests that use of WhatDoTheyKnow is not broadly different from the national picture – however this could be disguising variation within different kinds of disability.
43% of respondents were working full time, 10% were working part time and 21% were retired. A majority (57%) were university educated.
On ethnicity, most respondents declared ‘British’ or ‘English’. 16% were part of a BAME (Black, Asian and minority ethnic) group. Because of the small number of responses over a large range of ethnicities, a second ‘reduced’ option was created by grouping responses that just presents BAME/Not BAME/NA. This would make general trends statistically detectable, but may also disguise trends when different ethnic groups have effects in different directions.
Over time there is a small number of trends. There is a slow rise in the number of female respondents – from 24% to 33% in 2019. There was statistically a larger proportion of BAME respondents in 2015 and 2016 (19-20%) and fewer in 2012-13 (13%). The number of respondents with disabilities does not show any significant differences between years.
BAME respondents are more likely to write to Education, Central Government and Other than the general dataset, and less likely to write to health, local government, emergency services, military services, and media and culture. BAME survey respondents make up 5% of requests to media and culture and 29% of education respondents.
Female respondents are more likely than male respondents to write to education and health authorities than the general dataset, and are less likely to write to emergency services, media and culture, transport, and military and security services. Female survey respondents make up 14% of requests to military and security services, and 40% to education.
Respondents with disabilities are more likely to write to health-related authorities and the emergency services than the general dataset, and less likely to write to transport and education. Respondents with disabilities make up 9% of requests to education authorities and 26% for health authorities.
Retired respondents are more likely to write to environment-related and local authorities, and less likely to write to central government and education. Retired respondents make up 7% of requests to education authorities to 33% for environment-related authorities .
Reversing the lens to look at one type of authority, respondents writing to education authorities are more likely than the general dataset to be female (but still majority male) and more likely to be part of a BAME group (but still majority white). They are more likely to be below the age of 24 and less likely to be above 55 than the general dataset and (related to that) more likely to be in education and less likely to be retired.
46% of respondents said they were writing on behalf of ‘all people in the community’. This group was more likely to be retired, less likely to be part of BAME group, but more likely to be part of a community group (but not a political group alone).
20% said they were writing on behalf of themselves/family as well as similar people.
14% said they were writing on behalf of all people — this group was slightly more likely to be earning less than 12,500, have excellent internet access, and more likely to be involved in political activity (less likely to be part of a community group), to have made FOI requests before, and to make lots of FOI requests.
13% said they were writing on behalf of themselves or family. This group has a spread on age, but is more likely to be older than 75 (and less likely to be 45-54) than the general dataset. 46% are still university educated, but this is less than the general dataset and this group is more likely to be have secondary or technical college qualifications, and slightly likely to be part of a BAME group than the general dataset (while still majority not BAME). This group is more likely to not be involved in groups or to previously have made requests.
Previous FOI use
The profile of a user who had never made a request before using the site is in many respects similar to other users. This group contains slightly more 25-34 year olds and those in full time education. They are more likely to be making requests where the information is mostly relevant for themselves/family or people similar and more likely to not be involved in community or political groups.
While this survey has found some interesting things about our users, it’s currently overly-long and has a much lower response rate than some of our comparable surveys. We’re looking at the best way of modernising the questions and survey platform to replace this survey, while maintaining continuity with some of the trends identified above.
Research Mailing List
Sign up to our mailing list to hear about future research.
Every road user relies on signs, so keeping them tip-top is in everyone’s interest. Now Transport Focus have launched their Sort My Sign campaign, asking road users to help them do just that.
They’d like everyone to report any signs they spot that are dangerous, dirty, broken, or obscured.
To support this programme digitally, Transport Focus came to mySociety, asking if we could help create a simple and intuitive mapping interface where these issues could be reported.
Keep your eyes on the road
Specifically, the scheme covers signs on roads managed by Highways England, which means motorways and some A roads.
FixMyStreet was the obvious starting point — we already have a data layer for these roads, which means that your everyday FixMyStreet reports can be routed to Highways England rather than the council if they are the responsible body.
Plus, as we’ve detailed many times before, the FixMyStreet platform can be repurposed for any project dealing with location-based reports, and has in the past been put to all sorts of uses, from reporting empty homes to helping fight corruption.
Nonetheless, we perceived one potential challenge when it came to setting up sign reporting.
Don’t report and drive!
FixMyStreet is generally well-suited for people making reports on the go — in fact, thanks to the ‘use my location’ functionality, it is ideal for reporting issues like potholes or broken pavements on your mobile while out on a walk. But obviously, road signs are a slightly different matter. If you are driving, you certainly mustn’t be fiddling with your mobile phone, so ‘use current location’ is only helpful if you have an amenable passenger to make the report.
That’s fine — you can always make the report later of course: but that means you’ll need to know roughly where you were when you saw the sign, something that’s a bit trickier on a long drive than it might be on a stroll around your neighbourhood. FixMyStreet allows you to find any UK location with the input of a postcode or street name, but these are details you’re unlikely to have to hand if you have simply driven through.
After some thought we realised that, on a motorway, the location identifier most people will find easiest to recall will probably be the junction number.
So that set us a challenge: how could we best enable ‘search by junction number’?
Sign here…and here
Ideally, we wanted a user to be able to visit the Sort My Sign site and enter the name of a junction, just as they’d enter a postcode or street on the FixMyStreet homepage — and then to be taken to a map centred on that point.
But sourcing a mapping between motorway/junction number and co-ordinates proved surprisingly tricky. mySociety developer Matthew takes over the story.
“I first looked at OpenStreetMap data — its geocoder, Nominatim, worked really well for some junction numbers, but didn’t work at all for others. If a junction has been assigned a name (like J23 on the M6, which is known as ‘Haydock’) it can only be looked up by that name, not by number. But we wanted users to be able to look up junctions by number.
“I could also export all the junction data from OpenStreetMap, but the junction nodes alone aren’t linked to the motorway, so that looked like it would prove tricky to match up.”
FOI to the rescue
“But by a stroke of luck, I then discovered that someone had used another of mySociety’s services, our Freedom of Information site WhatDoTheyKnow, to make a request to Highways England asking for the positions of all the driver location signs (the repeaters every 100m or 500m along the motorways giving the name and distance from start).
“In response, Highways England had provided that information, so I knew I could use that to at least provide a mapping between location sign and geographic co-ordinates.
“Each sign also had information about what junction it was nearest or between, so by constructing an average of all the location sign co-ordinates associated with a particular junction, I came up with a pretty good estimate for the location of the junction itself.
“I added all the sign and junction data into a small SQLite database (which means it’s portable and doesn’t need to be associated with the main database) and wrote a little bit of code to spot when someone entered a junction name in any of a variety of different formats, then look up the matching location in this database”.
Signed, sealed, delivered
To test this out, Matthew had all his colleagues name their favourite junction… perhaps not to be recommended as a party game, but it did at least prove that his code had cracked the problem.
Something much appreciated by Head of Strategy at Transport Focus, Guy Dangerfield, who says, “mySociety has been excellent in understanding what we needed and finding ways to achieve our objectives.”
You can give the new system a go here — and perhaps bookmark the site so that you know where to report a sign next time you see one that needs fixing.
Once you’re safely off the road, that is.
We’re delighted to announce the schedule for TICTeC 2020, our two-day conference that focuses on the use and impacts of Civic Tech around the world. If this sounds good to you, you’d better book now, because spaces are limited.
Thanks to our sponsors, TICTeC is returning for its sixth year and this time will be held in Reykjavik on 24th and 25th March 2020. Councils in Iceland are pioneers in using digital tools to elicit feedback and engagement from its citizens on policies, expenditure and projects, so TICTeC 2020 will be a really unique occasion to hear about these, as well as many other innovations from across the world.
You can find out all about TICTeC over on the event’s website, and get a flavour of what Iceland is like as a place to visit in this video: