Artificial intelligence and machine learning seem to be everywhere at the moment – every day there’s a new story about the latest smart assistant, self-driving car or the impending take over of the world by robots. With FixMyStreet having recently reached one million reports, I started wondering what kind of fun things could be done with that dataset.
Inspired by a recent post that generated UK place names using a neural network, I thought I’d dip my toes in the deep learning sea and apply the same technique to FixMyStreet reports. Predictably enough the results are a bit weird.
I took the titles from all the public reports on fixmystreet.com as the training data, and left the training process to run overnight. The number crunching was pretty slow and the calculations had barely reached 5% in the morning. I suspect the training set was a bit too large, at over 1M entries, but end result still gives enough to work with.
The training process produces checkpoints along the way, which you can use to see how the learning is progressing. After 1000 iterations the model was starting to be aware that it should use words, but didn’t really know how to spell them:
Mertricolbes Ice does thrown campryings Sunky riking proper, badger verwappefing cars off uping is! Finst Knmp Lyghimes Jn fence Moadle bridge is one descemjop
After 15000 iterations it’s starting to get the hang of real words, though still struggling to form coherent sentences.
Untaxed cacistance. Broken Surface in ARRUIGARDUR. Widdy movering Cracked already nail some house height avenue. Light not worky I large pot hole Dumped shood road nod at street. Grim Dog man Ongorently obstructing sofas. This birgs. Serious Dirches
After 68000 iterations there seems to be enough confusion in the training data that things start to go south again with the default parameters:
Urgely councille at jnc swept arobley men. They whention to public bend to street? For traffic light not working
Tweaking the ‘temperature’ of the sampling process produces increasingly sensible results:
Large crumbling on pavement Potholes all overgrown for deep pothole Very van causing the road Very deep potholes on pavement Weeds on the pavement Several potholes in the road Rubbish Dumped on the road markings Potholes on three away surface blocking my peride garden of the pavement Potholes and rubbish bags on pavement Poor road sign damaged Poor street lights not working Dog mess in can on road bollard on pavement A large potholes and street light post in middle of road
As well as plenty of variations on the most popular titles:
Pot hole Pot hole on pavement Pot holes and pavement around Pot holes needings to path Pothole Pothole dark Pothole in road Pothole/Damaged to to weeks Potholes Potholes all overgrown for deep pothole Potholes in Cavation Close Potholes in lamp post Out Potholes in right stop lines sign Potholes on Knothendabout Street Light Street Lighting Street light Street light fence the entranch to Parver close Street light not working Street light not working develter Street light out opposite 82/00 Tood Street lights Street lights not working in manham wall post Street lights on path Street lights out
It also seems to do quite well at making up road names that don’t exist in any of the original reports (or in reality):
Street Light Out - 605 Ridington Road Signs left on qualing Road, Leave SE2234 4 Phiphest Park Road Hasnyleys Rd Apton flytipping on Willour Lane The road U6!
Here are a few of my favourites for their sheer absurdity:
Huge pothole signs Lack of rubbish Wheelie car Keep Potholes Mattress left on cars Ant flat in the middle of road Flytipping goon! Pothole on the trees Abandoned rubbish in lane approaching badger toward Way ockgatton trees Overgrown bush Is broken - life of the road. Poo car Road missing Missing dog fouling - under traffic lights
Aside from perhaps generating realistic-looking reports for demo/development sites I don’t know if this has any practical application for FixMyStreet, but it was fun to see what kind of thing is possible with not much work.
Help us innovate in Civic TechnologyDonate now
We are happy to confirm that FixMyStreet Professional (the service formerly known as FixMyStreet for Councils) has been accepted onto the GCloud9 procurement framework.
Why is this important?
Using GCloud9, which is overseen by the Crown Commercial Service, removes much of the admin burden from public sector teams who are seeking to procure cloud based software and makes it easier to get down to the question of who has the best product for their needs.
What is FixMyStreet Pro?
FixMyStreet Pro represents the outcome of our co-design project with Oxfordshire County Council to take our popular FixMyStreet platform and build in a new set of features that genuinely made it as useful as possible for their staff (and Council staff all over the UK).
With a focus on retaining the user focused design and approach FixMyStreet was known for we have added or improved functionality for Council customer service staff and introduced a whole new set of tools to support Council inspectors including the ability to manage their tasks from within the app and to work offline when out and about.
This project has also made some improvements to the wider user experience for citizens with new front-end features being added all the time based on user research and feedback.
Want to learn more?
You can find out more about the service over on the GCloud Digital Marketplace or check out our own product pages where you can also get in touch with us if you would like to see a demonstration of the service or learn more about how we might be able to help.
It’s that time of year again. Local elections are on the 4th of May and we have updated our boundary change checker. It also helpfully lets you know if your ward is not having elections (not that your author was unaware of course.)
On May 4th elections will be taking place across English, Welsh and Scottish councils as well as the the elections for the new ‘combined authority’ Mayors.
Ward boundaries are changing
You might think you already know where to vote, and who’s standing for election in your area.
But both are dictated by which ward you live in — and that may not be the one you’re used to, thanks to ongoing changes in ward boundaries.
There’s no need to worry, though. As before, we’ve got the data that will tell you whether your ward has changed. Just enter your postcode here.
As Mark mentioned last month I have recently joined mySociety as Product Manager in the Better Cities team. This is something of a departure for me as I have spent most of my career working for large, publicly funded institutions — places like the Office for National Statistics, Medical Research Council and Department for Environment, Farming and Rural Affairs to name but three.
That said a large part of all those roles was trying to convince colleagues of the benefits of using the kind of products and services organisations like mySociety provide so maybe it isn’t that big a leap after all.
It has been a long held ambition of mine to work for mySociety — looking back it was almost 10 years ago when I first mentioned them/us on my blog and the organisation has been a consistent influence on me ever since with a number of former staff, trustees and volunteers becoming friends and colleagues over the intervening years.
It was my gateway to the wider world of the ‘civic tech’ community to which I have been a proud contributor for many years now — most recently mainly through running a weekly jobs list for public service minded organisations seeking digital staff.
At the moment I am primarily focused on learning as much as I can about our FixMyStreet platform (including the ‘..for Councils’ product) and investigating the MapIt service as well. As I get up to speed with things I am also taking the opportunity to get about and about — attending events and trying to meet people who make use of our projects.
In the weeks to come I’ll be attending MeasureCamp in Cardiff (4th February), giving a talk at World IA Day in Manchester (18th February), speaking at BathHacked on the 22nd February (Bath), helping out at day one of Open Data Camp in Cardiff (25th February), going to ProductCampon the 4th March (London) and giving a lightning talk at the launch of the new Tech4Good group in Bristol on the 23rd March. All this as well as hosting my regular ‘minimal viable meet-up’ on the 8th February in Bristol.
If you are attending any of these events please and want to chat ‘Better Cities’, civic tech or just want to get hold of one of our lovely stickers please come and say hi — there are rarely any other attendees who sound as Bristolian as I do so I am usually easy to find. Also if there are any events you’d like to suggest I attend — either to give a talk about the mySociety ‘Better Cities’ work or just generally an opportunity for me to learn more about how people are using digital tools to engage with and influence local democracy please do let me know.
I am @jukesie on Twitter (be warned — I am something of a prolific tweeter!) and my email is firstname.lastname@example.org so please get in touch.
A couple of months ago we started a search to find three new team members for our Better Cities practice: a Product Manager, Web Developer and Sales & Partnership Manager.
We’ve now signed up our new colleagues and I’m excited to say that Matt Jukes will be joining as Product Manager. He’ll be leading the development and expansion of our Better Cities services and building upon our recent work to launch FixMyStreet 2.0.
Matt is a familiar name to many people in the UK Civic and Gov Tech sphere — most notably for the transformation work he led over the past couple of years at the ONS (Office of National Statistics). Matt brings with him an excellent pedigree of building and leading teams. What was of particular interest to us was the way that he talks openly and publicly about the projects he’s working on, which sits very well with our approach.
Stuart Harrison will be joining us in January from the ODI (Open Data Institute) as a new Web Developer on our commercial and Better Cities work. Stuart brings a wealth of public sector experience and is already an avid user of mySociety’s services. We’re very lucky to have snagged him, not least as he brings masses of insight and experience into how Open Data meets the commercial world.
Finally our new Sales & Partnership Manager Rachel Baker will start in early January
(we’ll let you know who they are once we’ve got that all wrapped up shortly). She comes with valuable entrepreneurial and marketing experience from previous startups and will also be working from our growing base around Bristol and Bath.
Rachel will work with our Local Authority partners in the UK and around the world, on FixMyStreet for Councils which has undergone a substantial revamp over the last few months. And will help us better connect with like-minded partners who might benefit from making use of our services — if this is of interest please get in touch.
And one departure
Finally and sadly we also say goodbye to our dear friend and colleague Ben Nickolls.
Ben has been with us as Head of Services for the past four years, working with me and the commercial team. He has been instrumental in turning around the fortunes of our services team over the past 18 months, giving us a firm basis for the public sector product and service business we are seeking to grow over the next couple of years.
Now with the groundwork done, Ben is off to pursue his startup ambitions with the wonderful Libraries.io project, which aims to document and provide access to all of the Open Source libraries in the world. This is a hugely ambitious project. We’re very excited for Ben and wish him all the luck — we’ll be watching his progress closely.
So all change for the new year in our Better Cities practice. There will be lots more to share on our progress in the coming months.
Post updated 20th December with news that Rachel is joining as our Sales & Partnership Manager.
The FixMyStreet codebase is used all over the world by people running versions of the site for their own country or jurisdiction. This week, we’re proud to announce the release of FixMyStreet version 2.0.
This version contains a wide array of new features that benefit FixMyStreet sites’ users, administrators, and the officials who receive reports. They include elements that the UK FixMyStreet was the first to trial, such as nicer-looking HTML emails for users and authorities, the ability to filter reports by multiple states and categories, a new admin user system with graduated permissions, and various bugfixes and development improvements.
Over the next few weeks, we’ll be publishing a series of blog posts over on fixmystreet.org/blog/, examining the changes in detail. If you run a FixMyStreet site, or you’re just interested in coding and technical issues, you may find them of interest. Meanwhile, here’s the broad overview.
New front-end features
HTML email: There is now the option for all emails sent by FixMyStreet to be HTML formatted where previously they were plain text only. This includes confirmation and questionnaire emails to the user, and report emails to the public body. These emails include any image added to the report, plus a small static map of the problem’s location.
State/category filtering and sorting of list pages: When viewing a list of reports, you can now filter and sort them in pretty much any way you choose, including sorting by most- or least-recently updated, newest or oldest, or most commented. You can also select multiple categories or states (e.g. “fixed”).
Pretty area highlighting on body pages: The highlighting of areas on a body page has been inverted, so that the unimportant parts of the map are shaded and you can interact more easily with reports on the page.
- Users can now update their own email address This was a frequent request from users and we’re glad to report that they can now do it themselves on their account page.
Performance improvements: When looking at reports from a list page, the other report pins stay visible so that it is easier to switch between them. The report itself is being pulled in behind the scenes, meaning the whole page does not need to reload. The map no longer extends underneath the sidebar and header, which makes things easier, and a scroll wheel can now zoom the map in and out.
Making privacy options clearer: The reporting form has been separated into public and private sections, to make it clearer which parts of what you provide will be made visible on the site.
Showing the relevant recipient: If you live in an area where there’s more than one body, the category you pick normally dictates which body we send your report to. Now, when you select the category we update the name of the body given at the top of the report page, if we know that the report will be sent there.
New admin user system
Admin users can now use the same log-in right across the site – whether they’re making a report like a standard user, or logging in to make edits and moderate the site.
In the past, the distinction between admin and other users was black and white. As an admin user, you had access to every part of the site, but users can now be given individual permissions for various layers of access. These include:
- Proxy users This layer grants the ability to create a report or update on behalf of a body, or as another user. We envisage this being useful in a body’s contact centre, where they receive a report over a phone and enter it into FixMyStreet as that user;
- Report editors Giving the power to edit a report’s category, state, or location. If the admin user changes the category, and that change means that a different body is now responsible for the report, it will be re-sent;
- List makers, who can compile their own shortlist of reports they wish to go and inspect. This may be useful for a contractor or team who wishes to compile the day’s tasks;
- Quick responders These users have access to response templates, allowing them to edit and publish templated updates;
- Prioritisers These users may set different priorities on reports;
- Trusted users A simple reputation system, which e.g. potentially lets reports from trusted users be actioned more quickly.
The admin report edit form has also been greatly improved, including a map to update a report’s location (and re-sending the report if the body changes), and much tidier layout.
Bugfixes and development changes
Bugfixes include updating the top-level domain (TLD) list for email validation, hiding authorities which don’t exist any more on the all reports page, and fixing the previously-broken photo preview display after form submission. We have dropped support for Internet Explorer 6.
If you’re a re-user of the codebase, there are a number of changes that will hopefully help you out. See the extended version of this blog post on fixmystreet.org for more details.
- HTML email: There is now the option for all emails sent by FixMyStreet to be HTML formatted where previously they were plain text only. This includes confirmation and questionnaire emails to the user, and report emails to the public body. These emails include any image added to the report, plus a small static map of the problem’s location.
Earlier this year we tweaked our strategy to better align our commercial work with our charitable projects. We’re now looking to hire a couple of experienced and motivated individuals to help us really turn up the heat on this approach.
Our work at mySociety covers three practice areas; Freedom of Information, Democracy and Better Cities. Each in their own way use different methods to give citizens more influence over those with power. Making it easy to access public information, or easier to understand what decisions mean and their implications for all of us.
Most of our work to date has been funded through grants and donations, but we believe that we can often make greater impact on a longer term basis where we work on a commercial footing, especially if we can bring in appropriate revenue which would complement our charitable income and help provide a more sustainable future for our organisation.
To boost the commercial skills we have within our team we are looking for an experienced Product Manager who can help set the strategy for how we position our products, develop the wider markets we operate in, bring in more public sector clients, help serve our current clients and create an environment in which our products can thrive.
To aid them in this quest we’re looking for a Sales and Partnership Manager to help us identify and engage with community groups, citizen engagement services, local authorities, technology providers and end users who would benefit from working with us to help more citizens to demand better.
To top it off we urgently need to hire at least one additional Web Developer to our commercial team with at least three years of programming experience in Ruby, Python, and/or Perl.
For each of these roles we’re looking for experience of working with or within local authorities or the wider public sector and civil society. They’ll be comfortable speaking with a broad range of people within local and central government, and their service providers, and will understand the needs of their end users – generally local residents. Importantly they’ll be comfortable working within a geographically distributed development team.
Help us learn and improve
The aim our Better Cities practice is to help people exert a little more control within their local communities – especially people who have never previously tried to make any such difference, or members of marginalised groups who might believe they have little chance of success in getting things changed. In particular we want to learn more about how best to deliver local community level services and to understand the complex needs of those currently under-represented by local government and public services.
Whilst we have over 10 years experience of delivering local services via FixMyStreet.com, we want to understand if such services actually give agency to those who lack it most to affect and impact their local communities, and if so in what way? Does this lead to further civic engagement and participation, if so how? If not can we adapt our approach to make this more likely? And where we currently fall short of representing these needs within our current services, what measures can we take to adapt existing services, or what new services might we create in their place?
As we continue to learn we’ll further build upon the FixMyStreet principles of issue reporting and resolution to cater for a variety of interesting and practical new use cases,targeting hot button policy areas around housing provision, health, education quality, work and benefits.
Importantly we’ll succeed if we ensure that our services are well used by a wider diversity of people in a wider spread of regions.
So if you think you can help us in these goals, have ample experience in creating and leading on the development of digital products and are motivated and energised by working with local communities, government and the public sector we’d very much like to hear from you.
You can apply here;
Product Manager – Closing date, 10am Friday 11th November
Sales and Partnership Manager – Closing date, 10am Friday 11th November
Web Developer – Closing date, 10am on Wednesday 26th October