Census data: there’s lots of it. It contains fascinating insights.
But as with many huge datasets, those insights are not always easy to find at first glance — nor is it easy for the untrained observer to see which parts are relevant to their own lives.
Wazimap in South Africa takes the country’s census data and turns it into something the user can explore interactively. Originally conceived as a tool for journalists, it turned out to be so accessible that it’s used by a much wider range of the population, from school children to researchers. It’s a great example of how you can transform dry data into something meaningful online, and it’s all done using free and open source tools.
Our points-to-boundaries mapping software MapIt is part of that mix, putting the data in context and ensuring that visitors can browse the data relevant to specific provinces, municipalities or wards.
We asked Greg Kempe of Code for South Africa, to fill us in on a bit more.
What exactly is Wazimap?
Wazimap helps South Africans understand where they live, through the eyes of the data from our 2011 Census. It’s a research and exploration tool that describes who lives in South Africa, from a country level right down to a ward, including demographics such as age and gender, language and citizenship, level of education, access to basic services, household goods, employment and income.
It has helped people understand not just where they work and live, but also that data can be presented in a way that’s accessible and understandable.
Users can explore the profile of a province, city or ward and compare them side-by-side. They can focus on a particular dataset to view just that data for any place in the country, look for outliers and interesting patterns in the distribution of an indicator, or draw an indicator on a map.
Of course Wazimap can’t do everything, so you can also download data into Excel or Google Earth to run your own analysis.
Wazimap is built on the open source software that powers censusreporter.org, which was built under a Knight News Challenge grant, and is a collaboration between Media Monitoring Africa and Code for South Africa.
Due to demand from other groups, we’ve now made Wazimap a standalone project that anyone can re-use to build their own instance: details are here.
How did it all begin?
Media Monitoring Africa approached Code for South Africa to build a tool to help journalists get factual background data on anywhere in South Africa, to help encourage accurate and informed reporting.
Code for South Africa is a nonprofit that promotes informed decision-making for positive social change, so we were very excited about collaborating on the tool.
Could MapIt be useful for your project? Find out more here
How exactly does MapIt fit into the project?
Mapit powers all the shape boundaries in Wazimap. When we plot a province, municipality or ward boundary on a map in Wazimap, or provide a boundary in a Google Earth or GeoJSON download, MapIt is giving Wazimap that data.
We had originally built a home-grown solution, but when we met mySociety’s Tony Bowden at a Code Camp in Italy, we learned about MapIt. It turned out to offer better functionality.
What level of upkeep is involved?
Wazimap requires only intermittent maintenance. We had municipal elections in August 2016 which has meant a number of municipal boundaries have changed. We’re waiting on Statistics South Africa to provide us with the census data mapped to these new boundaries so that we can update it. Other than that, once the site is up and running it needs very little maintenance.
What’s the impact of Wazimap?
We know that Wazimap is used by a wide range of people, including journalists, high school geography teachers, political party researchers and academics.
It has helped people understand not just where they work and live, but also that data can be presented in a way that’s accessible and understandable.
Code for South Africa has been approached a number of times, by people asking if they might reuse the Wazimap platform in different contexts with different data. Most recently, youthexplorer.org.za used it to power an interactive web tool providing a range of information on young people, helping policy makers understand youth-critical issues in the Western Cape.
We also know that it’s been used as a research tool for books and numerous news articles.
The success of the South African Wazimap has driven the development of similar projects elsewhere in Africa which will be launching soon, though MapIt won’t be used for those because their geography requirements are simpler.
What does the future hold?
As we’re building out Wazimap for different datasets, we’re seeing a need for taking it beyond just census data. We’re making improvements to how Wazimap works with data to make this possible and make it simpler for others to build on it.
Each new site gives us ideas for improvements to the larger Wazimap product. The great thing is that these improvements roll out and benefit anyone who uses it across every install.
Thanks very much to Greg for talking us through the Wazimap project and its use of MapIt. It’s great to hear how MapIt is contributing to a tool that, in itself, aids so many other users and organisations.
Need to map boundaries? Find out more about MapIt here
How is the data explosion transforming our world?
That’s the question that inspires the Big Bang Data exhibition, running from today until February 28 at Somerset House in London.
Alongside all kinds of data displays, data-inspired artwork and data-based innovations, the exhibition features our very own FixMyStreet and TheyWorkForYou as examples of websites that are using data for the common good.
The exhibits range from fun to thought-provoking to visually rather beautiful: we enjoyed Nicholas Felton‘s annual reports about himself, the Dear Data project, and innovative devices such as the fitness tracker for dogs. Most of all, of course, we enjoyed seeing our very own websites put into context and available for everyone to have a go with. 🙂
We’re delighted to have been included in this event, and we recommend a visit if you’re in the area. There’s plenty to keep you interested and informed for a good hour or two.
Users of Lothian Buses are more satisfied with the value for money of their bus journeys than anyone else in the country.
Passengers on the Oxford Park and Ride service find the seats the most comfortable. And the drivers of Trent Barton in Nottinghamshire give a friendly enough greeting, according to 95% of passengers.
These are the kind of insights that it’s now very easy to discover on the Passenger Focus website, thanks to the latest project by mySociety Services.
It’s an extension of the work we did last year to help the transport watchdog display their train satisfaction data. We’ve introduced a new design which, we hope, makes it much easier to explore the results of Passenger Focus’ annual passenger satisfaction survey.
We’ve used a new visual approach that is appropriate for the bus data: it makes it really easy to browse through 32 different survey categories, from cleanliness to safe driving.
When you have that many categories, drop-downs aren’t really an option, and we’re pleased with what we came up with to make it easy to make the most important categories prominent, while still allowing easy and intuitive access to the others.
We’ve also used responsive design, which means it performs beautifully whether viewed on mobile or at the desktop. Check it out for yourself here – be sure to resize your browser to see the mobile version kick in!
Not many people realise that we fund a proportion of our charitable work by carrying our commercial development and consultancy work for a wide range of clients.
Last year, we scoped, developed and delivered a real variety of digital tools and projects. Some of the projects were surprising. Some of them made us gnash our teeth, a bit, as we grappled with new problems. But all of them (and call us geeks if you like) got us very excited.
Here are just twelve of our personal high points from last year. If you have a project that you think we might be able to help you with in 2015, we’d love to hear from you!
1. We Changed the Way in Which Parliament Does Digital
This time last year, a small team from mySociety was poring over analytics, interview content and assorted evidence from Parliament projects dating back last 2-3 years, to help us put together a simple set of recommendations to conclude our review.
11 months later, Parliament have announced their first Head of Digital, fulfilling one of our key recommendations.
2. We helped the MAS and the FCA protect financial consumers
Two of our projects helped people financially.
We built the Money Advice Service’s (MAS) first responsive web application, the Car Cost Calculator.
This tool takes one simple thing you know (the car you wish to buy) and tells you roughly how much it’ll cost to run that car against any others you might be interested in. It has been one of MAS’ most successful online tools in terms of traffic and conversion.
We also built the Financial Conduct Authority’s Scam Smart tool, aiming to prevent financial scams.
This tool helps users considering a financial investment to check a potential investment. Users enter information about the type of investment, how they heard about it and the details of the company offering it to them and get back tailored guidance and suggested next steps to help them ensure the investment is bona fide.
3. We Gave Power to the People of Panama (soon)
Working with the The National Authority for Transparency & Access to Information (ANTAI) and the Foreign & Commonwealth Office (FCO), we set up our first government-backed instance of our Freedom of Information platform, Alaveteli, in Panama.
This project will ensure that Panama’s FOI legislation is promoted and used, but it will also shine a light on ANTAI, who are responsible for ensuring ministries and organisations publish their information, and handling case appeals.
4. We Mapped All the Public Services in Wales
After we extended the Mapumental API to produce data output suitable for GIS (geographical information systems), the Welsh Government were able to map public services in Wales for their Index of Multiple Deprivation calculations.
Over the course of the year they have calculated travel times for over seventy thousand points of interest.
5. We Launched a New Organisation in Four Weeks
Simply Secure approached us in dire need of a brand, an identity and a website to accompany the launch of their new organisation to help the world build user-friendly security tools and technologies.
Cue four weeks of very intense work for mySociety’s designer, supported by members of the commercial team. And we did it.
6. We Printed Stuff BIG (and found people jobs)
mySociety developer Dave Arter figured out how to generate A1 sized maps from Mapumental for every job centre in the UK – all 716 of them.
Xerox will be using these with the DWP to help job seekers find work that is within reach by public transport. As a byproduct, Mapumental now handles high-fidelity print based outputs: get in touch if that is of interest.
7. We Opened Up Planning Applications
With Hampshire County Council we had the opportunity to build a new application to help assist members of the public and business better understand what was happening around them. For us, it was also the first application in which we worked closely with a provider of a linked data store, in this case Swirrl.
When Open Planning goes live, it will look to help improve social engagement and the economy of Hampshire through better understanding and transparency of planning data.
8. We Proved (Again) That FixMyStreet Isn’t All About Potholes
After a spate of cyclists’ deaths in London last year, we felt that the moment was right to build something that would support cycle safety in the UK.
We launched Collideoscope on October the 7th with our first sponsor—Barts Charity, with the aim of generating data both on incidents involving cycles, and near misses.
9. We Helped Launch a Film
We built a tool for the British Museum, to go alongside the general release of Vikings Live. The Norse Names project brought a sense of context and personalisation to a dataset gathered by the University of Nottingham.
10. We Made Data More Exciting
In 2013, we built an interface to help people explore the data in the National Rail Passenger Survey (NRPS) data explorer for Passenger Focus.
This year, they asked us to build something similar for bus users. We’re entering the final week of development now, and the finished product should be launched in March.
The main aim of this site? To take data that could be considered pretty dry, and make it a lot more engaging.
11. We Fixed Yet More Potholes
This year Warwickshire, East Sussex, Hart & Harrogate joined the list of councils using FixMyStreet as their main street fault reporting platform.
That means that residents of those places can now make their reports direct from their council’s website, or via FixMyStreet, and either way they’ll have all the benefits of FixMyStreet’s smooth report-making interface.
12. We Showed Parliament the Way
And so, we end where we began. While Parliament were busy interviewing candidates for their new ‘Head of Digital’ position, we were commissioned to demonstrate what Hansard might look like were a platform like SayIt used instead of the largely print-based publishing mechanisms used today.
The result was shared internally. While SayIt may not be the end solution for Parliament, it’s great to have had some input into what that solution might be.
And in 2015…?
Got a project that you’d like us to be involved in?
Get in touch and tell us about it.
Eggs: Craig Sunter; Parliament: Greg Dunlap; Bubble car: Allen Watkin; To the Trains: Nic McPhee; Potholes: Tup Wanders; Parliament Duncan Harris. All Creative Commons.
Note: This post is a work in progress, I need your help to improve it, especially with knowledge of non-English sites
I was recently in Washington DC catching up with mySociety’s soul-mates at the Sunlight Foundation. As we talked about what was going on in the field of internet-enabled transparency, it came clear to me that there are now more identifiable categories of transparency website than there used to be.
Identifying and categorising these types of site turns out to be surprisingly useful. First, it can help people ask “Why don’t we have anyone doing that in our country?” Second, it can help mySociety to make sure that when we’re planning ahead we don’t fail to consider certain options that be currently off our radar. Also, it gives me an excuse to tell you about some sites that you may not have seen before.
Anyway, enough preamble. Here they are as I see them – please give me more suggestions as you find them. As you can see there’s a lot more activity in some fields than others.
1. Transparency blogs & newspapers – At the technically simplest, but most manual labour-intensive end of the scale is sites, commercial and volunteer driven, whose owners use transparency to help them to write stories. Given almost every political blog does this a bit, it can be hard to name specific examples, but I will note that Heather Brooke is the UK’s pre-eminent FOI-toting journalist/blogger, and we’ve just opened a blog for our awesome volunteers on WhatDoTheyKnow to show their FOI skills to an as-yet unsuspecting public.
2. What Politicians do in their parliaments – These sites primarily include lists of politicians, and information about their primary activities in their assemblies, such as voting or speaking. This encompasses mySociety’s TheyWorkForYou.com, Rob McKinnon’s one man labour of love TheyWorkForYou in NZ, Italy’s uber-deep OpenPolis.it (6 layers of government, anyone?), Germany’s almost-un-typable Abgeordnetenwatch, Romania’s writ-wielding IPP.ro, Josh Tauberer’sGovTrack.us, plus the bonny bouncing babies OpenAustralia and Kildare Street (Ireland). Of special note here are Mzalendo (Kenya) who unlike everyone else, can’t reply on access to a parliamentary website to scrape raw data from, and Julian Todd’s UNDemocracy (International), that has to fight incredible technical barriers to get the information out.
3. Databases of questions and answers posed to politicians – These sites let people post politicians questions, and the publish the questions and answers. The Germans running Abgeordnetenwatch (Parliament Watch) seem to have had considerable success here, with newspapers citing what politicians say on their site. Yoosk has some politicians in the UK on it, too.
4. Money in politics – This comes in two forms, money given to candidates (MAPlight), and money bunged by politicians to their favourite causes (Earmark watch). In the UK, as far as I know, the Electoral Commission’s database remains currently unscraped, perhaps because the data is so ungranular.
5. Government spending – where the big money goes. In the US the dominant site is FedSpending.org, and in the UK we have ukpublicspending.co.uk.
6. Websites containing bills going through parliament, or the law as voted on – This includes the increasingly substantial OpenCongress in the US which saw major traffic during the Health Care debates, and the UK government’s own Acts database and Statute Law Database. Much of the legal database field, however, remains essentially private.
7. Services that create transparency as a side effect of delivering services – Our own sites lead the way here: FixMyStreet‘s public problem reports and WhatDoTheyKnow’s FOI archive are both created by people who aren’t primarily using the site to enrich it – they’re using it to get some other service.
8. Election websites – These come in many forms, but what they have in common is their desire to shed light on the positions and histories of candidates, whether incumbents or new comers. The biggest beast here is Stemwijzer (Netherlands), probably in relative terms the most used transparency or democracy site ever. However these sites are popular in several places, the big but highly labour intensive VoteSmart (US), Smartvote.ch (Switzerland), plus others. mySociety is shortly to start to recruit constituency volunteers to help with our take on this problem, keep an eye on this blog if you want to know more.
9. Political document archives – This is a new category, now occupied by Sunlight’s Partytime archive for invitation to political events, and TheStraightChoice, Julian Todd and Richard Pope’s wonderful new initiative for archiving election leaflets and other paper propoganda.
10. Bulk data – Online transparency pioneer Carl Malamud doesn’t do sites, he does data. Big globs zipped up and made publicly available for coders and researchers to download and process. The US government has now stepped into this field itself with Data.gov, doubtless soon to be followed by data.gov.uk.
Please don’t shoot me if I’ve missed anything here, the world is a big place. But I thought that was a useful and interesting exercise, and I hope you’ll both find it useful, and help me improve it too. Comment away.