It’s been a few months since we first announced our Democratic Commons project under the banner of “shared code, data and resources where anyone can contribute, and anyone can benefit” — but if we’ve been silent since then, it’s certainly not for a lack of activity.
Quite the reverse, in fact: we’ve been busy bringing new team members on board and getting stuck in with the time-consuming and often fiddly process of data gathering and sharing.
When we’re in the midst of all this hard work, it’s sometimes hard to remember to talk about how everything’s going; but it’s always interesting, so here’s a snapshot of where we are now.
Those of us working on Democratic Commons are only a small team within the smallish organisation mySociety. Gathering in-depth data on politicians all around the world takes more time and more local knowledge than we have ourselves, so we’re working with partners located within our target countries.
Distintas Latitudes have been handling Latin America – they’ve been great at gathering data and explaining the various differences between the political systems in each country we’ve worked together on.
In India, Factly and Gender And Politics have done the most amazing job in gathering a full national and state level dataset for politicians right across the country. We were astounded, as that is a LOT of data (over 3,500 records and counting so far).
And in South East Asia we’re working with OCF, with whom we’ve had a long association (you may remember TICTeC Taiwan, for example). OCF have helped us with data for Taiwan and South Korea so far, and are set to work with us on seven more countries before December 2018.
Finally, a special mention goes to OpenLeb of Lebanon, who are working hard to start finding data in a country where data is not usually open. We genuinely could not do this work without our partners and we are eternally grateful for their help.
As is probably clear from the above, we often select which countries to work on by our ability to find a community or organisation that will extend help. A nice side effect of this is that we’re strengthening the connections and bonds between mySociety and organisations with similar missions in many different places.
Growing the community of such organisations across the world is going to be the primary focus of our new Community Manager Georgie, whom you will no doubt hear a lot from over the next few months.
She’s going to be finding out who’s already using data like this, who’s maintaining it, who’s interested in running projects with it or doing research — and seeing if there’s also an appetite there to keep the data up to date. This is because the data will really only be useful to people if it’s well maintained and current!
Working with Wikidata
Early on, we recognised that improving the political data available in Wikidata, rather than ringfencing it all within EveryPolitician, was going to be an efficient way to maximise the benefits of the Democratic Commons project.
What does this mean in practice? Well, in our first phase we’ve targeted 13 places in which to locate the data and load it into Wikidata: Mexico, Brazil, Colombia, Paraguay, Chile, Canada, Italy, Estonia, Lebanon, India, S Korea, Taiwan and Hong Kong.
Ultimately, we want to help make any and all information about politicians at every level freely and openly available via the Commons; but for now, our initial scope looks at representatives at national, regional, and city legislatures.
Now that a lot of this work is being done on or in Wikidata, we’re creating tools to make processes smoother and faster. The main ones of these are around verifying data and creating statements in Wikidata; we hope that when we’ve completed these they’ll be valuable to the whole Wikidata community beyond just the Democratic Commons project.
Step by step
We’re focusing on getting what we’re terming ‘Outline Data’ for each place loaded into Wikidata first. This type of data helps us model the political system, as it tells us what the legislature looks like — for example, whether it is unicameral, bicameral or different to those; what it calls members of the legislature; what term the legislature is on and how long that lasts; and often how many seats that legislature contains.
Once we have that outline data, we then need some information about people holding seats in those legislatures. We try and start with five examples of each type of role at each level, then we can send this ‘Seed Data’ off to hopefully crowdsource the rest of the data: more on that in a bit!
Meanwhile, our GIS expert Will is working on boundaries. Boundary data is hard — like, really hard! This is one of the most challenging areas of the project but it’s also one of the most important. Without electoral boundaries we don’t know what area a politician is representing and a lot of the tools we think this data would be useful for just won’t work.
However, boundaries aren’t always released openly or completely, especially when it comes to local level constituencies, and even when we do find them, understanding whether we have all the data we need to represent politicians correctly can be really tricky.
Because we like to keep really busy, we’ve also been starting to collaborate with other organisations such as Open Knowledge and CLEA on how to raise the visibility of the availability (or lack, more likely) of open sources of official boundary data.
Working with Facebook
You may remember our ongoing work to connect Facebook users with their politicians after an election, in countries around the world.
We’re also working with Facebook to run some crowdsourcing experiments that will gather more data on politicians. I mentioned ‘Seed Data’ above. For each country, this gets fed to Facebook, and allows them to create questions which they can send to users to ask them who their representatives are at different levels of government.
We then get this data back and our partners help us verify it and put it into Wikidata so the it becomes open and available for anyone to use. Facebook has a reach we would never be able to manage on our own.
So that’s where we are
As I’ve hopefully demonstrated in this post, the work is extremely challenging. That’s why we’re sometimes a little slow in updating where we’ve got to — but we genuinely believe that that having this data out there in the open will pave the way for so many exciting new political data-based projects and research. And so, onwards!
Image: Ben White (Unsplash)
As you’ll remember from our previous blogpost, the Global Legislative Openness Week — AKA GLOW — provided us with a great opportunity to support events and spread the word about our ambitious Wikidata project with groups around the world.
In the end we sponsored nine events (in Slovenia, Sweden, Croatia, Bulgaria, Italy, Greece, Spain, Wales and, amazingly, Nepal) and sent representatives to support seven of them in person. This meant that Tony, the lead for the EveryPolitican project, had an interesting 10 days getting the most out of budget airlines and managing to attend six of the events. He would have liked to have made it seven but EasyJet don’t go to Nepal! Also special thanks to Lucas who represented the project in Greece and Bulgaria taking a little of the pressure from Tony.
— OBC Transeuropa (@BalkansCaucasus) November 27, 2017
Five of the countries (Croatia, Greece, Italy, Slovenia and Spain) moved from very little or no coverage of political data in Wikidata to our two-star (or better) indicator — this is a (very) rough guide to how good the data is in a country; we are tracking what information already exists for the primary house of a national legislature which you can learn more about on our Wikidata project page. Wales was already a ‘three-star’ country and the others are coming along nicely.
— Open Knowledge NP (@okfn_np) December 1, 2017
We couldn’t be happier with the response and contributions made to the Democratic Commons by the participants — and we’re extremely keen to do more events early next year, collaborating with Wikimedia communities to make this happen. If that sounds interesting take a look at our original blogpost about the events and get in touch if it seems up your street.
Ahora @tmtm nos introduce al wikiproyecto @everypolitician, toda la info aquí >> https://t.co/8w1EhNNd9Q ¡En esta sesión vamos a añadir los primeros datos españoles! @mySociety @medialabprado pic.twitter.com/NyKX3AVmLG
— Wikimedia España (@wikimedia_es) November 29, 2017
All in all these events really felt like a culmination of all the learnings and activities undertaken in the project so far. The strides we have taken in understanding the best way to model this kind of data in Wikidata and the tools that have been built are what made it possible to make such a big impact in each of the countries in just a day or two. Not all of these lessons were easy to learn but they are really starting to pay dividends now.
We really want to keep this momentum up and build even more relationships with Wikimedia communities who are interested in contributing to the Democratic Commons in their countries so I can’t reiterate it enough – please do get in touch if you would like to get involved.
One more thing
If you’ve read this with great interest to the very end, then you are just the sort of person who we’re looking for! We currently have a vacancy for a Political Researcher to help us kickstart this kind of work in up to 100 countries, supporting our ambitions in the Democratic Commons project. See the job description here.
If you know your way around Wikidata, we’d love you to join in with the global string of events taking place for GLOW next week.
We’re very keen to get as many people as possible helping to improve the quality of Wikidata’s information on politicians. Why? Well, let’s take a quick look at a recent story that hit the news.
A new Bundestag
With Germany’s new parliament gathering for the first time on October 24, der Spiegel took the opportunity to examine their male-to-female balance, in the context of legislatures across the world. At around 31% female, they noted, the Bundestag now sits at the better end of the scale: parliaments almost everywhere are male-dominated.
How were they able to make such an assessment? As they note at the foot of their article, they used data on politicians’ gender from our EveryPolitician project.
A further exploration looked at age — they discovered that on average their parliamentarians were very slightly younger than in previous years — and they note as an aside that here in the UK, we have in Dennis Skinner the oldest MP in Europe, while Mhairi Black is the second-youngest by a whisker.
These are the kind of insights we seek to increase through our work with Wikidata as we help to boost the quality of their politician data: we consider such analysis not only interesting, but important. Whether or not countries wish to encourage fair representation across age groups and gender — not to mention many other categories — their decisions should at least be based on facts.
As things stand, there are only a handful of countries where data is good enough to be able to make such comparisons: in our vision, journalists, researchers — and anyone else — will be able to turn to Wikidata to find what they need. The forthcoming Global Legislative Openness Week (GLOW) gives us all an opportunity to put a rocket under the quality and quantity of data that’s available to people making analyses like these, that stand to benefit us all.
How to get involved
GLOW runs from next Monday until the 30th November, and we’re encouraging people — wherever you live in the world — to get together and improve the data on national-level politicians for your country.
We’re already expecting a good number of groups to run events. Get-togethers are confirmed in Slovenia, Bulgaria, Italy, Greece, Spain and more — once final details are firmed up, we think there’ll be action in other countries across the globe. Now how about you? As we said in our post last month, a concentrated effort from a small group of people can really make a difference.
We’re especially keen to encourage folk who have some experience of contributing to Wikidata: we reckon that, for this particular drive, you need to already know your way around a bit. So if that’s you, do come forward!
Start by having a look at this page, which outlines what we hope to achieve; we’ll be adding more detail this week too. You can add your country to the list if you’d like to, or explore what’s missing in the data of those countries already listed.
Or, if Wikidata’s all new to you, why not put out some feelers and see if there’s anyone who can show you the ropes while you work together? One good way is to see if there’s a Wikimedia User Group local to you.
What exactly will you be doing?
Here’s a bit more detail on what a workshop will look like.
The idea is to improve information in Wikidata about members of your country’s legislature. The ‘Progress Indicators’ on this page will give you guidance: typically you’ll be working through tasks like adding any missing “position held” statements and biographical data. We’re asking folk to prioritise current politicians, with information for historic members an added bonus if time permits.
Once sufficient data is available in Wikidata, the real fun begins! Your workshop attendees will be able to query the data to answer questions such as:
1) Can the gender breakdown and average age of members of the current legislature be calculated?
2) Can that be broken down per political party/group, or (where appropriate) by region?
3) Can you compare those figures for the legislature vs. the cabinet?
4) How far back can you generate those for?
And if the ideas start to flow, building queries and visualisations to answer other questions will also be very useful.
Let us know if you have any questions before the week begins — we’re going to be very busy during GLOW, but we’ll do our absolute best to help.
Image: Alex Iby (Unsplash)
Global Legislative Openness Week (GLOW) celebrates open, participatory legislative processes across the globe.
Back in 2015 we marked GLOW by setting up a challenge: could we get politician data for 200 countries up on EveryPolitician within the week, with the help of the global community? The answer was a resounding yes, and the challenge was a massive success. We ended up with data for 201 countries in the end, thanks to help from awesome people from all over the world.
This year, we’re running another challenge: to get as many Wikidata workshops focusing on political data to happen during GLOW as possible.
Fancy helping with this challenge? Read on…
This is all part of our Wikidata/EveryPolitician project.
The project aims to improve political data in Wikidata, so that it can be used more easily for projects, research or investigations that hold politicians to account. Examples of where good political data is vital include in parliament-tracking websites (like in Zimbabwe, Nigeria and Ukraine) and in cross-border journalistic investigations (like the Panama Papers).
Providing this data in consistent and structured formats across countries means the people running these accountability projects spend less of their time gathering the data and more on actually using and interpreting it, to keep tabs on those in power. This project is all part of our mission to contribute to the Democratic Commons.
One of the best ways to improve and use political data on Wikidata is to get people together in person to work on their country’s data. So, that’s the aim with this latest GLOW challenge, and we’d love for as many groups around the world to host Wikidata workshops as possible!
The aim of these Wikidata workshops is to:
- Improve political information in Wikidata so that developers, researchers and journalists (or anyone!) can use the data in their investigations and accountability projects.
- Use and query existing political data in Wikidata to see what interesting questions can be answered when data is available in consistent and structured formats.
Workshop attendees will go away with:
- Increased knowledge of how Wikidata works and how to contribute to it
- A better understanding of why good political data is so vital and how it can be used
- New connections to the global community of people who care about accountability issues
- A warm, fuzzy feeling of satisfaction that they’ve helped with the global accountability movement 😉 (We hope so anyway!)
Not sure what such an event might look like? Read up on our recent Wikifying Westminster workshop: it really showed us how much can be done when a few people get together in a room.
Funding and support available
Thanks to the Wikimedia Foundation, we’re able to offer some support to individuals/groups who are interested in running Wikidata workshops like these during GLOW. This will differ on a case by case basis but includes:
- A small amount of funding to help cover event costs
- A review of your country’s existing political information in Wikidata and some pointers about possible next steps
- Ideas for how you and your attendees can:
a) Use the data for interesting research and projects, and
b) Improve the data for future research queries/projects
- In-person support during your event – if you’d like one of our EveryPolicitian/Wikidata team to come to your event to present and participate, we can do this (if our budget allows!)
- Access to a dedicated Slack channel which connects you with other groups around the world who are also running events during GLOW.
Workshops can take place at any time within GLOW week, which is from 20th-30th November 2017 (yes, that is a long week!).
So, if you’d like to be part of this global challenge to improve and use political information in Wikidata, we’d be thrilled to hear from you. Please do get in touch: firstname.lastname@example.org
Earlier this month, Mark laid out the concept of a Democratic Commons for the Civic Tech community: shared code, data and resources where anyone can contribute, and anyone can benefit.
He also talked about exploring new models for funding the kind of work that we do in our Democracy practice at mySociety.
For many years, our Better Cities work has been proof of concept for one such model: we provide data and software as a service (FixMyStreet, MapIt, Mapumental) to paid clients, the revenue from which then funds our charitable projects. Could a similar system work to sustain our Democracy practice?
That’s the hope, and with Facebook who we first worked with during the UK General Election in June, providing the data that helped people see and connect with their elected representatives, we’ve already seen it in action.
This kind of project is positive on multiple levels: it brings us an income, it brings the benefits of democratic engagement to a wider audience than we could reach on our own, and it contributes data back into EveryPolitician and Wikidata, that everyone can use.
The UK election was only the first for which we did this work: we’ve gone on to provide the same service for the French elections and more recently for the rather more eventful Kenyan ones — currently on hold as we await the re-run of the Presidential election next month. And now we’re doing the same for the German elections, where candidate data is being shared this week.
As we’re learning, this is definitely not one-size-fits-all work, and each country has brought its own interesting challenges. We’re learning as we go along — for example, one significant (and perhaps obvious) factor is how much easier it is to work with partners in-country who have a better understanding of the sometimes complex political system and candidates than we can ever hope to pick up. Much as we might enjoy the process, there’s little point in our spending days knee-deep in research, when those who live in-country can find lists of candidates far more quickly, and explain individual levels of government and electoral processes far better.
Then, electoral boundaries are not always easy to find. We’ve used OpenStreetMap where possible, but that still leaves some gaps, especially at the more granular levels where the data is mainly owned and licensed by the government. It’s been an exercise in finding different sources and putting them all together to create boundary data to the level required.
Indeed, that seems to be a general pattern, also replicated across candidate data: at the national level, it’s easy to find and in the public domain. The deeper you go, the less those two descriptors hold true. It was also at this point that we realised how much, here in the UK, we take for granted things like the fact that the spelling of representatives’ names is usually consistent across a variety of sources — not always a given elsewhere, and currently something that only a human can resolve!
What makes all the challenges more worthwhile, though, is that we know it’s not just a one-off push that only benefits a single project. Nor is the data going straight into Facebook, never to be seen again.
Much of what we’re collecting, from consistent name data to deep-level boundaries data, is to be made available to all under open use licenses. For example, where possible we can submit the boundaries back to OpenStreetMap, helping to improve it at a local granular level across whole countries.
The politician data, meanwhile, will go into Wikidata and EveryPolitician so that anyone can use it for their own apps, websites, or research.
There are also important considerations about how this type of data will be used and where and when it is released in the electoral process; finding commercial models for our Democracy work is arguably a more delicate exercise than on some of our other projects. But hopefully it’s now clear exactly how a project like this can both sustain us as a charity, and have wider benefits for everyone — the holy grail for an organisation like us.
At the moment it’s unclear how many such opportunities exist or if this is a one-off. We’re certainly looking for more avenues to extend the scope of this work and keen to hear more ideas on this approach.
Your contributions help us keep projects like EveryPolitician up and running, for the benefit of all.Donate now
Last Saturday (August 19th) at Newspeak House in London, mySociety and Wikimedia UK held the “Wikifying Westminster” workshop, a day-long event to encourage people to get involved with Wikidata, but also to give a taste of what people can build with the data that is already there.
The vision: one day, complex investigations which currently take researchers a lot of time, such as “how many MPs are descended from people who were also MPs” or “how many people named X were MPs in year Y”, will be answerable with data from Wikidata using a single SPARQL query…
…but we’re not quite there yet. Currently, some data is scattered all over separate databases (which sometimes get shut down or disappear); some is just plain missing; and most frustrating of all, some is in place but there’s no apparent way to get it out of the database.
In order to make this vision a reality, we need to experiment with the data, find ways to check how complete it is, and explore what questions we can currently answer with it. Events like Wikifying Westminster are the perfect opportunity to do just that.
After a brief introduction to Wikidata and the EveryPolitician project, we split into two groups: one focused on learning how to use Wikidata, while the other focused on working on mini-projects.
Here’s a taste of what happened…
The learning track began by introducing new users to the basic Wikidata editing principles (or “getting data into Wikidata”). Participants were able to put their new skills into action immediately, by adding missing data on British MPs, who were mostly lacking dates and places of birth.
By the end of the first session, good progress had been made, particularly on obtaining dates of birth for current British parliamentarians. For some reason, though, it proved much harder to find these for women than for men: we can only speculate as to why that might be (do some still adhere to the idea that a woman shouldn’t reveal her age?!).
We were also given an introduction to SPARQL, a language used to query information on databases (or “getting data out of Wikidata”). Lucas Werkmeister introduced the Wikidata query service and explained a few tricks to help with using it. Participants were later able to put this to the test by running progressively difficult test queries such as “All current UK MPs” or “Who is the youngest current MP?”
Also, Navino Evans showed us the potential of reusing data, talking about Histropedia, which he co-created with Sean McBirnie. Histropedia is an awesome tool that lets you visualise thousands of topics on interactive timelines: you can browse through existing ones or create a new one from scratch.
This group both worked on improving data and looked at how well we could answer some simple “stepping stone” queries (i.e. small questions to which we already knew some of the answers) as a heuristic of how good the data in Wikidata already is. You can see and contribute questions to the list of test queries here.
Some more details:
Improving data. The focus here was on the Northern Ireland Assembly, for which Wikidata now has full membership history back to the foundation of the Assembly, and on adding academic degrees of cabinet ministers. Starting from an excellent spreadsheet of the undergraduate universities and subjects of UK politicians and ministers (going back to John Major’s cabinets), we tried to upload that data on the relevant items, adding the qualifier “academic major” (P812) to the property “educated at” (P69). In this case, the key problem we found was that we weren’t sure how to model when people did joint subjects, like “Maths and Politics”, convincing us to concentrate on the more obvious subjects first.
Answering some unusual and/or intriguing questions. Inspired by a prior finding that there are more FTSE 100 CEOs named John than there are female ones, and that John is historically the most common name of UK parliamentarians, we thought we’d find out when exactly the John-to-female balance was toppled amongst the UK’s MPs (hint: not until 1992).
Going back further in history, we queried the first time each given name was recorded in Parliament, this was inspired by a recent news article about an MP who claimed he was the first “Darren” in the Commons.
Some ideas were also born that we weren’t able to see through, for various reasons. For example, could we discover which, if any, MPs are descended from people listed in the UCL’s ‘Legacy of British Slave-owners’ database? An interesting question, but at the moment, the answer is ‘no’, partly because child-parent relationships are currently inconsistently modelled in Wikidata, and partly because of the nature of Wikidata and ancestry: if there is someone who doesn’t exist in Wikidata (e.g. Grandad Bob, the painter) in the family chain, Wikidata can’t bridge the gap between a present day MP and the slave owner who might be their ancestor.
This is just the beginning
Work, of course, is still ongoing: all pre-1997 UK data is still to be inserted or improved on Wikidata, and so much more is missing – family connections, academic degrees, links to other databases, and all sorts of “unusual stuff” that can be used for interesting queries.
This data is crucial if we want to be able to answer the really big questions which Wikidata should one day be capable of helping us explore, about what politicians do.
We can do that together!
We hope that events like this give people an easy way in to Wikidata and also show them what’s already possible to achieve with the data. Over the coming months, we are hoping to support more events of this type around the world. If you are interested in getting involved, here’s how:
- Want to improve your country’s data? Events like this can be a great way to help kickstart activities and find other people who share your goals. We are happy to help out and support people in other countries to do so.
- Are you already organising or planning to organise a similar workshop around Wikidata? Make sure it is listed on the Wikidata Event page!
- Do you want to attend future workshops? Follow us on Twitter to stay updated about events that we are running, and ones that other people are too!
We’re also always looking for feedback and suggestions on workshop and event formats that might also work. Have you already run similar workshops? Let us know your impressions and suggestions on email@example.com!
Feature image credits: Mark Longair
This post is by Tony Bowden and Lucy Chambers from the EveryPolitician team. Today we officially launch our collaboration with Wikidata – here’s what to expect…
The story so far
You might have been following the progress: since 2015, through our project EveryPolitician, we’ve been gathering data for every national legislature in the world, from thousands of sources, and sharing it.
Now, two years on, we’ve started to see some great results. For example:
- It’s much easier to build simple Parliament Tracking sites (as mySociety partner organisations have in Zimbabwe and Nigeria). Those running the sites can work on providing information and context to hold politicians accountable — and don’t have to worry about wrangling data and software.
- Tools that allow citizens to write to their representatives (like Majlis Nameh in Iran, based on the WriteInPublic software, or Oxfam’s UK and Australia Campaigning Tool) can now be deployed in days rather than weeks, allowing groups to focus on local customisation.
- Vote-tracking sites like TheyVoteForYou in Australia can be more easily adapted to other countries — such as Ukraine, again without worrying about the major task of sourcing the politician data.
- Projects that highlight politicians’ activity can be augmented to show extra information against those politicians — so for example, Politwoops can show party affiliation next to politicians’ tweets.
- Investigative journalists can cross-match our lists of politicians to other sources (e.g. investigating shell companies for a recent Private Eye article or with the Panama Papers).
- mySociety’s collaboration with Facebook makes it easy for people to connect with their newly elected representatives.
All of this is possible because of two main tenets of EveryPolitician: having gathered the data, we structure it consistently, and we share it freely.
Not just current data
We’ve also discovered that there is a huge value for everyone in retaining historic information, too:
- Old data tends to disappear from many official sites, for example, when a new government comes into power and / or when parliaments decide to remodel their websites.
- Sometimes even the official sources no longer exist: for example, in Burkina Faso the Parliament building was burned down by protesters in 2014 and many crucial documents were lost.
- Research projects and sites run by civil society organisations sometimes run out of funding and have to shut down — meaning that the data people were relying on can vanish overnight.
The future: going one step further
So that’s all great, but we think EveryPolitician can do still more to help the worldwide community of Civic Tech coders and activists.
In particular, we want to go beyond data showing who the politicians are, and also provide information on what they do. That’s because we can see real value in the ability to answer questions like:
- When we look at politicians who vote on issues such as gay marriage, smoking bans, tax on sugary drinks, etc, anywhere in the world — are there any broad correlations like age or gender?
- As countries elect more women MPs, do those women also gain equivalent representation in committees? Does this affect attendance and participation rates?
- Are there standard political career paths that can be observed anywhere in the world? So for example, do certain Cabinet positions limit future progression; and if so, would it be feasible to spot politicians who are on the way up, or on their way out?
- Do politicians who move from the lower to upper house act, and vote, differently?
- Do politicians change their voting activity after certain types of intervention, for example after receiving funding from oil companies?
… and there are undoubtedly millions more questions, each one just as interesting and with answers that could enrich our understanding of the world. We are aiming for a future in which each should be answerable within minutes, rather than form the basis of a multi-year post-graduate research project.
How we plan to get there
You may remember our recent proposal to integrate more deeply with Wikidata. We’re delighted to say that our proposal was accepted, and that makes EveryPolitician’s path very clear.
Wikipedia is fast becoming one of the best sources of political information in many countries: it’s often updated more quickly than major news outlets or official parliamentary sites are.
Which is great, but there’s an issue when it comes to using that information for projects like the ones we’ve mentioned above: Wikipedia contains largely unstructured information (that is to say, information that comes in a wide variety of different formats) — as you’d expect from any project with multiple contributors and, often, a free-text input.
There are also a lot of differences between Wikipedias in different countries. Some countries’ data (particularly countries where the Wikipedia community is larger) gets updated very quickly. Smaller language Wikipedias can’t rely on such a large pool of editors, and it tends to be longer before they are updated.
Additionally, and as you might expect, Wikipedia content will appear fastest in the countries most directly affected by the change being documented, so for example when there are elections in Estonia, the Estonian Wikipedia may show the results almost immediately, but it can take a while for those changes to trickle into all languages.
But of course, nothing’s ever quite that simple. While Wikipedia has a wealth of unstructured political information, on Wikidata there’s still an awful lot of data missing. You may recall that we recently ran a drive to ensure that every country at least had its head of government entered, but that’s just the beginning: in order to answer the kinds of questions we mention above, we really need to ensure there’s consistently-structured information for all legislators, all elections… and much more.
What you’ll see in the coming months
While we plan for EveryPolitician to retain its own identity and keep its own front end, we’re excited to say that over the next few months, we’ll be teaming up with Wikidata communities across the world.
Our first objective is see if we can bring Wikidata up to the same level as EveryPolitician for as many countries as possible.
And once we hit that target, we plan to go much further. The beauty of Wikidata is that you can add pretty much any information you want to add to politicians (indeed, to anything!), so local communities can decide for themselves which information is most pertinent and make sure that it’s included.
To help us make all of this happen, we’re expanding. If you’ve read this far and you’re still finding the project interesting, the chances are you’d be a great addition to the team! We’re looking for a new community co-ordinator, working from anywhere in the world (compatible timezones allowing): you can see all the details here.
How you can get involved
There are plenty of ways that you can help with this drive to improve the structured information on Wikidata. Here are the obvious ones:
- There are active missions always ongoing on the Wikiproject Heads of State and Government. There’s one there now – check them out!
- Would you be interested in organising a push to Improve Wikidata for your country? Get in touch on firstname.lastname@example.org and we’ll do what we can to help.
- If this all sounds interesting, but you’re not sure where to start, or you’re unfamiliar with Wikidata – drop us an email and tell us what you’re interested in. We’re more than happy to help you get started.
Last you heard from from the EveryPolitician team we were talking about integrating EveryPolitician more deeply with Wikidata.
Here’s a quick update on what to expect in the coming months and a note on how you can get involved – we are going to need a lot of help!
Before we can make headway on a deeper integration, there are some pretty foundational pieces of the puzzle we need to put in place.
Some key data is missing, and you may be surprised to discover that it is pretty basic stuff. Take a look at this for example:
This is from a report we have generated to highlight gaps in the links between items. Notice how not even all of the offices have been filled in on the country page, let alone who occupies those offices?
This is the type of foundational data we are hoping to get in place over the next little while, and you can help…
How you can get involved
Over the coming weeks we’re going to be conducting a few experiments in public and trying to get data into Wikidata. We’ll need as much help as we can get!
The experiments will focus on using Wikidata to attempt to answer some questions we find interesting and see how we can expose gaps and inconsistencies in the data. In doing this, we’ll be pointing to specific reports we have generated and asking you to help us fill in the gaps.
Our first question will be:
“What is the gender breakdown of heads of government across the world?”
We’ll be blogging about the investigation over on Medium. The first post is already there:
Keen to dive straight in? Help us fill in the blanks you see in this report! If you are familiar with Wikidata, you will probably be able to get started straight away, but there are also tips and pointers in the Medium post if you want a bit more guidance.
You can get notified of the challenges to complete the data as soon as they are published by following the EveryPolitician bot on Twitter.
Want even more details on our plans?
We should also mention that we’ve updated our proposal to the Wikimedia Foundation to make a couple of things clearer about the problem we are trying to solve and our proposed solution (the proposal is still being reviewed). If you are interested in the nitty-gritty, that’s the best place to get the full overview of what we are planning.
Our EveryPolitician project makes data on the world’s politicians available in a useful, consistent format for anyone to use. If you’ve been following our progress, you’ll know we’ve already collated a lot of data (over 72,000 politicians from 233 countries). The work on adding to the depth and breadth of that data is ongoing, but EveryPolitician data is already being used to do interesting things.
Previously we looked at Politwoops as an example of EveryPolitician data being used to augment existing data.
In that case, the useful data for Politwoops was the politicians’ party affiliation. But our team (a handful of humans and one very busy bot) collects richer data than just that. EveryPolitician data includes contact information for politicians.
At mySociety, we know how powerful this particular kind of data can be. For example, our WriteToThem site makes it easy for UK citizens to contact their representatives (WriteToThem grew out of the earlier online service FaxYourMP, and uses the now more common technology of email).
Of course, there’s nothing especially radical about collecting email addresses of politicians… or phone numbers, Twitter handles, or Facebook pages. Indeed, many individuals and groups do just that. But an important difference with EveryPolitcian is that we’re not just collecting data (which happens to include those things, as well as a host of others) but also making it available so it’s easy to use. We do that by putting it out in consistent, useful formats.
For many projects, downloading a CSV of current politicians from EveryPolitician will be enough. That can be opened as a spreadsheet, and if one of those columns is called
Opening a spreadsheet is just one way of accessing the data. Our own use of EveryPolitician data to power the “Write in Public” MajlisNameh site for ASL19 (see this blog post for more about that) demonstrates a more programmatic approach.
But the whole point of making data available like this isn’t so that we can use it. It’s for other people, other groups. Anyone can build more nuanced or complex services with this data too.
For example, the people at Represent.me have built a sophisticated platform for gathering opinions and votes that can be shared with politicians and constituency MPs. It’s a system of information-gathering that has a network of citizens at one end feeding into their political representatives at the other. They use EveryPolitician’s data to populate their system with information about those representatives, including contact details, for each country they operate in.
And, because we make sure our data is consistently formatted, it’s a good general solution. As they cover more areas, they can expect the code they’ve written to ingest the EveryPolitician data in the countries they’re already operating in to also work as they expand into others.
If you’re running a project that needs such data, you could invest time and effort finding and collecting it all yourself. But it’s almost inevitable that you’d be using the same public sources that we are anyway — after all, we try to identify and use all the sources we can, merging them together into one, collated whole — so really it makes sense to simply take the data from EveryPolitician. Remember, too, that once our bot has been told about a source, it checks it daily for changes and updates too. So instead replicating the effort we’re already doing to gather the same data you need, you’re free to focus on developing the way your project uses that data… while we hunker on down and get on with collecting it.
Inevitably, as with all software projects, there’s always lots more to do, but already the value of providing useful data — and especially contact information — in a consistent format is clear.
Image: Telegraph Chambers (Montreal) CC BY-NC-ND 2.0 by Andre Vandal
Over the last two years, we’ve gathered data on the top-level politicians of almost every country in the world, and made it accessible to developers everywhere through our project EveryPolitician.
Now we’d like to take a step that we believe will benefit more people, and further extend the usefulness of this extensive dataset. We’re proposing to integrate more deeply with Wikidata, to fill the gaps in their coverage and provide consistent, linked data to their global community.
Wikidata is the central storage for the structured data each of its Wikimedia sister projects including Wikipedia, Wikivoyage, Wikisource, and others. Wikidata also provides support to many other sites and services beyond just Wikimedia projects so the combination of EveryPolitician’s data with the reach of Wikidata’s community is pretty compelling.
So in many places, the aims of the EveryPolitician and Wikidata projects are already aligned. We already synchronise EveryPolitician data with the good quality data available in Wikidata where we find it, and we feed back our own additions. As our datasets improve, it seems prudent to combine efforts, and resources, in one place.
If you play an active part within the Wikidata community, or are someone who would benefit from this initiative, we’d very much appreciate your support. Please do add your endorsements or thoughts at the foot of that page if you’d like to see the project go ahead.