1. Placing EveryPolitician on hold

    After over five years of active development we have decided to pause work on the EveryPolitician project for the foreseeable future.

    In this post we’ll outline where we are leaving things, how you can make use of the data that does exist, and how you might be able to help migrate or transfer some of what we’ve collected over to services like Wikidata.

    What’s in place today

    The EveryPolitician project is, as its name suggests, based on the simple idea to gather accurate and up-to-date data on every politician in the world, collated and shared in a consistent format for free download and use by researchers, democracy projects, campaigners and individual citizens.

    Over the course of the project we have gathered, structured and shared data on 78,382 politicians from 233 countries and territories presented on EveryPolitician.org via hundreds of scrapers run on morph.io and hosted on GitHub, producing the data on everypolitician-data.

    Mostly the data covers the main chambers of recent parliaments around the world, but it also includes thousands of entries for previous parliaments, in some cases going back decades.

    This has been a sizeable undertaking, involving a handful of very talented developers and colleagues within mySociety, as well as contributions from dozens of other organisations and individuals, many of whom make use of the data within their own projects.

    The reality is that this work is hugely time consuming, complex and requires not just expert knowledge but a commitment to go deep into the intricacies of parliamentary data in order to make it comprehensible to a wider group of users. And looking to the next couple of years this task is only ever going to increase in complexity — too much for one underfunded organisation.

    We therefore intend to freeze the current data as it currently stands, and it will continue to be available for download and reuse. We just can no longer commit to keeping this data up to date.

    Always playing catch up

    The challenge with data projects like EveryPolitician, beyond the complexity of understanding the structures and relationships within hundreds of individual parliaments (every parliament is an edge case in some way), is that the data is always steadily going out of date.

    Across the world’s national parliaments there is an election somewhere roughly once a week, and that’s often when parliaments choose to update their websites, sometimes breaking our scrapers and changing the format of the data. Throughout the life of a parliament you might expect a few percent of MPs to change, sometimes more in different systems, so keeping on top of those individual changes is a sizeable task – especially where errors or duplications occur.

    In addition to managing the hundreds of scrapers, we also included data from other sources — increasingly from Wikidata. Over the past 18 months we’ve been attempting to migrate more and more of what we’ve learned on EveryPolitician over to Wikidata via the WikiProject every politician.

    Where the project goes next

    EveryPolitician was built on the many years of work we had already delivered in this area, through PopIt, Poplus and working with Popolo. We knew what was needed, what worked and what didn’t.

    We saw the potential to create an Open Corporates for political data, and hoped that EveryPolitician would be able to attract grant funding to grow, and potentially develop appropriate commercial services in support.

    However, after five years of significant investment we just don’t have the funding to continue this work on our own.

    In time we hope to be able to continue to contribute again to the wider availability of political data, and with hindsight it’s clear that Wikidata should be the natural global home for this type of data – benefitting from much greater reach, the contribution of motivated individuals in each country, and from the wider Wiki community.

    As part of our contribution to Wikidata, we’ve created numerous tools to support the cross-referencing, verification, and supported update of data between EveryPolitician and the Wikiproject. This is still something of a work in progress, but we see it as a key way that others might contribute and take on aspects of the project in the future.

    In the meantime we hope that many people continue to make use of the wealth of data that’s already been collected.

    If you have a specific interest in a country, group of legislatures or some other combination, perhaps you can consider adding the kind of data that  EveryPolitician has collected to Wikidata. We have no further resources to devote to this work; however if you do have an interest in taking some of this on then we will try to advise what options might best suit.

    Image: Jelle van Leest

  2. EveryPolitician data plays a key role in campaigning! But we need more stories like this…

    As we’ve highlighted in recent posts, EveryPolitician is an open dataset.

    We’ve always been strong advocates of open data, but there’s no doubt that it come with its own challenges. For example, when data is freely and openly available, without even the need for registration, we have very little idea of who is accessing it. That, in turn, makes it hard to prove that the project is having impact…and subsequently to find funders to support the maintenance of the project.

    So we were fortunate that user research interviews for the Democratic Commons led us to Andrew from New/Mode. New/Mode deliver advocacy and engagement tools that are used by hundreds of the top campaign, nonprofits and advocacy organisations around the world.  

    These tools are connecting people to their representatives, so information is key: specifically, information on who politicians are and how to contact them. And that’s just what EveryPolitician is, in part*, providing for New/Mode’s tools which are used by groups in Australia, Canada, the US and the UK.

    We asked Andrew what impacts have been created through New/Mode’s tools, and he told us that:

    • In the UK, ONE’s supporters sent 6,500 emails to MPs over the space of a week, helping to successfully pressure MPs to vote for a Sanctions and Anti Money Laundering Bill that increases transparency and cracks down on global corruption.
    • In the US, Win Without War used New/Mode tools with EveryPolitician data to block a defence bill that would have given Trump more nuclear access. The Sunrise Movement is currently using New/Mode tools to push for swift action on climate change.
    • In Canada, Canadians for Justice and Peace in the Middle East prompted 16,000 emails to Canadian MPs in support of Trudeau’s comments condemning violence against unarmed Palestinian protesters.

    We need more of these stories to help us build a picture of who uses EveryPolitician and why it is important, to make a case for why we should keep working on it. As mySociety’s Mark Cridge outlined in a previous post, we’ve recognised that EveryPolitician can only become sustainable at scale as part of a wider community effort, which is why we are collaborating with Wikidata — but we still need the resources to do that.

    Any ideas, or suggestions, please let us know.

    *PS, In case you were wondering which APIs New/Mode uses, here is a breakdown:

    • Currently, Open North’s Represent is providing the bulk of the data for Canadian politicians. But senators’ data and Twitter handles for the MPs and senators are pulled from EveryPolitician.
    • For the US, Google Civic does a good job of providing the bulk of information, but again EveryPolitician is used for congressional fax numbers and to fill in any blanks with Google Civic data.
    • In the UK, New/Mode are using another mySociety tool, Maplt alongside EveryPolitician. EveryPolitician data is only available for the national level of politicians as yet.
    • For Australia where they focus on national politicians, the data is drawn from a mixture of Open Australia and again EveryPolitician.

    Photo by roya ann miller on Unsplash

     

  3. How academics are using Wikidata to look for links between legislative behaviour and the biographies of Members of Congress

    Earlier this year, we were fortunate enough to be contacted by Brian Keegan, Assistant Professor in Information Science at the University of Colorado Boulder, who specialises in the field of network analysis.

    Brian and his team were planning to mine the official biographies of every legislator published by the Library of Congress – going back to the first Congress in 1789 –  and add the information as structured data to Wikidata. Having heard of our involvement with  WikiProject Every Politician, they wanted to understand more about contributing.

    The research team, which included professors from the Libraries, Political Science and Information Science departments, planned to combine this biographical data with more common data in political science about voting and co-sponsorship, so that interesting questions could be asked, such as “Do Ivy League graduates form cliques?” or “Are medical doctors more likely to break with their party on votes concerning public health?”. Their hypothesis was that the biographical backgrounds of legislators could play an important role in legislative behaviours.

    However, the first big step before questions could be asked (or SPARQL queries made) was supporting undergraduate students to enter biographical data for every member of Congress (going right back to the first) on Wikidata. This has not generally made it into the datasets that political scientists use to study legislative behaviour, and as students began to enter data about these historical figures, it quickly became apparent why: non-existent nations, renamed cities, and archaic professions all needed to be resolved and mapped to Wikidata’s contemporary names and standardised formats.

    Nine months on, the team and ten undergraduates have revised over 1,500 Wikidata items about members of Congress, from the 104th to the 115th Congresses (1995-2018) and the 80th– 81st Congresses (1947-1951), which is 15% of the way through all members dating back to the first Congress in 1789!

    They started running SPARQL queries this summer.

    Joe Zamadics, a political science PhD student who worked on the project explained the potential of combining these data:  “One example we tried was looking at House member ideology by occupation. The graph below shows the ideology of three occupations: athletes, farmers, and teachers (in all, roughly 130 members). The x-axis shows common ideology (liberal to conservative) and the y-axis shows member’s ideology on non-left/right issues such as civil rights and foreign policy.  The graph shows that teachers split the ideological divide while farmers and athletes are more likely to be conservative.”

     

    The team are keen to highlight the potential that semantic web technology such as  Wikidata offers to social scientists.

    For the full Q + A with Brian and Joe see the mySociety Medium post. 

    Photo by Jomar on Unsplash

     

  4. How Global Witness is using EveryPolitician data to spot potential corruption

    Since 2015, mySociety have collected and shared open data on the world’s politicians via the EveryPolitician project. 

    And while we receive emails from across the world pretty much on a weekly basis, asking us to update a dataset, we still can’t say exactly who uses the EveryPolitician data, and for what purpose.

    This is largely because we want to place as few barriers as possible to using the data. Asking folk to fill in a form or register with us before they access data which we believe ought to be free and accessible? Well, that would be counter to the whole concept of Open Data.

    But that said, it’s really useful for us to hear how the data is being put to use, so we were very pleased when Global Witness sent us their report, The Companies we Keep.

    This fascinating read shares the results of their analysis of the UK’s Persons of Significant Control Register (PSC) in which Global Witness used EveryPolitician data to see if there are politicians who are also beneficial owners of a company registered to the UK.

    In order to interpret and compare datasets (which also included sources such as the Tax Justice Network Financial Secrecy Index), Global Witness and Datakind UK built two tools:

    • An automated system for red-flagging companies
    • A visual tool for exploring the PSC register and other associated public interest datasets

    The red flagging tool can be used to uncover higher risk entries, which do not indicate any wrongdoing but could be in need of further investigation… such as the 390 companies that have company officers or beneficial owners who are politicians elected to national legislatures, either in the UK or in another country.  

    The report also highlights some of the challenges faced by Companies House that prevent the register from fulfilling its full potential to help in fighting crime and corruption. We recommend a full read: you’ll find it here.

    It is very helpful for us to demonstrate the uses of EveryPolitician data, both for our own research purposes and to enable us to secure the funding that allows us to go on providing this sort of service.

    If you have or know of more examples of the data being used, please get in touch with me, Georgie. And if you value open, structured data on currently elected politicians, you should get involved with the Democratic Commons; this is a developing a community of individuals and organisations working to make information on every politician in the world freely available to all, through the collaborative database Wikidata.  


    Photo by G. Crescoli on Unsplash

     

     

  5. Open data on elected Politicians and the power of community

    You may remember that thanks to a grant from the Wikimedia Foundation, mySociety has been working to support increasingly authoritative data on the world’s politicians, to exist on Wikidata as a key part of developing the concept of the Democratic Commons.

    And, this summer mySociety welcomed two members of staff to support with the community work around both Wikidata and the Democratic Commons. In May, I (Georgie) joined in the role of ‘Democratic Commons Community Liaison’ and in late June I was joined by Kelly, mySociety’s first ever ‘Wikimedia Community Liaison’… and it’s about time you started to hear more from us!

    I’ve been climbing the learning curve: exploring the potential moving parts of a global political data infrastructure,  finding out how the communities of Wikidata and Wikipedia operate, attempting to take meaningful notes at our daily meetings for the tool the team developed to improve political data on Wikidata and making sense of the complexity in creating interface tools to interpret the political data already in Wikidata. Oh, and supporting a “side-project” with Open Knowledge International to try and find every electoral boundary in the world (can you help?).  

    And if you are in any of the relevant open Slack channels (what is Slack?), you may have seen my name on the general introduction pages, as I have been shuffling around the online community centres of the world — off Wikidata Talk that is — trying to find  the people interested in, or with a need for, consistently and simply formatted data on politicians, but who aren’t already part of the Wikidata community.

    That’s because, the issue the Democratic Commons seeks to address is the time-consuming business of finding and maintaining data on politicians, work that we suspect is duplicated by multiple organisations in each country (often all of them having a similar aim), that is slowing down delivering the stuff that matters. This has certainly been mySociety’s experience when sharing our tools internationally.

    And the solution we propose  — the Democratic Commons — is that if people and communities worked together to find and maintain this data, it would be better for everyone…  ah the paradox of simplicity.

    Update on efforts to support the Democratic Commons concept

    With each interaction and conversation that we’ve had about the Democratic Commons with partners, we’ve continued to learn about the best role for us to play. Here are some initial actions and thoughts that are shaping the work; please feel free to comment, or even better,  get involved  🙂

    Making sure the concept is a good fit through user research
    We have set a goal to carry out user research on the concept of the Democratic Commons. So far, we have lined up calls with campaign staff  (who are interested in using and supporting open political data through their UK campaigning work) and journalists in Nigeria (who have expressed a need for the data) and I am lining up more calls — if you have a need for or can contribute political data, let’s talk.

    Bringing the Open Data/Civic Tech and Wiki communities together?
    From my experience to date, the Civic Tech and Wiki communities appear to operate quite separately (I am very open to being proved wrong on this point!).

    I am just getting started within the Wikidata/ Wikimedia communities (that’s more for Kelly) but on the Open Data/ Civic tech side,  there are questions about data vandalism and the potential to trust the data from Wikidata, arguments on the benefit of using Wikidata (especially where you already have a lot of useful data) and on whether there is a need to invest time in learning SPARQL, the query language that allows faster retrieval and manipulation of data from databases.

    Misconceptions are not unusual in communities online or offline, but it is a gap that our work focus, communications and tools hope to help close. If you have ideas on blogs, video tutorials or articles to share to read around these concepts, please get in touch.

    Working openly in existing global communities (off Wiki)
    We are aware that, off-Wiki mySociety is leading the work to develop the Democratic Commons, however, we know that we need to be delivering this work in the open for it to be owned by other people outside of mySociety, and finding the right homes to talk about it (off Wiki) has been important.  In order to work openly, we have a shared #DemocraticCommons Slack channel with mySociety and Code for All; see ‘Get involved’ below to find out how to join the conversation.

    We also plan to document the learning involved in the process through blog posts and documentation, to be uploaded publicly.

    And, supporting local communities to develop, where possible
    A global network such as Code for All is very useful in supporting a concept like the Democratic Commons, however, the bulk of need for the data will likely be country-specific. Together with our partners and collaborators, we are exploring what is needed and how to support local communities:

    • Through the remainder of our Wikimedia Foundation Grant, we are supporting community events and editathons: in Lebanon with SMEX, in France with newly formed organisation F0rk, and in Spain with Wikimedia España.
    • Some groups we are working with, such as Code for Pakistan, plan to set up a channel on their Slack instance and use their Whatsapp community to discuss the data use and maintenance.  
    • In my own country, the UK, we are talking to mySociety’s community and collaborators to understand how the Democratic Commons could benefit organisations and work in practice here. If you want to be involved in this work, please contact me.
    • We are listening to understand what support is needed with collaborators in the global South, as we’re well aware that it is a lot to ask people to work on a voluntary basis and that adequate support is needed. I hope we can share the learning and use it to shape any future projects that may emerge.

     

    How to get involved in the Democratic Commons?

    • Contribute to the Wikidata community: If you are Wikidata user, or keen to learn,  visit the Wikidata project page on political data. If you need guidance on tasks, do feel free to add to the Talk page to ask the community, or get in touch with Kelly, our Wikimedia Community Liaison: kelly@mysociety.org.
    • Join the conversation on Code for All Slack:  If you would like to join the Slack conversation, join here: https://codeforall.org/ (scroll down and find the ‘Chat with us’ button).
    • Look for electoral boundary data: We are working with Open Knowledge to find electoral boundary data for the whole world. See more about that here.
    • Keep up to date and subscribe to our Medium blog: Sometimes these Democratic Commons posts are a bit too in-depth for the general mySociety readership, so for those who are really interested, we plan to share all we are learning here.
    • Share the concept with contacts: Please share the message on your platforms and encourage potential users to take part in research and get involved. We recognise that our view  — and reach — can only be anglo-centric, and we’d so appreciate any translations you might be able to contribute.
    • Tell us (and others) how you think you would use the data: This can’t just be about collecting data; it’s about it being used in a way that benefits us all. How would the Democratic Commons help your community? We would love people to share any ideas, data visualisations, or theories, ideally in an open medium such as blog posts.  Please connect with Georgie to share.
    • Something missing from this list? Tell us! We’re @mySociety on Twitter or you can email georgie@mysociety.org or kelly@mysociety.org .

    Image: Toa Heftiba

  6. Help us find the world’s electoral boundaries

    We, and Open Knowledge International, are looking for the digital files that hold electoral boundaries, for every country in the world — and you can help.

    Yeah, we know — never let it be said we don’t know how to party.

    But seriously, there’s a very good reason for this request. When people make online tools to help citizens contact their local politicians, they need to be able to match users to the right representatives.

    So head on over to the Every Boundary survey and see how you can help — or read on for a bit more detail.

    Data for tools that empower citizens

    If you’ve used mySociety’s sites TheyWorkForYou — or any of the other parliamentary monitoring sites we’ve helped others to run around the world — you’ll have seen this matching in action. Electoral boundary data is also integral in campaigning and political accountability,  from Surfers against Sewage’s ‘Plastic Free Parliament’ campaign, to Call your Rep in the US.

    These sites all work on the precept that while people may not know the names of all their representatives at every level — well, do you? — people do tend to know their own postcode or equivalent. Since postcodes fall within boundaries, once both those pieces of information are known, it’s simple to present the user with their correct constituency or representative.

    So the boundaries of electoral districts are an essential piece of the data needed for such online tools.  As part of mySociety’s commitment to the Democratic Commons project, we’d like to be able to provide a single place where anyone planning to run a politician-contacting site can find these boundary files easily.

    And here’s why we need you

    Electoral boundaries are the lines that demarcate where constituencies begin and end. In the old days, they’d have been painstakingly plotted on a paper map, possibly accessible to the common citizen only by appointment.

    These days, they tend to be available as digital files, available via the web. Big step forward, right?

    But, as with every other type of political data, the story is not quite so simple.

    There’s a great variety of organisations responsible for maintaining electoral boundary files across different countries, and as a result, there’s little standardisation in where and how they are published.

    How you can help

    We need the boundary files for 231 countries (or as we more accurately — but less intuitively — refer to them, ‘places’), and for each place we need the boundaries for constituencies at national, regional and city levels. So there’s plenty to collect.

    As we so often realise when running this sort of project, it’s far easier for many people to find a few files each than it would be for our small team to try to track them all down. And that, of course, is where you come in.

    Whether you’ve got knowledge of your own country’s boundary files and where to find them online, or you’re willing to spend a bit of time searching around, we’d be so grateful for your help.

    Fortunately, there’s a tool we can use to help collect these files — and we didn’t even have to make it ourselves! The Open Data Survey, first created by Open Knowledge International to assess and display just how much governmental information around the world is freely available as open data, has gone on to aid many projects as they collect data for their own campaigns and research.

    Now we’ve used this same tool to provide a place where you can let us know where to find that electoral boundary data we need.

    Where to begin

    Start here  — and please feel free to get in touch if anything isn’t quite clear, or you have any general questions. You might want to check the FAQs first though!

    Thanks for your help — it will go on to improve citizen empowerment and politician accountability throughout the world. And that is not something everyone can say they’ve done.

    Image credit: Sam Poullain

  7. Exploring the representation of women around the world

    The Inter Parliamentary Union release a report each year detailing changes in the representation of women across the world. In 2017, women represented 23.4% of all MPs – which is less than half of the proportion of women in the population at large.

    While the picture for the last decade shows a positive trend, there is nothing inevitable about ever-increasing representation of women. The IPU report notes that while Albania and France’s representation of women rose by 10% and 12% respectively, other countries saw a decline. Improved representation of women is often a result of decisions deliberately taken to improve representation, rather than being a natural outcome of unstoppable social forces.

    One of the pitfalls of international comparisons is that it obscures some of the drivers of good and poor representation. Increased representation of women is often uneven, and concentrated more in some parties rather than others. As Miki Caul points out, international comparisons of relative representation of women overlook “the fact that individual parties vary greatly in the proportion of women MPs within each nation”. Similarly, Lena Wängnerud arguescross-country studies tend to miss variations between parties within a single system. Variations in the proportion of women to men are even greater across parties than across nations”.

    To understand more about this, we’ve built an experimental mini site to examine the roles of parties in driving the representation of women. Using data from EveryPolitician.org (which contains gender and party information for a number of countries), we can explore the respective contributions of different parties to representation of women.

    For this it’s not enough to look at the gender ratios of all the parties individually, as those with the best proportional representation of women are often quite small — for instance,  the Green Party in the UK has 100% female representation, in the form of its one MP.

    Instead, what we look at is the respective contributions to the total gender ratio. For each party we look at how much better or worse the proportional representation of women would be if you ignored that party’s MPs.

    For instance in the UK, while the gender ratio of the current House of Commons is around 32%, the Labour Party’s ratio is around 44%. If you take out the Labour Party the representation of women in the House of Common as a whole drops to 23%.

    For our purposes, the Labour Party is the UK’s Most Valuable Party (MVP) — ignoring it leads to the largest reduction in the representation of women. For each country, the gap between the ‘gender ratio’ and the ‘gender ratio ignoring the MVP’ gives a new metric of how to understand the gap in gender representation. Where this number is high, it means that the role of individual parties is very important; where it is lower it means that the ratio is not strongly driven by party effects. For instance, the gender ratio in the United States is strongly driven by party effects, while in Bolivia it is not.

    Countries with a wide gap between the ‘ratio ignoring the best party’ and ‘ratio ignoring the worst party’ tend to be countries that use majoritarian electoral systems, like the UK. Pippa Norris shows that systems using majoritarian electoral systems tend to have a poorer representation of women than those using proportional representation, but also that there is a lot of variation within each family of electoral systems and “the basic type of electoral system is neither a necessary nor a sufficient condition to guarantee women’s representation”.

    Our analysis shows that parties have different levels of agency to improve the overall representation of women depending on the party structure created by the electoral system. Countries that use proportional representation tend to show smaller party effects because there are usually more parties with fewer MPs — and so the ability of any one party to shift the overall representation is reduced. Conversely, in FPTP parliaments with only a few major parties, a large amount of change can happen by only one of these major parties taking measures to improve their internal representation of women.

    For example, while Germany’s CDU and the UK’s Conservative party have a similar representation of women at the national level (20.5% and 21.14% respectively), the Conservative party has more than twice the leverage to affect the overall representation of women simply by changing their own policy.

    There are limits to using the proportional representation of women as a single measure for the political representation of women. As mySociety’s Head of Research Rebecca Rumbul has previously shown, even bodies with relatively good representation of women like the National Assembly for Wales can then fall down on other areas – with a low proportion of oral evidence to consultations and committees coming from women. While the UK’s Conservative party performs poorly on the proportion of MPs, it has conversely selected more female party leaders and Prime Ministers.

    Importantly, looking at the representation of women as a single figure also obscures the important role of social factors as such class or race in shaping which women are represented. Creating a metric for comparison across many different countries is inherently reductive and discards important information about local context in every instance.

    Our goal with this website has been to re-complicate the international comparison by moving away from a single national statistic for representation in a way that assigns agency to political actors within each country. Variations among these parties (and international variations in this variation) reflect that representation of currently under-represented groups isn’t a natural fact of life in a given country, but reflects choices made – and that other choices can lead to different outcomes.

    This is still a work in progress and we acknowledge there will be holes in how this data has been applied. Lack of gender information for all countries means that some countries that have high representation of women (such as Rwanda) are not addressed. This means that it shouldn’t be taken as a comprehensive ranking — but we hope it is useful as a jumping off point for thinking about the representation of women in parliaments across the world.

    We have detailed our methodology here, including known issues with the data. This is an early experiment with the data and we welcome feedback on the website here; or get in touch through the contact details here.

    The data the site is built on can be downloaded from everypolitician.org.

    You can explore the website here, or sign up to the research newsletter here.


    Image: Photo by Joakim Honkasalo on Unsplash

  8. Using mySociety data to explore the representation of women

    A key part of mySociety’s research agenda is understanding how Civic Technology is (or isn’t) helping under-represented groups in society access government services and their representation. In 2015 we released a report  Who Benefits from Civic Technology, that explored variations in usage of Civic Tech in various countries and demographics. You can read or download it here.

    In this blog post I’m going to talk a bit about how we’ve internally tried to apply our data to understanding the under-representation of women in politics and as users of our services, as well as some interesting things that external researchers have found using our data.

    EveryPolitician

    Our EveryPolitician dataset contains information on current (and in some cases historical) politicians for a large number of countries around the world. For a large number of representatives, this includes gender information.

    However, a key problem of international comparisons of the representation of women is, as Miki Caul points out, that it “overlooks the fact that individual parties vary greatly in the proportion of women MPs within each nation”. Similarly, Lena Wängnerud argues “cross-country studies tend to miss variations between parties within a single system. Variations in the proportion of women to men are even greater across parties than across nations”.

    Fortunately, this is exactly the kind of problem that an international dataset like EveryPolitician is well placed to examine – on Thursday we’ll be using a new mini-site to explore the gender and party information contained in EveryPolitician to give a sense of the international picture and the party-level differences within each country. Stay tuned! Or you can download the data yourself (there are APIs for Python, Ruby and R) and try and beat us to it.

    TheyWorkForYou

    TheyWorkForYou makes it easy to search through the history of what has been said in Parliament, and we make the data (based on the Hansard dataset but more consistently formatted) freely available to download. As essentially a download of a very large amount of text, getting insights from this dataset is a bit more complicated, but potentially very rewarding.

    Jack Blumenau has a paper based on TheyWorkForYou data using language to analyse whether appointing female ministers changes how other female MPs participate in debates. Looking at “half a million Commons’ speeches between 1997 and 2017, [he demonstrates] that appointing a female minster increases the participation of women MPs in relevant debates by approximately one third over the level of female participation under male ministers” – and that “female MPs also became more influential in debates under the purview of female ministers […] female ministers respond in a systematically different fashion to the speeches of female MPs.” In this case, influence is a measure of whether the language an individual used is then taken up by others, and this kind of analysis shows how the TheyWorkForYou dataset can be used to demonstrate not just counts of how many women were in Parliament, but the substantive effects of women holding office on the political process.

    As Myf talked about yesterday, TheyWorkForYou’s Commons content now extends back to 1918, and so includes every speech by a female MP ever made. We hope this is a useful resource for anyone interested in exploring the history of the representation of women in the UK and have plans for a small project in the upcoming months to show in a simple way how this data can be used (please sign up to our mailing list if you’re interested in hearing about this when it’s completed).

    TheyWorkForYou data can either be accessed through an API, or downloaded as formatted XML files.

    FixMyStreet and WriteToThem

    Understanding the under-representation of women is important across our services. Where men and women are experiencing different issues and concerns, imbalances in access (or use of access) potentially lead to differences in resource allocation.

    The majority of reports on FixMyStreet.com are reported by men – but to make things more complicated, it’s not just that women make fewer reports, but women report substantively different kinds of reports.

    Reka Solymosi, Kate Bowers and Taku Fujiyama investigated FixMyStreet reports and found (by determining gender from names of problem reporters) that different kinds of reports are more likely to be reported by men and women – they suggest that at “first glance it appears that men are more likely to report in categories related to driving (potholes and road problems), whereas women report more in categories related to walking (parks, dead animals, dog fouling, litter)”.

    If different kinds of reports are differently gendered, this complicates thinking about how to improve how women use the website – as potential users are having substantially different experiences of problems in the real world well before they interact with the site. We have to engage with the nuance of this kind of finding to understand how to redress issues of access to services.

    We’re currently in the process of extending this kind of analysis to our other service. For WriteToThem, we’ve learned that while the majority of people using the service to write to MPs are male (around 60%), this picture is different depending on the level of government – for instance the gender balance for people writing to councils is pretty close to 50/50.

    As part of this, we’re investigating whether having the same gender as their representative makes people more likely to make contact. This has some interesting preliminary findings, and we hope to have more to say about this towards the end of the year.

    Our research in this area is ongoing, and we’re keen to help people use our data to investigate under-representation – especially where you have expertise or knowledge that we don’t. If you’d like to discuss potential uses of the data please get in touch, or sign up to our mailing list to hear about future research releases.

    Image: Theresa May’s first PMQs: © UK Parliament/Jessica Taylor (CC by-nc/2.0)

  9. Democratic Commons: an update

    It’s been a few months since we first announced our Democratic Commons project under the banner of “shared code, data and resources where anyone can contribute, and anyone can benefit” — but if we’ve been silent since then, it’s certainly not for a lack of activity.

    Quite the reverse, in fact: we’ve been busy bringing new team members on board and getting stuck in with the time-consuming and often fiddly process of data gathering and sharing.

    When we’re in the midst of all this hard work, it’s sometimes hard to remember to talk about how everything’s going;  but it’s always interesting, so here’s a snapshot of where we are now.

    Partnering up

    Those of us working on Democratic Commons are only a small team within the smallish organisation mySociety. Gathering in-depth data on politicians all around the world takes more time and more local knowledge than we have ourselves, so we’re working with partners located within our target countries.

    Distintas Latitudes have been handling Latin America – they’ve been great at gathering data and explaining the various differences between the political systems in each country we’ve worked together on.

    In India, Factly and Gender And Politics have done the most amazing job in gathering a full national and state level dataset for politicians right across the country. We were astounded, as that is a LOT of data (over 3,500 records and counting so far).

    And in South East Asia we’re working with OCF, with whom we’ve had a long association (you may remember TICTeC Taiwan, for example). OCF have helped us with data for Taiwan and South Korea so far, and are set to work with us on seven more countries before December 2018.

    Finally, a special mention goes to OpenLeb of Lebanon, who are working hard to start finding data in a country where data is not usually open. We genuinely could not do this work without our partners and we are eternally grateful for their help.

    Creating community

    As is probably clear from the above, we often select which countries to work on by our ability to find a community or organisation that will extend help. A nice side effect of this is that we’re strengthening the connections and bonds between mySociety and organisations with similar missions in many different places.

    Growing the community of such organisations across the world is going to be the primary focus of our new Community Manager Georgie, whom you will no doubt hear a lot from over the next few months.

    She’s going to be finding out who’s already using data like this, who’s maintaining it, who’s interested in running projects with it or doing research — and seeing if there’s also an appetite there to keep the data up to date. This is because the data will really only be useful to people if it’s well maintained and current!

    Working with Wikidata

    Early on, we recognised that improving the political data available in Wikidata, rather than ringfencing it all within EveryPolitician, was going to be an efficient way to maximise the benefits of the Democratic Commons project.

    What does this mean in practice? Well, in our first phase we’ve targeted 13 places in which to locate the data and load it into Wikidata: Mexico, Brazil, Colombia, Paraguay, Chile, Canada, Italy, Estonia, Lebanon, India, S Korea, Taiwan and Hong Kong.

    Ultimately, we want to help make any and all information about politicians at every level freely and openly available via the Commons; but for now, our initial scope looks at representatives at national, regional, and city legislatures.

    Now that a lot of this work is being done on or in Wikidata, we’re creating tools to make processes smoother and faster. The main ones of these are around verifying data and creating statements in Wikidata; we hope that when we’ve completed these they’ll be valuable to the whole Wikidata community beyond just the Democratic Commons project.

    Step by step

    We’re focusing on getting what we’re terming ‘Outline Data’ for each place loaded into Wikidata first. This type of data helps us model the political system, as it tells us what the legislature looks like — for example, whether it is unicameral, bicameral or different to those; what it calls members of the legislature; what term the legislature is on and how long that lasts; and often how many seats that legislature contains.

    Once we have that outline data, we then need some information about people holding seats in those legislatures. We try and start with five examples of each type of role at each level, then we can send this ‘Seed Data’ off to hopefully crowdsource the rest of the data: more on that in a bit!

    Meanwhile, our GIS expert Will is working on boundaries. Boundary data is hard — like, really hard! This is one of the most challenging areas of the project but it’s also one of the most important. Without electoral boundaries we don’t know what area a politician is representing and a lot of the tools we think this data would be useful for just won’t work.

    However, boundaries aren’t always released openly or completely, especially when it comes to local level constituencies, and even when we do find them, understanding whether we have all the data we need to represent politicians correctly can be really tricky.

    Because we like to keep really busy, we’ve also been starting to collaborate with other organisations such as Open Knowledge and CLEA on how to raise the visibility of the availability (or lack, more likely) of open sources of official boundary data.

    Working with Facebook

    You may remember our ongoing work to connect Facebook users with their politicians after an election, in countries around the world.

    We’re also working with Facebook to run some crowdsourcing experiments that will gather more data on politicians. I mentioned ‘Seed Data’ above. For each country, this gets fed to Facebook, and allows them to create questions which they can send to users to ask them who their representatives are at different levels of government.

    We then get this data back and our partners help us verify it and put it into Wikidata so it becomes open and available for anyone to use. Facebook has a reach we would never be able to manage on our own.

    So that’s where we are

    As I’ve hopefully demonstrated in this post, the work is extremely challenging. That’s why we’re sometimes a little slow in updating where we’ve got to — but we genuinely believe that that having this data out there in the open will pave the way for so many exciting new political data-based projects and research. And so, onwards!

    Image: Ben White (Unsplash)

  10. GLOWing the extra mile

    As you’ll remember from our previous blogpost, the Global Legislative Openness Week — AKA GLOW — provided us with a great opportunity to support events and spread the word about our ambitious Wikidata project with groups around the world.

    In the end we sponsored nine events (in Slovenia, Sweden, Croatia, Bulgaria, Italy, Greece, Spain, Wales and, amazingly, Nepal) and sent representatives to support seven of them in person. This meant that Tony, the lead for the EveryPolitican project, had an interesting 10 days getting the most out of budget airlines and managing to attend six of the events. He would have liked to have made it seven but EasyJet don’t go to Nepal! Also special thanks to Lucas who represented the project in Greece and Bulgaria taking a little of the pressure from Tony.

    Five of the countries (Croatia, Greece, ItalySlovenia and Spain) moved from very little or no coverage of political data in Wikidata to our two-star (or better) indicator — this is a (very) rough guide to how good the data is in a country; we are tracking what information already exists for the primary house of a national legislature which you can learn more about on our Wikidata project page. Wales was already a ‘three-star’ country and the others are coming along nicely.

    We couldn’t be happier with the response and contributions made to the Democratic Commons by the participants  — and we’re extremely keen to do more events early next year, collaborating with Wikimedia communities to make this happen. If that sounds interesting take a look at our original blogpost about the events and get in touch if it seems up your street.

    All in all these events really felt like a culmination of all the learnings and activities undertaken in the project so far. The strides we have taken in understanding the best way to model this kind of data in Wikidata and the tools that have been built are what made it possible to make such a big impact in each of the countries in just a day or two. Not all of these lessons were easy to learn but they are really starting to pay dividends now.

    We really want to keep this momentum up and build even more relationships with Wikimedia communities who are interested in contributing to the Democratic Commons in their countries so I can’t reiterate it enough – please do get in touch if you would like to get involved.

    One more thing

    If you’ve read this with great interest to the very end, then you are just the sort of person who we’re looking for! We currently have a vacancy for a Political Researcher to help us kickstart this kind of work in up to 100 countries, supporting our ambitions in the Democratic Commons project. See the job description here.

     

     

    Photo by Margo Brodowicz on Unsplash