1. Got a list of postcodes and need to match them to administrative areas?

    It’s a more common problem than you might think: given a list of postcodes, how can you match them to the administrative and electoral areas, such as wards or constituencies, that they sit within?

    MapIt’s data mapping tool gives a quick, easy and cheap solution: just upload your spreadsheet of postcodes, tell it which type of area you want them matched to, and the data is returned to you  — complete with a new column containing the information you need.

    The tool can match your postcodes to every type of data that MapIt offers in its API, including council areas, Westminster constituencies, parish wards and even NHS Clinical Commissioning Groups (CCGs).

    If that doesn’t sound like something you can imagine being useful, let’s look at a few hypothetical use cases (and if you have an actual case that you’d like to tell us about, please do let us know  — we’re always keen to hear how our tools are being used).

    Organisations, charities and campaigns sometimes need to match postcodes to administrative areas

    Membership organisations, charities and campaigns usually collect the addresses of supporters, but don’t commonly ask them who their MP is (even if they did ask, most people in the UK don’t actually know the name of their MP).

    But when a campaign asks followers to contact their MPs, it’s helpful to be able to suggest an angle based on whether the MP is known to be sympathetic to their cause, or not — indeed, there’s arguably no point in contacting MPs who are already known to be firmly on board.

    So: input a spreadsheet of supporters’ postcodes, and get them matched to the associated Westminster constituencies.

    For more advanced usages, organisations might match the MapIt tool’s output of postcodes with other datasets to discover the answers to questions like:

    • Which members in a disability group have fewest GPs in their area, and might be finding it difficult to get help for their condition?
    • Which supporters of a transport charity live in regions less served by public transport, and would be likely to take action to campaign for improved bus and train services?
    • Which people affiliated to an ecological organisation live in predominantly rural areas and could help with a wildlife count?

    Researchers sometimes need to match postcodes to administrative areas

    Researchers often need to correlate people, institutions or locations with the boundaries they fall within.

    They might have a list of postcodes for, say, underperforming schools, and want to find out whether they are clustered within authorities that have similar characteristics, like cuts to their funding or an administration that has a political majority one way or the other.

    Teamed with other datasets, MapIt can help towards answering important questions like the number of people each CCG serves, how unemployment rates vary in different European regions, or average house prices within parliamentary constituencies.

    Journalists sometimes need to match postcodes to administrative areas

    Investigative or data journalists may obtain long spreadsheets full of postcodes in the course of their work, perhaps as a result of having submitted Freedom of Information requests to one or more authorities.

    Perhaps they have the address of every university in the country, and there’s an election coming up — during the summer holidays. Knowing that students will mostly be in their home constituencies, they might be able to make informed predictions about how votes in the university towns will be affected.

    Or let’s say that a journalist has gathered, from local councils, an address for every library scheduled to close. This could be compared with another dataset — perhaps literacy or crime rates — to draw conclusions over what impact the closures would have.

    Part of a wider service

    The MapIt machine

     

    The one-off data mapping tool is just one service from mySociety’s MapIt, which is best known for its API.

    This provides an ongoing service, typically for those running websites that ask users to input geographical points such as postcodes or lat/longs, and return tailored results depending on the boundaries those points fall within.

    MapIt powers most mySociety sites, for example:

    • When you drop a pin on the map while using FixMyStreet, MapIt provides the site with the administrative boundaries it falls within, so that the site can then match your report with the authority responsible for fixing it.
    • When you type your postcode into WriteToThem, Mapit gives the site the information it needs to to display a list of every representative, from local councillor up to MEP, who represents your area.
    • If you search for your postcode on TheyWorkForYou, MapIt tells the site what your Westminster constituency is and the site matches that to your MP. You can then be taken to their page with a record of how they have voted and everything they’ve said in Parliament.

    Give it a try

    Find out more about MapIt here or have a go at uploading a spreadsheet into the data mapping tool.

    If you’re not sure whether it’s the right tool for your needs, feel free to drop us a line — and, as we said before, if you are already using it to good effect, please do let us know.

    Image: Thor Alvis

  2. Using WhatDoTheyKnow to uncover how schoolchildren’s data was used to support the Hostile Environment

    Freedom of Information forms the basis of many a campaign that seeks to expose hidden facts, or stories which should be in the public eye.

    We spoke to Jen Persson, Director of defenddigitalme, about that organisation’s tireless campaign to get to the truth on the collection, handling and re-use of schoolchildren’s personal data in England.

    What emerged was a timeline of requests and responses — sometimes hard fought for — which when pieced together reveal secrecy, bad practice and some outright falsehoods from the authorities to whom we entrust our children’s data. Perhaps most striking of the findings was the sharing of data with the Home Office in support of their Hostile Environment policy.

    As Jen describes defenddigitalme’s campaign, “It began with trying to understand how my daughter’s personal information is used by the Department for Education; it became a campaign to get the use of 23 million records made safe”.

    It’s a long tale, but definitely worth the read.

    December 2012: consultations and changes

    The story begins here, although it would still be a couple of years before Jen became aware of the issues around children’s data, “despite — or perhaps because of — having three young children in school at the time”.

    Why did no one at all seem to know where millions of children’s personal data was being sent out to, or why, or for how long?

    As Jen explains, “During the Christmas holidays, the Department for Education (DfE) announced a consultation about changing data laws on how nationally stored school pupil records could be used, proposing that individual pupil-level records could be given away to third parties, including commercial companies, journalists, charities, and researchers. Campaigners raised alarm bells, pointing out that the personal data would be highly identifying, sensitive, and insecure — but the changes went through nonetheless.”

    2014: discovering the power of FOI

    Jen came across that change in law for herself when reading about a later, similar data issue in the press: there were plans to also make available medical records from GP practices. This prompted her first foray into FOI, “to answer some of the questions I had about the plans, which weren’t being published”.

    I feel strongly that if I am going to ask for information which has a cost in time and capacity in the public sector, then it should mean the answers become available to everyone.

    And that first step got her thinking:

    “At around the same time I asked the DfE a simple question, albeit through a Subject Access rather than FOI request: What personal data do you hold about my own child?

    “My Subject Access request was refused. The Department for Education would not tell me what data they held about my children, and as importantly, could not tell me who they had given it to.

    “There was nothing at all in the public domain about this database the DfE held, beyond what the campaigners in 2012 had exposed. It wasn’t even clear how big it was. How was it governed? Who decided where data could be sent out to and why? How was it audited and what were the accountability mechanisms? And why was the DfE refusing its lawful obligations to tell me what they held about my daughter, let me correct errors, and know where it had gone? Why did no one at all seem to know where millions of children’s personal data was being sent out to, or why, or for how long?

    “Prior to all this, I’d never even heard of Freedom of Information. But I knew that there was something wrong and unjust about commercial companies and journalists being able to access more personal data about our children than we could ourselves.

    I worded some questions badly. I learned how to write them better. And I’m still learning.

    “I needed to understand how the database operated in order to challenge it. I needed to be able to offer an evidenced and alternative view of what could be better, and why. FOI was the only way to start to obtain information that was in the public interest.

    “I believed it should be published in the public domain. WhatDoTheyKnow is brilliant at that. I feel strongly that if I am going to ask for information which has a cost in time and capacity in the public sector, then it should mean the answers become available to everyone.”

    And so Jen went on a crash course to learn about FOI, reading books by Heather Brookes and Matthew Burgess, and WhatDoTheyKnow’s own guidance pages.

    “I tried to ask for information I knew existed or should exist, that would support the reasons for the changes we needed in data handling. I worded some questions badly. I learned how to write them better. And I’m still learning.”

    2015: sharing children’s personal data with newspapers

    That was just the beginning: at the time of writing, Jen has made over 80 FOI requests in public via WhatDoTheyKnow.com .

    Through FOI, defenddigitalme has discovered who has had access to the data about millions of individuals, and under what precepts, finding such astonishing rationales as: “The Daily Telegraph requested pupil-level data and so suppression was not applicable.” The publication “wished to factor in the different types of pupil” attending different schools.

    Jen explains: “This covered information on pupil characteristics related to prior attainment: gender, ethnic group, language group, free school meal eligibility (often used as a proxy for poverty indicators) and SEN (Special Educational Needs and disability) status, which were deemed by the Department to be appropriate as these are seen as important factors in levels of pupil attainment.”

    But with such granular detail, anonymity would be lost and the DfE were relying only on “cast iron assurances” that the Telegraph would not use the data to identify individuals.

    2016: sharing children’s nationality data with the Home Office

    In a Written Question put by Caroline Lucas in Parliament in July 2016, the Minister for Education was asked whether the Home Office would access this newly collected nationality data. He stated: “the data will be collected solely for the Department’s internal use […]. There are currently no plans to share the data with other government departments unless we are legally required to do so.”

    But on the contrary: defenddigitalme’s subsequent requests would disclose that there was already a data sharing agreement to hand over data on nationality to the Home Office, for the purposes of immigration enforcement and to support the Hostile Environment policy.

    Jen says: “As part of our ongoing questions about the types of users of the school census data, we’d asked whether the Home Office or police were recipients of pupil data, because it wasn’t recorded in the public registry of data recipients.

    The Home Office had requested data about dependents of parents or guardians suspected of being in the country without leave to remain.

    “In August 2016, a FOI response did confirm that the Home Office was indeed accessing national pupil data; but to get to the full extent of the issue, we had to ask follow up questions. They had said that “since April 2012, the Home Office has submitted 20 requests for information to the National Pupil Database. Of these 18 were granted and 2 were refused as the NPD did not contain the information requested.

    “But the reply did not indicate how many people each request was for. And sure enough, when we asked for the detail, we found the requests were for hundreds of people at a time. Only later again, did we get told that each request could be for a maximum agreed 1,500 individuals, a policy set out in an agreement between the Departments which had started in 2015, in secret.

    “In the October afternoon of the very same day as the school census was collecting nationality data for the first time, this response confirmed that the Home Office had access to previously collected school census pupil data including name, home and school address: “The nature of all requests from the Police and the Home Office is to search for specific individuals in the National Pupil Database and to return the latest address and/or school information held where a match is found in the NPD.”

    The Home Office had requested data about dependents of parents or guardians suspected of being in the country without leave to remain.

    “In December 2016, after much intervention by MPs, including leaked letters, and FOI requests by both us and — we later learned —  by journalists at Schools Week, the government published the data sharing agreement that they had in place and that was being used”.

    It had been amended in October 2016 to remove the line on nationality data, and allowed the data to be matched with Home Office information. It had also been planned to deprioritise the children of those without leave to remain when allocating school places, shocking opposition MPs who described the plan variously as “a grubby little idea” and, simply, “disgusting”.

    Other campaigners joined the efforts as facts started to come into the public domain. A coalition of charities and child rights advocates formed under the umbrella organisation of Against Borders for Children, and Liberty would go on to support them in preparing a judicial review. ABC organised a successful public boycott, and parents and teachers supplied samples of forms that schools were using, some asking for only non-white British pupils to provide information.

    Overall, nationality was not returned for more than a quarter of pupils.

    2017: behind the policy making

    Through further requests defenddigitalme learned that the highly controversial decision to collect nationality and country of birth from children in schools — which came into effect from the autumn of 2016 — had been made in 2015. Furthermore, it had been signed off by a little known board which, crucially, had been kept in the dark.

    “I’d been told by attendees of the Star Chamber Scrutiny Board meeting that they had not been informed that the Home Office was already getting access to pupil data when they were asked to sign off the new nationality data collection, and they were not told that this new data would be passed on for Home Office purposes, either. That matters in my opinion, because law-making relies on accountability to ensure that decisions are just. It can’t be built on lies”, says Jen.

    The process of getting hold of the minutes from that significant meeting took a year.

    Jen says, “We went all the way through the appeals process, from the first Internal Review, then a complaint to the Information Commissioner. The ICO had issued a Decision Notice that meant the DfE should provide the information, but when they still refused the next step was the Information Rights First Tier Tribunal.

    “Two weeks before the court hearing due, the DfE eventually withdrew its appeal and provided some of the information in November 2017. Volunteers helped us with preparation of the paperwork, including folk from the Campaign for Freedom for Information. It was important that the ICO’s decision was respected.”

    2018: raised awareness

    In April last year, the Department confirmed that Nationality and Country of Birth must no longer be collected for school census purposes.

    However, Jen says, “Children’s data, collected for the purposes of education, are still being shared monthly for the purposes of the Hostile Environment. There’s a verbal promise that the nationality data won’t be passed over, but since the government’s recent introduction of the Immigration Bill 2018 and immigration exemption in the Data Protection Act, I have little trust in the department’s ability in the face of Home Office pressure, to be able to keep those promises.

    “The Bill includes a blanket sweeping away of privacy rights, highlighted by the 3 Million campaign, again thanks to FOI: Every EU citizen applying for Settled Status to accept its Privacy Policy that allows it to share all data with “public and private sector organisations in the UK and overseas.”

    “Disappointingly”, says Jen, “the government has decided instead of respecting human rights to data protection and privacy on this, to create new laws to work around them.

    The direction of travel for change to manage data for good, is the right one.

    “It’s wrong to misuse data collected for one purpose and on one legal basis entrusted for children’s education, for something punitive. We need children in education, it’s in their best interests and those of our wider society. Everyone needs to be able to trust the system.

    “That’s why we support Against Borders for Children’s call to delete the nationality data.

    “A positive overall outcome, however”, she continues, “is that in May 2018, the Department for Education put the sharing of all pupil level data on hold while they moved towards a new Secure Access model, based on the so-called ‘5-Safes’. The intention is distribute access to data with third parties, not distribute the data itself. The Department resumed data sharing in September but with new policies on data governance, working hard to make pupil data safer and meet ‘user needs’. The direction of travel for change to manage data for good, is the right one.”

    2019: Defenddigitalme continues to campaign

    Defenddigitalme has come a long way, but they won’t stop campaigning yet.

    People working with FOI is really important, even and perhaps especially when it doesn’t make the press, but provides better facts, knowledge, and understanding.

    Jen says, “Raw data is still distributed to third parties, and Subject Access, where I started, is still a real challenge.

    “The Department is handing out sensitive data, but can’t easily let you see all of it, or make corrections, or tell you which bodies for sure it was given to. Still, that shouldn’t put people off asking about their own or their child’s record, or opting out of the use of their individual record for over 14s and adult learners, and demand respect for their rights, and better policy and practice. The biggest change needed is that people should be told where their data goes, who uses it for how long, and why.

    “Access to how government functions and the freedom of the press to be able to reveal and report on that is vital to keep the checks and balances on systems we cannot see. We rely on a strong civil service to work in the best interests of the country and all its people and uphold human rights and the rule of law, regardless of the colour of government or their own beliefs. People working with FOI is really important, even and perhaps especially when it doesn’t make the press, but provides better facts, knowledge, and understanding.

    “FOI can bring about greater transparency and accountability of policy and decision making. It’s then up to all of us to decide how to use that information, and act on it if the public are being misled, if decisions are unjust, or policy and practice that are hidden will be harmful to the public, not only those deciding what the public interest is.

    “WhatDoTheyKnow is a really useful tool in that. Long may it flourish.”

  3. See maps of FixMyStreet reports across the UK

    With funding from the Consumer Data Research Centre (CDRC) we’ve been working with researchers from the University of Sheffield and University of Sterling to open up FixMyStreet data for researchers.

    For an example of the kind of thing that can be done with this data, this group have produced maps for every local authority in the UK, mapping FixMyStreet reports against indices of deprivation (a few examples: Sheffield, Harrogate and Cardiff). These can be explored on our mini-site, where for each authority you can also download a printable poster with additional statistics.

    If you’d like to know more about what these maps mean and what we learned from the process, there’s a report exploring what we learned here.

  4. Improving the quality and consistency of political data in Wikidata

    What we’ve done — and what we want to do

    Wikidata now has up-to-date and consistent data on political position holders in current national legislatures for at least 39 countries (and work in progress for over 60 countries), thanks to work by volunteer community members on the Wikiproject every politician. mySociety worked as part of this project with a Wikimedia Foundation grant in 2017-18.

    There is now a real possibility for Wikidata to become the definitive source of data about democracies worldwide — but only if that data can be maintained sustainably. A significant risk is that elections and other major political changes quickly render data on political position holders and legislatures in Wikidata out-of-date.

    We’re proposing a Wikidata post-election updating toolkit project, which aims to ensure that data on elected representatives is substantially correct and complete within a month following an election, leading to improved quality and consistency of data in Wikidata over time. We’ll work as part of the Wikidata community to create and signpost tools and pathways that help contributors to quickly, easily and consistently update data following an election or other political change.

    How community members can get involved in the project

    If you’re already active around data relevant to political position holders, legislatures, or elections in Wikidata, we’d like your feedback and help to test the new tools and guidance and ensure that they are consistent with the emerging consensus around modelling these types of data.

    In particular, if you live in a country or major region that has an upcoming election, please talk to us about piloting the tools! We’d like for you to test the project tools and guidance to update data following your country’s election, and to give us feedback on the value and appropriateness of the approach in your context and political system.

    In general, we’re keen to encourage discussion and evaluation of Wikidata as a source of current position holder data.

    Please review our proposal

    If you’re interested in this, and are active on Wiki projects, please have a look and review our proposal here.

    Image: Mike Alonzo

  5. Are you spending too much time looking for data on UK Politicians?

    Forgive me if the title of this post makes us sound like a price comparison site — it’s just that if you are, mySociety is interested to hear from you.

    We’re hoping to hear from people who spend a lot of energy collating data on UK politicians — where you have to go through a process of collecting basic info like politicians’ names, parties, and the areas they represent, before you can even get to the real work of your project.  Specifically, we are interested in learning more about the impact this additional effort has on your work; the staff time it is costing your organisation, or the issues it creates in connecting citizens to their representatives.

    Recently, mySociety met with Democracy Club and Open Data Manchester to discuss the lack of open data on UK councillors, what could or should be done about it, and by whom. Sym from Democracy Club has brilliantly covered the who, what and how background to our meeting in a series of posts and I really recommend reading these.

    But first things first. We all recognised that before we travel too far down the road of planning something, we need to understand why.

    Why should there be open access to basic data on all of our elected representatives?

    Collectively, we agreed that the basic data on our elected representatives should be available as structured, consistent and reusable public information; who represents you at each level of government should be a public good and we believe that there is an obligation on authorities to ensure this information is made freely available in a structured way. The arts and sciences already recognise this concept of ‘commoning’; the same beliefs underpin mySociety advocating for a Democratic Commons. Plus, like Sym,  we agree that “access to good information is vital to a well-functioning democracy”.

    However, tangible examples are better than abstract beliefs, which is why we are interested in finding cases that demonstrate the potential social impact from opening up this data.

    We already know that:

    • Open Data Manchester spent a lot of time collating data on English Councillors so that they could match who local representatives with the most localised level of deprivation profile, a Lower Super Output area. ODM hope that “the dataset will add to the understanding of the local political landscape in England” and will allow for further enquiry where patterns of representation exist.
      Oh, here is Open Data Manchester’s beautiful visualisation of “The deprivation profile of each local authority (most deprived from the top left, down)  and the party with the most power”
    • We are aware of a number of charities that are independently gathering and maintaining basic data on politicians, a duplication of efforts and resource that could be better spent in other ways. And, that the cost of accessing Councillor Data is preventing some small charities running e-campaigns.
    • Organisations like Global Witness are using EveryPolitician data to spot potential corruption — but this data currently exists only at the national level, both for the UK and internationally.
    • And, we recognise that there are many commercial players in this space who provide complete and up to date data on politicians, which often includes more detailed biographical or political background. That’s not what we’re trying to replicate; instead, we all feel that there should be basic fundamental and up to date data on who our politicians are, freely available for anyone to use for any purpose.

    We would love to have more examples to add to this!  If you — or someone you know — has an idea for a piece of research or service that you could run if only this data existed, or spends time moaning about finding it, please get in touch with me, georgie@mysociety.org.

     

    P.S if you would like a copy of ODM data visualisation, it is available to buy/ download in A2 

    Photo by David Kennedy on Unsplash

  6. How academics are using Wikidata to look for links between legislative behaviour and the biographies of Members of Congress

    Earlier this year, we were fortunate enough to be contacted by Brian Keegan, Assistant Professor in Information Science at the University of Colorado Boulder, who specialises in the field of network analysis.

    Brian and his team were planning to mine the official biographies of every legislator published by the Library of Congress – going back to the first Congress in 1789 –  and add the information as structured data to Wikidata. Having heard of our involvement with  WikiProject Every Politician, they wanted to understand more about contributing.

    The research team, which included professors from the Libraries, Political Science and Information Science departments, planned to combine this biographical data with more common data in political science about voting and co-sponsorship, so that interesting questions could be asked, such as “Do Ivy League graduates form cliques?” or “Are medical doctors more likely to break with their party on votes concerning public health?”. Their hypothesis was that the biographical backgrounds of legislators could play an important role in legislative behaviours.

    However, the first big step before questions could be asked (or SPARQL queries made) was supporting undergraduate students to enter biographical data for every member of Congress (going right back to the first) on Wikidata. This has not generally made it into the datasets that political scientists use to study legislative behaviour, and as students began to enter data about these historical figures, it quickly became apparent why: non-existent nations, renamed cities, and archaic professions all needed to be resolved and mapped to Wikidata’s contemporary names and standardised formats.

    Nine months on, the team and ten undergraduates have revised over 1,500 Wikidata items about members of Congress, from the 104th to the 115th Congresses (1995-2018) and the 80th– 81st Congresses (1947-1951), which is 15% of the way through all members dating back to the first Congress in 1789!

    They started running SPARQL queries this summer.

    Joe Zamadics, a political science PhD student who worked on the project explained the potential of combining these data:  “One example we tried was looking at House member ideology by occupation. The graph below shows the ideology of three occupations: athletes, farmers, and teachers (in all, roughly 130 members). The x-axis shows common ideology (liberal to conservative) and the y-axis shows member’s ideology on non-left/right issues such as civil rights and foreign policy.  The graph shows that teachers split the ideological divide while farmers and athletes are more likely to be conservative.”

     

    The team are keen to highlight the potential that semantic web technology such as  Wikidata offers to social scientists.

    For the full Q + A with Brian and Joe see the mySociety Medium post. 

    Photo by Jomar on Unsplash

     

  7. Open data on elected Politicians and the power of community

    You may remember that thanks to a grant from the Wikimedia Foundation, mySociety has been working to support increasingly authoritative data on the world’s politicians, to exist on Wikidata as a key part of developing the concept of the Democratic Commons.

    And, this summer mySociety welcomed two members of staff to support with the community work around both Wikidata and the Democratic Commons. In May, I (Georgie) joined in the role of ‘Democratic Commons Community Liaison’ and in late June I was joined by Kelly, mySociety’s first ever ‘Wikimedia Community Liaison’… and it’s about time you started to hear more from us!

    I’ve been climbing the learning curve: exploring the potential moving parts of a global political data infrastructure,  finding out how the communities of Wikidata and Wikipedia operate, attempting to take meaningful notes at our daily meetings for the tool the team developed to improve political data on Wikidata and making sense of the complexity in creating interface tools to interpret the political data already in Wikidata. Oh, and supporting a “side-project” with Open Knowledge International to try and find every electoral boundary in the world (can you help?).  

    And if you are in any of the relevant open Slack channels (what is Slack?), you may have seen my name on the general introduction pages, as I have been shuffling around the online community centres of the world — off Wikidata Talk that is — trying to find  the people interested in, or with a need for, consistently and simply formatted data on politicians, but who aren’t already part of the Wikidata community.

    That’s because, the issue the Democratic Commons seeks to address is the time-consuming business of finding and maintaining data on politicians, work that we suspect is duplicated by multiple organisations in each country (often all of them having a similar aim), that is slowing down delivering the stuff that matters. This has certainly been mySociety’s experience when sharing our tools internationally.

    And the solution we propose  — the Democratic Commons — is that if people and communities worked together to find and maintain this data, it would be better for everyone…  ah the paradox of simplicity.

    Update on efforts to support the Democratic Commons concept

    With each interaction and conversation that we’ve had about the Democratic Commons with partners, we’ve continued to learn about the best role for us to play. Here are some initial actions and thoughts that are shaping the work; please feel free to comment, or even better,  get involved  🙂

    Making sure the concept is a good fit through user research
    We have set a goal to carry out user research on the concept of the Democratic Commons. So far, we have lined up calls with campaign staff  (who are interested in using and supporting open political data through their UK campaigning work) and journalists in Nigeria (who have expressed a need for the data) and I am lining up more calls — if you have a need for or can contribute political data, let’s talk.

    Bringing the Open Data/Civic Tech and Wiki communities together?
    From my experience to date, the Civic Tech and Wiki communities appear to operate quite separately (I am very open to being proved wrong on this point!).

    I am just getting started within the Wikidata/ Wikimedia communities (that’s more for Kelly) but on the Open Data/ Civic tech side,  there are questions about data vandalism and the potential to trust the data from Wikidata, arguments on the benefit of using Wikidata (especially where you already have a lot of useful data) and on whether there is a need to invest time in learning SPARQL, the query language that allows faster retrieval and manipulation of data from databases.

    Misconceptions are not unusual in communities online or offline, but it is a gap that our work focus, communications and tools hope to help close. If you have ideas on blogs, video tutorials or articles to share to read around these concepts, please get in touch.

    Working openly in existing global communities (off Wiki)
    We are aware that, off-Wiki mySociety is leading the work to develop the Democratic Commons, however, we know that we need to be delivering this work in the open for it to be owned by other people outside of mySociety, and finding the right homes to talk about it (off Wiki) has been important.  In order to work openly, we have a shared #DemocraticCommons Slack channel with mySociety and Code for All; see ‘Get involved’ below to find out how to join the conversation.

    We also plan to document the learning involved in the process through blog posts and documentation, to be uploaded publicly.

    And, supporting local communities to develop, where possible
    A global network such as Code for All is very useful in supporting a concept like the Democratic Commons, however, the bulk of need for the data will likely be country-specific. Together with our partners and collaborators, we are exploring what is needed and how to support local communities:

    • Through the remainder of our Wikimedia Foundation Grant, we are supporting community events and editathons: in Lebanon with SMEX, in France with newly formed organisation F0rk, and in Spain with Wikimedia España.
    • Some groups we are working with, such as Code for Pakistan, plan to set up a channel on their Slack instance and use their Whatsapp community to discuss the data use and maintenance.  
    • In my own country, the UK, we are talking to mySociety’s community and collaborators to understand how the Democratic Commons could benefit organisations and work in practice here. If you want to be involved in this work, please contact me.
    • We are listening to understand what support is needed with collaborators in the global South, as we’re well aware that it is a lot to ask people to work on a voluntary basis and that adequate support is needed. I hope we can share the learning and use it to shape any future projects that may emerge.

     

    How to get involved in the Democratic Commons?

    • Contribute to the Wikidata community: If you are Wikidata user, or keen to learn,  visit the Wikidata project page on political data. If you need guidance on tasks, do feel free to add to the Talk page to ask the community, or get in touch with Kelly, our Wikimedia Community Liaison: kelly@mysociety.org.
    • Join the conversation on Code for All Slack:  If you would like to join the Slack conversation, join here: https://codeforall.org/ (scroll down and find the ‘Chat with us’ button).
    • Look for electoral boundary data: We are working with Open Knowledge to find electoral boundary data for the whole world. See more about that here.
    • Keep up to date and subscribe to our Medium blog: Sometimes these Democratic Commons posts are a bit too in-depth for the general mySociety readership, so for those who are really interested, we plan to share all we are learning here.
    • Share the concept with contacts: Please share the message on your platforms and encourage potential users to take part in research and get involved. We recognise that our view  — and reach — can only be anglo-centric, and we’d so appreciate any translations you might be able to contribute.
    • Tell us (and others) how you think you would use the data: This can’t just be about collecting data; it’s about it being used in a way that benefits us all. How would the Democratic Commons help your community? We would love people to share any ideas, data visualisations, or theories, ideally in an open medium such as blog posts.  Please connect with Georgie to share.
    • Something missing from this list? Tell us! We’re @mySociety on Twitter or you can email georgie@mysociety.org or kelly@mysociety.org .

    Image: Toa Heftiba

  8. A data refresh for Collideoscope

    As Zarino explained in his recent blog post, we’ve recently spent time talking to road safety advocates and cycling groups, as we prepare for some big improvements to Collideoscope.

    This has resulted in a shortlist of the tickets we’ll be working on, which you’re welcome to browse (and comment on, though this requires a GitHub account).

    Collideoscope, like many mySociety projects, is a website of two halves. On the one hand, it invites those involved in a cycling collision or near miss to contribute information to a database; on the other, it provides an output of all that aggregated data for planners, researchers, campaigners and anyone else who will find it useful.

    We’ll shortly be making some changes to the site so that its purpose and functionality are crystal clear; but in the meanwhile the next important step was to import the most recent batch of STATS19 data.

    STATS19 is the form the police fill in when road accidents are reported, lending its name to the dataset released annually by the Department of Transport. We include this data on Collideoscope alongside our users’ reports: we just take the reports which refer to cycling incidents, and with this latest update we’re now displaying everything from 2013 up to 2016, the most recent data available.

    That means, when you browse the site, you can see at a glance how many incidents have occurred in a specific area, not just from our users but from the primary national accident database too. Just click the checkbox (‘show reports from the Department of Transport’) at the top of the page to include them on the map.

    So that’s our most recent bit of housekeeping; now watch this space for some bigger changes to Collideoscope.

    Image: Charisse Kenion

  9. Help us find the world’s electoral boundaries

    We, and Open Knowledge International, are looking for the digital files that hold electoral boundaries, for every country in the world — and you can help.

    Yeah, we know — never let it be said we don’t know how to party.

    But seriously, there’s a very good reason for this request. When people make online tools to help citizens contact their local politicians, they need to be able to match users to the right representatives.

    So head on over to the Every Boundary survey and see how you can help — or read on for a bit more detail.

    Data for tools that empower citizens

    If you’ve used mySociety’s sites TheyWorkForYou — or any of the other parliamentary monitoring sites we’ve helped others to run around the world — you’ll have seen this matching in action. Electoral boundary data is also integral in campaigning and political accountability,  from Surfers against Sewage’s ‘Plastic Free Parliament’ campaign, to Call your Rep in the US.

    These sites all work on the precept that while people may not know the names of all their representatives at every level — well, do you? — people do tend to know their own postcode or equivalent. Since postcodes fall within boundaries, once both those pieces of information are known, it’s simple to present the user with their correct constituency or representative.

    So the boundaries of electoral districts are an essential piece of the data needed for such online tools.  As part of mySociety’s commitment to the Democratic Commons project, we’d like to be able to provide a single place where anyone planning to run a politician-contacting site can find these boundary files easily.

    And here’s why we need you

    Electoral boundaries are the lines that demarcate where constituencies begin and end. In the old days, they’d have been painstakingly plotted on a paper map, possibly accessible to the common citizen only by appointment.

    These days, they tend to be available as digital files, available via the web. Big step forward, right?

    But, as with every other type of political data, the story is not quite so simple.

    There’s a great variety of organisations responsible for maintaining electoral boundary files across different countries, and as a result, there’s little standardisation in where and how they are published.

    How you can help

    We need the boundary files for 231 countries (or as we more accurately — but less intuitively — refer to them, ‘places’), and for each place we need the boundaries for constituencies at national, regional and city levels. So there’s plenty to collect.

    As we so often realise when running this sort of project, it’s far easier for many people to find a few files each than it would be for our small team to try to track them all down. And that, of course, is where you come in.

    Whether you’ve got knowledge of your own country’s boundary files and where to find them online, or you’re willing to spend a bit of time searching around, we’d be so grateful for your help.

    Fortunately, there’s a tool we can use to help collect these files — and we didn’t even have to make it ourselves! The Open Data Survey, first created by Open Knowledge International to assess and display just how much governmental information around the world is freely available as open data, has gone on to aid many projects as they collect data for their own campaigns and research.

    Now we’ve used this same tool to provide a place where you can let us know where to find that electoral boundary data we need.

    Where to begin

    Start here  — and please feel free to get in touch if anything isn’t quite clear, or you have any general questions. You might want to check the FAQs first though!

    Thanks for your help — it will go on to improve citizen empowerment and politician accountability throughout the world. And that is not something everyone can say they’ve done.

    Image credit: Sam Poullain

  10. Using mySociety data to explore the representation of women

    A key part of mySociety’s research agenda is understanding how Civic Technology is (or isn’t) helping under-represented groups in society access government services and their representation. In 2015 we released a report  Who Benefits from Civic Technology, that explored variations in usage of Civic Tech in various countries and demographics. You can read or download it here.

    In this blog post I’m going to talk a bit about how we’ve internally tried to apply our data to understanding the under-representation of women in politics and as users of our services, as well as some interesting things that external researchers have found using our data.

    EveryPolitician

    Our EveryPolitician dataset contains information on current (and in some cases historical) politicians for a large number of countries around the world. For a large number of representatives, this includes gender information.

    However, a key problem of international comparisons of the representation of women is, as Miki Caul points out, that it “overlooks the fact that individual parties vary greatly in the proportion of women MPs within each nation”. Similarly, Lena Wängnerud argues “cross-country studies tend to miss variations between parties within a single system. Variations in the proportion of women to men are even greater across parties than across nations”.

    Fortunately, this is exactly the kind of problem that an international dataset like EveryPolitician is well placed to examine – on Thursday we’ll be using a new mini-site to explore the gender and party information contained in EveryPolitician to give a sense of the international picture and the party-level differences within each country. Stay tuned! Or you can download the data yourself (there are APIs for Python, Ruby and R) and try and beat us to it.

    TheyWorkForYou

    TheyWorkForYou makes it easy to search through the history of what has been said in Parliament, and we make the data (based on the Hansard dataset but more consistently formatted) freely available to download. As essentially a download of a very large amount of text, getting insights from this dataset is a bit more complicated, but potentially very rewarding.

    Jack Blumenau has a paper based on TheyWorkForYou data using language to analyse whether appointing female ministers changes how other female MPs participate in debates. Looking at “half a million Commons’ speeches between 1997 and 2017, [he demonstrates] that appointing a female minster increases the participation of women MPs in relevant debates by approximately one third over the level of female participation under male ministers” – and that “female MPs also became more influential in debates under the purview of female ministers […] female ministers respond in a systematically different fashion to the speeches of female MPs.” In this case, influence is a measure of whether the language an individual used is then taken up by others, and this kind of analysis shows how the TheyWorkForYou dataset can be used to demonstrate not just counts of how many women were in Parliament, but the substantive effects of women holding office on the political process.

    As Myf talked about yesterday, TheyWorkForYou’s Commons content now extends back to 1918, and so includes every speech by a female MP ever made. We hope this is a useful resource for anyone interested in exploring the history of the representation of women in the UK and have plans for a small project in the upcoming months to show in a simple way how this data can be used (please sign up to our mailing list if you’re interested in hearing about this when it’s completed).

    TheyWorkForYou data can either be accessed through an API, or downloaded as formatted XML files.

    FixMyStreet and WriteToThem

    Understanding the under-representation of women is important across our services. Where men and women are experiencing different issues and concerns, imbalances in access (or use of access) potentially lead to differences in resource allocation.

    The majority of reports on FixMyStreet.com are reported by men – but to make things more complicated, it’s not just that women make fewer reports, but women report substantively different kinds of reports.

    Reka Solymosi, Kate Bowers and Taku Fujiyama investigated FixMyStreet reports and found (by determining gender from names of problem reporters) that different kinds of reports are more likely to be reported by men and women – they suggest that at “first glance it appears that men are more likely to report in categories related to driving (potholes and road problems), whereas women report more in categories related to walking (parks, dead animals, dog fouling, litter)”.

    If different kinds of reports are differently gendered, this complicates thinking about how to improve how women use the website – as potential users are having substantially different experiences of problems in the real world well before they interact with the site. We have to engage with the nuance of this kind of finding to understand how to redress issues of access to services.

    We’re currently in the process of extending this kind of analysis to our other service. For WriteToThem, we’ve learned that while the majority of people using the service to write to MPs are male (around 60%), this picture is different depending on the level of government – for instance the gender balance for people writing to councils is pretty close to 50/50.

    As part of this, we’re investigating whether having the same gender as their representative makes people more likely to make contact. This has some interesting preliminary findings, and we hope to have more to say about this towards the end of the year.

    Our research in this area is ongoing, and we’re keen to help people use our data to investigate under-representation – especially where you have expertise or knowledge that we don’t. If you’d like to discuss potential uses of the data please get in touch, or sign up to our mailing list to hear about future research releases.

    Image: Theresa May’s first PMQs: © UK Parliament/Jessica Taylor (CC by-nc/2.0)