There was some excitement here at mySociety this week, as the People’s Assembly website launched in South Africa. It’s the result of a year’s partnership with PMG and a good test of some of our newest collaborative software.

The site contains a vast amount of information, all available in the same place for the first time, and offering a simple way for South African citizens to keep an eye on what their representatives are doing. There are pages for each representative, Hansard and parliamentary Questions and Answers, records of members’ interests, and more.

Locating, processing and displaying this data was quite a challenge: it has been taken from a wide range of sources, and came in an even greater range of formats, including PDF documents, Word documents, Excel files, CSV files and sometimes just e-mailed lists of information.

But perhaps most significant is the site’s Representative Locator function. For the first time, South African citizens can now find out, with ease, who represents them – not as simple as it might seem at first.

The Proportional Representative system means that members of the National Assembly and National Council of Provinces are not directly elected from constituencies.  Political parties are, however, funded to run constituency offices and to allocate representatives to those offices.  We believe that this is the first time this data has been consolidated and presented as a simple search tool.

The software that runs the site

As you’ll know if you read our recent blog post about SayIt, our recent focus has been reaching out to provide software for civic or democratic-focused websites anywhere in the world.

The idea is that such groups no longer need worry about writing code from scratch, since we’ve already done it – and their energies can be better expended on gathering data or adjusting the software to work within the local governmental systems.

People’s Assembly is a great example of this. It utilises two underpinning pieces of technology:

Firstly, the Pombola platform, our software for running parliamentary monitoring websites.

If you’re reading this in the UK, you may be familiar with our own parliamentary monitoring site, TheyWorkForYou. Pombola provides several tools that make it easy to do much of what TheyWorkForYou does: it provides a structured database of the names and positions of those in power; it allows people to look up their elected representatives by inputting their location, and to isolate and see what a specific MP has contributed to discussions in Parliament’s committees and plenaries; albeit, in the case of  Hansard,  after a six-month delay necessitated by South Africa’s own protocols.

We first developed Pombola for Kenya’s Mzalendo.com, and it’s been re-used for ShineYourEye.org in Nigeria and Odekro.org in Ghana.  It’s superb to see this re-use, as it’s exactly what we set out to acheive.

Secondly, People’s Assembly is the very first site to use SayIt, which is embedded as a Django app to power the Hansard, Questions and Committees content. SayIt is one of our Components, built under the Poplus project, and we’re truly delighted  to see it in place, proving its worth and being used as we first envisaged.

Thanks are due

The main work on the People’s Assembly has been funded by the Indigo Trust, and the SayIt component work was funded by Google.org as part of the Poplus Project. We also wish to thank Geoff Kilpin, who helped greatly with the scrapers and templating.

Find us in the Google hangout here

You know, we can write all we like about our latest launch, SayIt – but it’s often easier to understand stuff face-to-face.

So we’ll be in a Google Hangout on Thursday, 13th February at 4pm GMT, talking through SayIt’s various features and our plans for future development.

You might like to join us if you’re:

• A member of an organisation anywhere in the world, and you want to know how you can use SayIt to put transcripts online
• A developer who would like to try working with SayIt
• Someone with technical knowledge who is interested in volunteering to help us parse other people’s transcripts
• Anyone who is just plain intrigued, or has questions

What is a Hangout?

It’s a Google tool that allows people to chat from an internet browser, with video – similar to Skype, but you can access it simply by pasting a URL into your browser.

How to join

Find us in the Google hangout here

You don’t need a Google account or even a webcam to view our Hangout.

 Not a good time?

Don’t worry if you can’t make it – we’ll be recording the whole chat for posterity, and we’ll post the link here once it’s available.

Until about two years ago I was quite actively involved in the Open Data movement. I sat in on the 2007 gathering in California where the first Open Data Principles were drafted, and later sat on the Transparency Board at the UK government.

I stopped being involved in early 2012 because I saw a couple of things happening. First, the Open Data baton had been picked up by dedicated, focused advocates like the Open Data Institute and the Open Knowledge Foundation, who could give 100% to fighting this fight (I always had to fit it around managing a growing organisation with other goals). And second I felt that the surge of relatively meaningful data releases in the country I live in (the UK) had pretty much come to an end. The real policy action and innovation will now happen in more rapidly-changing countries where transparency is a more visceral issue.

Still, despite walking away, I remained optimistic. It seemed more or less impossible to imagine that in twenty years’ time that there wouldn’t be quite a bit more Open Data around, especially in rich countries. But given the virtually-zero political gain to be had from this agenda in countries like the UK, where is said data actually going to come from?

Learning from Microsoft (really)

The more I thought about it, the more I realised that we’d already seen the answer in the form of Microsoft. Throughout the 1990s the .doc and .xls standard rose and took over governments around the world, even though there was never anything like a clear policy process that drove that decision.

There was certainly no high profile ‘Microsoft Government Partnership’ with international conferences and presidential speeches. Instead there was a safe, ‘no brainer’ product that governments bought to solve their problems, and these data standards came with it. The pressure on governments to do anything at all probably came from the fact that the private sector had widely adopted Office first.

I think that a recurrence of this phenomenon – change-through-replacing-old-computers – is where Open Data at real scale is going to come from. I think it’s going to come from old government computers being thrown away at their end-of-life and replaced with new computers that have software on them that produces Open Data more or less by default.

The big but

However, there’s a big BUT here. What if the new computers don’t come with tools that produce Open Data? This is where SayIt comes in, as an example of a relatively low-cost approach to making sure that the next generation of government IT systems do produce Open Data.

SayIt is a newly launched open source tool for publishing transcripts of trials, debates, interviews and so on. It publishes them online in a way that matches modern expectations about how stuff should work on the web – responsive, searchable and so on. It’s being built as a Poplus Component, which means it’s part of an international network of groups collaborating on shared technologies. Here’s JK Rowling being interviewed, published via SayIt.

But how does this little tool relate to the business of getting governments to release more Open Data? Well, SayIt isn’t just about publishing data, it’s about making it too – in a few months we’ll be sharing an authoring interface for making new transcripts from whatever source a user has access to.

We hope that having iterated and improved this authoring interface, SayIt can become the tool of choice for public sector transcribers, replacing whatever tool they use today (almost certainly Word). Then, if they use SayIt to make a transcript, instead of Word, then it will produce new, instantly-online Open Data every time they use it.

The true Open Data challenge is building brilliant products

But we can’t expect the public sector to use a tool like SayIt to make new Open Data unless it is cheaper, better and less burdensome than whatever they’re using now. We can’t – quite simply – expect to sell government procurement officers a new product mainly on the virtues of Open Data.  This means the tough task of persuading government employees that there is a new tool that is head-and-shoulders better than Excel or Word for certain purposes: formidable, familiar products that are much better than their critics like to let on.

So in order for SayIt to replace the current tools used by any current transcriber, it’s going to have to be really, really good. And really trustworthy. And it’s going to have to be well marketed. And that’s why we’ve chosen to build SayIt as an international, open source collaboration – as a Poplus Component. Because we think that without the billions of dollars it takes to compete with Microsoft, our best hope is to develop very narrow tools that do 0.01% of what Word does, but which do that one thing really really well. And our key strategic advantage, other than the trust that comes with Open Source and Open Standards, is the energy of the global civic hacking and government IT reform sector. SayIt is far more likely to succeed if it has ideas and inputs from contributors from around the world.

Regardless of whether or not SayIt ever succeeds in penetrating inside governments, this post is about an idea that such an approach represents. The idea is that people can advance the Open Data agenda not just by lobbying, but also by building and popularising tools that mean that data is born open in the first place. I hope this post will encourage more people to work on such tools, either on your own, or via collaborations like Poplus.

.

Photo by Troy Morris (CC)

Transcripts – the written records of who says what in a conversation – aren’t sexy.

However, they can be very important, or even historic. They can reveal big plans that will affect lots of people, and they are a basic requirement of political accountability.

But the way in which transcripts are made available online today doesn’t reflect this importance. They tend to be published as hundreds of PDFs, and look more or less like they were made in the 1950s.

We think that the people who are affected by the decisions and plans announced in transcribed meetings deserve better.

What is SayIt?

SayIt is an open source tool for publishing speeches, discussions and dialogues, simply and clearly, online. Search functionality is built in, you can link to any part of a transcript, and the whole thing works nicely on mobile devices.

SayIt can be used either as a hosted service, or it can be built directly into your own website, as a Django app. Here are some examples of what it looks like in its hosted, standalone form:

• The Leveson Inquiry – An investigation into press ethics (did you know Hugh Grant’s middle name?)

• The Charles Taylor trial – The former president of Liberia is tried for crimes against humanity in an epic seven year trial

• The complete works of Shakespeare – as a demonstration of SayIt’s flexibility in handling different kinds of transcript

However, SayIt’s main purpose is to be built into other sites and apps.  We don’t have a live demo of this today, but one of our international partners will soon be launching a new Parliamentary Monitoring site which uses SayIt to publish years of parliamentary transcripts.

SayIt is also 100% open data compatible, and we use a cut-down version of the Akoma Ntoso open standard for data import.

What isn’t SayIt?

SayIt is:

• Not a site full of data curated and uploaded by mySociety – it’s a tool for redeployment all over the net. We’ll host deployments where that’s helpful to people, though.

• Not primarily about Britain – whilst we’re a social enterprise based in the UK, SayIt has been built with an international perspective. We hope it will serve the needs of people watching politicians in places like Kenya and South Africa.

• Not solely a mySociety project – it’s actually an international collaboration, via the Poplus network (see more below).

• Not (yet) a tool to replace Microsoft Word as the way you write down transcripts in the first place. This is coming as we move from Alpha to Beta, though.

Why are we building SayIt?

SayIt is one of the Poplus Components. Poplus is a global collaboration of groups that believe it is currently too difficult and expensive to build effective new digital tools to help citizens exert power over institutions.

Poplus Components are loosely joined tools, mostly structured as web services, that can be used to radically decrease the development time of empowerment sites and apps.

SayIt is the newest component, and aims to reduce the difficulty and cost of launching services that contain transcripts – in particular websites that allow people to track the activities of politicians. Using SayIt or other Poplus components you can build your site in whatever language and framework suits your wishes, but save time by using the components to solve time-consuming problems for you.

The founders of Poplus are FCI in Chile, and mySociety in the UK – and we are hoping that the launch of SayIt will help grow the network. The project has been made possible by a grant from Google.org, while early iterations were aided by the Technology Strategy Board.

Interested in publishing transcripts via SayIt? Here’s what to do…

Having taken a look at the demos, we hope at least some of you are thinking ‘I know of some transcripts that would be better if published like this’.

If you are interested, then there are two approaches we’d recommend:

• If you’re a coder, or if you have access to technical skills, read about how to convert your data into the open standard we use. Then talk to us about how to get this data online.

• If you don’t have access to technical skills,  get in touch about what you’re interested in publishing, and we’ll explore the options with you.

Note to coders – We’ve not yet spent a lot of time making SayIt easy to deploy locally, so we know it may be a challenge. We’re here to help.

Where might SayIt help?

SayIt comes from a desire to publish the speeches of politicians. But we know that there are many other possible uses, which is why we built the Shakespeare demo.

We think SayIt could be useful for publishing and storing transcripts of:

• Local council meetings

• Court hearings

• Election hustings

• Academic research interviews and focus groups

• Academic seminars, lectures, etc

• Plays

• Market research focus groups

• Historic archives of events such as a coronation or key debate

These are just a few of our ideas, but we bet you have others – please do tell us in the comments below.

What’s coming next

At the moment, SayIt only covers publishing transcripts, not creating them. Needless to say, this lack of an authoring interface is a pretty big gap, but we are launching early (as an Alpha) because we want to know how you’ll use it, what features you want us to build, and what doesn’t work as well as we anticipated. We also want to see if we can attract other people to co-develop the code with us, which is the real spirit of the Poplus network.

We’ll also be adding the ability to subscribe to alerts so that you’ll get an email every time a keyword occurs (just as you can on our other websites, such as FixMyStreet, TheyWorkForYou and WhatDoTheyKnow). This feature will come into its own for ongoing series of transcripts such as council meetings.

Image by Columbia Phonograph Co. [Public domain], via Wikimedia Commons