1. Sending multiple FOI requests: the WhatDoTheyKnow Pro batch feature

    When we started building WhatDoTheyKnow Pro, our toolkit for professional users of FOI, we knew that there was one feature which would be a game-changer for such users: the ability to send a request to multiple authorities at once.

    In this blog post, we examine what we wanted the tool to do, how we are guarding against abuse, and finally we’ll give a step by step walkthrough of the interface.

    Investigative news stories or in-depth pieces of research often require information from a multitude of different sources. By gathering statistics or information from multiple authorities, journalists, activists and researchers can build up a previously-unseen picture, for example of how widespread a particular problem is, or where there are inequities in medical provision across the country.

    It’s something that many professional users of FOI are doing already, usually with the aid of their own homemade spreadsheets on which they keep track of requests made, dates by which replies should be expected, which bodies have responded, which need chasing, and of course the information held in the responses themselves.

    The standard WhatDoTheyKnow website already provides several helpful features that you just don’t get with a DIY system: it has all the right email addresses for authorities, for example; it guides you through the FOI process; and it will send you an email reminder when the deadline for response arrives — even taking bank holidays into account.

    But we knew that in order for our batch request feature to woo people away from their spreadsheets, it needed to do more than those homebuilt systems, some of which have been refined over several years and work well, even if a bit clunkily, for their owners.

    Power and responsibility

    One important consideration was uppermost in our minds when it came to batch requests: it costs authorities time and money to respond to each request, and of course that multiplies with batch requests. We are keen to promote responsible use of FOI, so we want to fold appropriate safeguards and guidance into whatever system we build.

    As mentioned, with WhatDoTheyKnow Pro we’re focusing on features that are genuinely useful for professional users of FOI, but we also want to help those users make better, more focused requests — ones that are more likely to get useful responses and see the light of day as news stories. So it was important that, in making it simple to send multiple requests, we also help users find the most suitable authorities to send their requests to.

    With that in mind, here are some balances we’ve put in place:

    • Users are limited in how many batch requests they can send within any one month — so there’s no chance to go too wild.
    • There’s a limit to the number of authorities that can be added to a single batch: we set this to be the number of local authorities in the UK, which is a logical sector to survey in this way.
    • Before users do a batch mailout, we encourage the sending of an initial request that goes to just a few authorities. This safeguard can reveal where a request is flawed, so for example, if the data you get back is not what you need or in the wrong format, you don’t have to send to the full list all over again.
    • We provide advice on cost limits to encourage succinct batch requests.
    • Authorities have the facility to report a request which is unsuitable for review by our administrators.
    • We’re rolling out the batch request functionality gradually to vetted WhatDoTheyKnow Pro users so that we can gradually learn how people use it in practice, and course-correct as necessary..

    Testing and improvements

    So far, the batch feature is only available to a select group of test users, who are giving us feedback on how they’re finding it. There’s certainly nothing like having your code being used by real people to help you see where improvements might be made!

    That said, it’s been a very gratifying process. With the help of our test users, we’ve seen that the batch request functionality has the potential to be immensely helpful to professional users of FOI; even genuinely game-changing. We are certain that with the sending tools, we’ve created a service that really adds value for this sector.

    We’re now in the next phase, and turning our attention to improving the functionality that helps users deal with incoming responses when they come in. This already exists in a basic form, and thanks to our testers, we’ve identified which improvements we need to make. We’re already working on incorporating them. But that is definitely material for the next update  — for now, let’s take a look at just how the batch request function works.

    Using batch request

    There are three parts to making any request, whether you’re doing it yourself or using WhatDoTheyKnow Pro:

    1. Creating the request
    2. Managing the responses
    3. Analysing the results

    The batch request functionality builds on our super-simple FOI workflow tools for WhatDoTheyKnow Pro, extending them to make larger investigations much easier.

    Creating the Request

    The first step is compiling a list of authorities to send the request to. From the compose screen, you might search on a keyword (for example, ‘hospital’, ‘Birmingham’, or ‘Birmingham hospitals’) and then add the authorities you’re interested in.

    Add authorities to your WhatDoTheyKnow Pro batch request

    Add authorities to your WhatDoTheyKnow Pro batch request

    Each authority is added to a recipient list and WhatDoTheyKnow Pro creates a ‘mail merge’ setup. You’ll see how many authorities you’re writing to in the compose interface.

    You can then draft your request. The special `Dear [Authority name]` salutation gets automatically replaced with each of the selected authorities when you send your batch.

    Mail merge on WhatDoTheyKnow Pro

    Finally, before sending you can choose a privacy duration.

    Setting a privacy duration on WhatDoTheyKnow Pro

    At this point you can either go straight ahead and send your request, or save the draft and come back to it later.

    Managing Responses

    Once you’ve sent your batch request, you’re going to receive a lot of replies from authorities. This is where WhatDoTheyKnow Pro’s functionality really comes into its own, keeping all that clutter out of your email inbox.

    Here’s what it looks like: the first thing you’ll see is a high-level progress bar showing you the overall progress of your batch. There are three main states that help you manage the requests in the batch:

    • In progress (yellow): This means that there’s no action needed by you – you’re waiting on the authority to respond with an acknowledgement or the information you’ve requested.
    • Action needed (red): When a request in the batch receives a response from the authority, you’ll need to check it out. We mark the response as “action needed” for you to review and decide what to do next.
    • Complete (green): Once there’s no further action needed – either you’ve got the information you asked for, the authority didn’t have the information, or they’ve refused and you don’t want to challenge them – the request moves to the ‘complete’ state, so you know you don’t need to think about it until you come to analyse the data.

    Clicking the title of the batch reveals the individual requests and their progress status. From there, you can click through, read the response and update the status.

    All requests status page on WhatDoTheyKnow Pro

    Analysing Results

    Now you’ve got all your data, it’s time to compare the results from different authorities.

    Sometimes authorities will reply in the main correspondence.

    Authorities who reply within the body of their response

    Other authorities respond with one or more attachments. You can view these inline or download them to your computer.

    Authorities who respond with an attachment

    If you’re dealing with a batch sent to lots of authorities, sometimes it’s easier to just download everything. You can download a Zip file containing all the correspondence and attachments for each request via the “Actions” menu. From there you can pull out the attachments that contain the raw data and plug the numbers or answers in to your spreadsheet so that you can compare across authorities.

    Download a whole batch response on WhatDoTheyKnow Pro

    You can sign up to WhatDoTheyKnow Pro today and receive 1 month free with the voucher code BLOGMARCH18. Make some requests to try out the FOI workflow tools for professionals, and get in touch to request to join the waiting list for batch access.

    If your FOI requests have made the news, let us know! Send us links to your published stories and we’ll throw in an extra month of WhatDoTheyKnow Pro for free. Your stories help us improve WhatDoTheyKnow Pro.

    Image: Peretz Partensky (CC by-sa/2.0)

  2. WhatDoTheyKnowPro has launched… quietly.

    If you happen to have visited the WhatDoTheyKnowPro page recently, you might have noticed that we’ve changed from offering free beta testers’ accounts to rolling out the payment interface.

    Yes, we’ve launched! But not with a bang: having quietly introduced the payment option, we wanted to wait for a while and check that there were no issues before making a public announcement. Consider this that public announcement.

    Just a reminder at this stage, in case you haven’t followed along with previous blog posts: WhatDoTheyKnowPro is a Freedom of Information toolkit which provides extra help with sending and organising requests. It’s designed to provide extra functionality for journalists and other people who use FOI in their jobs; if you’re a standard user of WhatDoTheyKnow, there’s nothing to  pay and there never will be. We’ll always keep the site, and all its current functionality, free for everyone.

    The question of price

    Launching WhatDoTheyKnowPro provoked an interesting debate on where to position it, pricewise.

    It’s not that we’ve never put a price on any of our services before: mySociety is a social enterprise, and we charge for some use of our APIs; our council clients pay us for FixMyStreet Pro; you can buy transit-time maps on Mapumental. These sources of income are just part of what help us to provide our core citizen-facing services for free.

    And we’re hardly trailblazers in that respect: in fact, it was fascinating to read the Knight Foundation’s recent report Scaling Civic Tech and see how common it is in our sector to rely on a variety of revenue streams, from user donations to philanthropic grants, to paid-for services.

    But while we may have experience in charging for our services, it’s definitely the first time we’ve had to price up a Freedom of Information toolkit for journalists and professionals!

    What’s it worth?

    We were effectively in the position of many an enterprise startup: with a market proposition that doesn’t exist in this exact form anywhere else. How do you know how to price something in those circumstances? Set it too low and you could miss out on important revenue; too high and you’ll alienate potential customers, many of whom are freelance journalists paying for their work tools out of their own pockets.

    Still, this project has been a process of tackling problems and questions thoughtfully — from deciding which features to include in this initial version, to debating how to encourage journalists to link back to the news stories they’ve created once they’re live. Perhaps we could bring the same approach to pricing.

    Ask the experts

    Fortunately, with over 100 beta testers, we had a pool of users who knew the service well enough to be able to give an expert opinion on how much value it was bringing them. Sending out a survey brought some very useful responses, not to mention feedback about what our beta users liked and what they’d appreciate in future roll-outs.

    WDTKPro survey

    That said, we’re well aware of research indicating that people are not always experts on what they are actually willing to pay.

    And of course, it makes sense that different people will attach different values to a service, depending not only on their own finances but, in this case all sorts of other factors such as how frequently they use FOI and how accustomed they are to paying for technology.

    Faced with responses to the survey that ranged between a suggested price of £1.00 a month to £50.00, that was, in some strange sense, reassuring to know.

    To be fair, those were the extremes. There was a good consensus in the middle and that helped us decide on an introductory price of £10 a month.  We’ll assess this after a few months to see whether it’s bringing the number of sign-ups we expect.

    Open for business

    If you’re a journalist or someone who uses FOI in your work, you can now go and give WhatDoTheyKnowPro a go! We hope you’ll let us know how you find it.


    Image: Sven Scheuermeier (Unsplash)

  3. Six stories from WhatDoTheyKnowPro

    Our Freedom of Information toolkit for journalists, WhatDoTheyKnowPro, will have its official launch very soon — and we’re glad to see that it’s already beginning to help generate high-profile news stories based on FOI requests.

    During development, several journalists have been putting it through its paces and offering us invaluable feedback which has helped us shape the service — and meanwhile their activity is also uncovering stories of genuine interest. These give a taste of exactly what kind of investigative stories can be supported by WhatDoTheyKnowPro, which makes requests to multiple bodies simpler, as well as organising the responses so that they’re easier to manage.

    Here are the WhatDoTheyKnowPro-generated stories we’ve heard about so far:

    Student Brexit campaign received ‘as much funding as necessary to win’

    Open Democracy and the Ferret uncovered how Vote Leave used a loophole to funnel hundreds of thousands of pounds through a student’s small-scale campaign. The story was subsequently run by multiple other news outlets and legal proceedings towards a judicial review have begun.

    British police trained officers in repressive regimes

    Scrutiny of documents from the College of Policing revealed that much of its income was coming from countries where there is concern about human rights. The story, by Lucas Amin, was run by the Guardian in September.

    Public servants and Scottish ministers paid thousands of pounds to dine with Obama

    Investigative journalism platform the Ferret uncovered this story in July, detailing how much public money was spent on senior staff attending a charity dinner with Barack Obama.

    Links between Mark Hoban and Price Waterhouse Cooper

    The Times and the Daily Mail both ran this story from Patrick Hosking, which revealed the former financial secretary’s ‘forgetfulness’ over prior links with Price Waterhouse Cooper.

    Infighting in UKIP over the name Patriotic Alliance

    Both anti-far right activists and UKIP officials tried to stop Arron Banks from registering a new political party called the Patriotic Alliance, another story run by the Ferret revealed.

    Where British holiday-makers get arrested most often

    Back in June, this story by Claire Miller and James Rodger analysed figures from the FCO on where Brits had been detained and obtained consular support, allowing them to state which countries had the most arrests, and how figures had changed over time. The story ran in the Birmingham Mail and was also picked up by other publications in the Trinity Mirror Group.

    We’re delighted to see such good use being made of WhatDoTheyKnowPro, and we anticipate many more stories emerging once it has fully launched.


    Image: Michael Pittman (CC by-sa/2.0)

  4. Alaveteli Pro: batch requests, a news story, and going international

    Jenna Corderoy, Alaveteli Professional Advocate, brings us an update on the project.

    Since our last blog post on Alaveteli Professional — our Freedom of Information toolkit for journalists, campaigners and activists — there have been a few exciting developments.

    Batch work

    The batch request feature is coming along nicely: this will allow users of the service to send one Freedom of Information request to multiple authorities and help them to easily manage large volumes of responses.

    We’re going to be working with a small group of our beta testers to develop this feature and make sure we release it in a useful and responsible form (click here to apply as a beta tester and get a year’s free access to WhatDoTheyKnowPro, the UK version of the service).

    Getting results

    We’ve been pleased to see the first news story to emerge as the result of a request made through WhatDoTheyKnowPro: a response to the Foreign and Commonwealth Office showed which are the countries where UK holidaymakers are most likely to get arrested. The full list was covered in the Birmingham Mail.

    Further afield

    But we also have plans for this Freedom of Information toolkit to go international: Alaveteli Professional will be a bolt-on option for anyone already running an FOI site on our software platform Alaveteli.

    In April, mySociety team members traveled to roll out the first such project, with Info Pro Všechny, the Czech Republic’s Alaveteli site.

    We were able to introduce beta users to the features we’ve been developing, such as the ability to keep requests private until the story has been published.

    While in the Czech Republic, we held a roundtable discussion with journalists and campaigners, swapping Freedom of Information battle stories and sharing tips and tricks for getting the best results when submitting requests for information, as well as experiences of filing requests to European Union institutions.

    mySociety was also invited to give a talk to student journalists based in Olomouc about Info Pro Všechny and Alaveteli Professional in general, discussing success stories generated from our Freedom of Information sites from around the world.

    Next up

    We’re currently working on subscription options, which will allow us to officially launch WhatDoTheyKnowPro as a paid-for service in the UK, and later in the year, we plan to introduce the Pro toolkit to the Belgian Alaveteli site Transparencia.be, which has been making a splash in Belgian politics.

    Photo by Amador Loureiro (Unsplash)

  5. WhatDoTheyKnowPro: ready for beta users

    If you’ve been watching the progress of our Freedom of Information toolkit for journalists, campaigners and activists, you might be interested to know that we are now accepting applications for pre-launch access.

    Successful applicants will have an early opportunity to put the service through its paces. WhatDoTheyKnowPro will launch as a paid-for service, but as a beta tester you’ll have up to a year’s access for free, because we’re keen to see how you’ll use it — and to hear your feedback on which features are useful.

    What’s WhatDoTheyKnowPro?

    WhatDoTheyKnowPro is our first launch of Alaveteli Professional, accessed via the WhatDoTheyKnow website and specifically for UK users who utilise FOI in their work or campaigning.

    It’s the first instance of the service we plan to make available to other Freedom of Information sites running around the world on our Alaveteli platform.

    What you’ll get

    Since we last caught up with Alaveteli Professional, we’ve made really concrete progress with several of the features that, at the time of that blog post, were just entries on our long to-do list.

    Here’s what users will access:

    • The ability to keep requests private until your story has been published
    • A powerful private dashboard that helps you track and manage your FOI projects
    • A super-smart to-do list that makes it easier to follow the progress of your requests
    • Action alerts that nudge you when it’s time to take the next step in a request

     

    screenshot-requests-78dd427b981787204f3ddfe4eda11cb1939e93263d29398d309b48f14cdb1be2  screenshot-dashboard-34a9936fd33b54729b033c322bc1bc360116ae85b2d4fec5182dd22f58283745
     

    Batch benefits

    And very soon, we’ll also be carefully rolling out the batch request features, which will allow you to:

    • Make one request to multiple authorities
    • Manage large volumes of responses and easily keep track of the status of each request
    • Get regular updates as the responses come in, without overwhelming your inbox

     
    screenshot-batch-list-7f71f614b844db3686cc6731d7725aec10e244dc327f88b5c87c02382b6f6d71 screenshot-batch-selection-d59dda9d21302f321502c8033a254a27e6cb3f2579c4ac16c5f8006eed2a43ac

     

    We’re excited about the batch feature in particular, and we know that many of our prospective users are, too. At the same time, we’ve heard some concerns that it might encourage a scattergun approach that wastes authorities’ time.

    Our planned development will ensure that people use this feature responsibly, and, consequently, get the best returns from it. This will include a prompt to send a smaller batch initially, so that the remainder of the requests can be refined based on the quality of information that is returned — there’s nothing worse that asking every council in the country for information and then realising that you’ve worded your question in a way that means you can’t use the resulting data!

    At the moment, batch requests can’t be made on WhatDoTheyKnow without help from the site administrators. We’re aware that many journalists and activists already make many batch requests outside WhatDoTheyKnow for this very reason. We’d like more of these requests to be released in public (we estimate that around 15% of UK FOI requests are made via our site): so by including this capability in WhatDoTheyKnowPro, we hope we’ll not only be steering people to use those powers sensibly, but that much more information will also end up in the public domain — maximising its usefulness.

    Apply for free access

    If that all sounds exciting, then apply here. We’d love to hear how you plan to use WhatDoTheyKnowPro.

  6. The latest from Alaveteli Professional: prototyping, testing, reducing risk

    Last time we updated you about Alaveteli professional, the Freedom of Information toolset for journalists that we’re building, we were just coming out of our discovery phase.

    Since then, we’ve made strides through the alpha and early beta part of our development process. In alpha, the idea is to build dummy versions of the tool that work in the minimum way possible — no bells and whistles — to test concepts, and our assumptions. Having thought hard about the potential problems of Alaveteli Professional, now is the time for us to try the approaches that we believe will solve them, by making prototypes of how the tool might work and testing them with a very small group of users.

    In the early stages of beta, our priority has been to get to the point where a Freedom of Information request can go through all its various processes, from composition to response, with the features that a journalist user would need. Once that’s in place, it allows us to add other features on top and see how they would integrate.

    This pattern —  discovery, alpha, beta, release — is a well-tested method by which to produce a final product that works as it should, while avoiding costly mistakes.

    Risk management

    Alpha and beta testing, perhaps unexcitingly, are all about the reduction of risk: in the words of the startup mantra, it’s good to ‘fail fast’— or rather, it’s better to know early on if something doesn’t work, rather than spend time and money on something that doesn’t fit the bill.

    So, for Alaveteli Professional, what are the risks that have been keeping us awake at night?

    We think the biggest priority is to ensure that there’s actually added value for journalists in using a service like this. Clearly, the Freedom of Information process is already available to all, whether via our own site WhatDoTheyKnow, or directly.

    We need to be able to demonstrate tangible benefits: that Alaveteli Professional can save journalists time; help them be more efficient in managing their requests; maybe help them get information that otherwise wouldn’t be released; and give them access to rich data they wouldn’t otherwise be able to access.

    For all we said about failing fast, the alpha phase also meant committing to some fairly big technical decisions that, ideally, we wouldn’t like to reverse.

    Decisions like, do we build the service into the existing Alaveteli codebase, or go for a new standalone one (we went for the former)? From the user’s point of view, should Alaveteli Professional look like a totally different site, or like a registration-only part of WhatDoTheyKnow (we chose the latter)?

    And onto beta

    As we move from alpha to beta, we’re finding out what happens when real users make real requests through the service, and making adjustments based on their feedback.

    What do they think of the way we’ve implemented the ability to embargo requests – does it make sense to them? Do they trust us to keep embargoed requests private? Are they able to navigate between different interfaces in a way that seems intuitive? mySociety designer Martin has been figuring out how to take the cognitive load off the user and give them just the information they need, when they need it.

    We’re also returning to prototyping mode to work out how to implement new features, like the ability to send round robin requests to multiple authorities, in an effective and responsible way. The other half of our design team, Zarino, has been showing us that a slideshow in presentation mode can be an effective tool for demonstrating how users might interact with an interface.

    As we continue to round out the feature set in the UK, we’re also cooking up plans in the Czech Republic so that later in the year we can present the tools to a new audience of journalists there and again, use their feedback to make the tools more flexible so that they can be used in different jurisdictions.

    As you can see, there’s lots going on, and we’re all really excited to be finally getting some real life users in front of the tools that we’ve been thinking about, and working on, for so many weeks. Don’t forget to sign up to the mailing list if you’d like to keep up with Alaveteli Professional as it develops.


    Image: Jeff Eaton (CC BY-SA)

  7. Why we believe a digital FOI tool can improve Kenyan journalism, and empower citizens at the same time

    Back in December we told you about our application to innovateAFRICA, for funding to launch our Alaveteli Professional project in Kenya.

    Well, we’re delighted to say we’ve been shortlisted for a grant. innovateAFRICA judges will take a few weeks to consider shortlisted applications, and winners will be announced on 30th January.

    In the meantime, we thought we’d ask the project’s coordinators, Henry Maina from ARTICLE 19 East Africa and Louise Crow from mySociety, to describe the project in a bit more detail and explain why they think it’s so important.

    What is the Alaveteli Professional project?

    Louise: Alaveteli Professional is a new toolset that we are currently building as a companion service to our existing Alaveteli software. Alaveteli is mySociety’s open-source platform for making public freedom of information (FOI) requests to public bodies.

    Alaveteli Professional will provide journalists and those who use FOI in their work with extra functionality and training to ease the process of raising, managing and interpreting FOI requests, which can be a very time consuming and overwhelming task. This is so that they can spend their valuable time on creating more high-impact journalism and research that holds public authorities to account.

    Why bring the Alaveteli Professional project to Kenya?

    Henry: The project will enable more Kenyan journalists to utilise one critical tool in their armoury: namely the Freedom of Information law enacted on 31st August 2016. It will also complement our earlier training of 25 journalists on the FOI law.

    Louise: innovateAFRICA funding will allow us to bring our newly developed toolset to the Kenyan context. The toolset will have already been tried and tested by journalists in the UK and Czech Republic, so we’ll use examples of how these European journalists have successfully used the platform to generate stories in our trainings with Kenyan media. Simply building these tools is not, on its own, enough. For this reason, the Alaveteli Professional project in Kenya will also involve refining the tools for the Kenyan context, the training of journalists, the creation of support materials and the provision of direct assistance in making and analysing requests.

    From ARTICLE 19’s experience of training Kenyan journalists on the new FOI law, how will the Alaveteli Professional project help them with their work?

    Henry: ARTICLE 19 has trained journalists on the Freedom of Information laws in Rwanda, Kenya, Uganda and South Sudan. In all our past training, we created manual request protocols and follow-up required making telephone calls. The Alaveteli Professional project will help most journalists to easily file, track and share information about information requests in an easy to engage, review platform.

    Why is it so important for journalists and citizens alike to hold authorities to account in Kenya?

    Henry: First, journalists and citizens are keen to understand why and how their public servants and officials take decisions. Second, citizens have a right to participate in the management of public affairs and effective engagement is only possible if the citizens are well informed.

    Will the project also benefit Kenyan citizens who aren’t journalists?

    Louise: Yes. Providing journalists with the extra toolset requires us to first install a standard version of Alaveteli. Therefore, alongside citizens in 25 other countries in the world, Kenyan citizens will be able to use the platform to easily send requests to public authorities, or, as all responses to requests are published on the site, browse already-released information.

    Citizens will also benefit even if they don’t use the site at all: they’ll benefit from news stories that expose corruption and mismanagement or missing funds and so on, and thus hold those in power to account.

    What impact will the project have on Kenyan information officers/civil servants?

    Henry: The project is likely to have great impact on Kenyan information officers and public officials. First, it will offer an objective platform to recognise and reward civil servants that enhance access to information as they will be able to manage requests more efficiently. Second, given the trend in questions, officers will be aware of the information that they can and should proactively disclose to lessen individual requests. Third, it will bolster ARTICLE 19’s ongoing work of training information officers that seeks to help them better understand the law and their obligations under it. Four, most of the government decisions will gain traction with citizens as there will be publicly available information on why and how such decisions were arrived at.

    What lasting impact do you hope the project will achieve?

    Henry: The Kenyan government will be more transparent and accountable, journalists will be more professional and their stories more credible and factual, allowing the country to entrench democratic values.

    Louise: As with all our Alaveteli projects, we hope the project will amplify the power of Freedom of Information and open government, by giving a broad swathe of citizens the information they need to hold those in power to account, and to improve their own lives.

    How you can help

    So there you are — a little more detail on why we hope to bring Alaveteli Professional to Kenya. We hope you can see the value as much as we can! If so, and you’d like to help support the project, please do tweet with the hashtag #innovateAFRICA: every such public show of support brings us a little closer to winning the grant.

    Image: The iHub (CC)

  8. Forward into alpha: what we’ve learned about Alaveteli Professional

    As we recently mentioned, one of mySociety’s current big projects is Alaveteli Professional, a Freedom of Information toolset for journalists.

    It’s something we wanted to build, and something we believed there was a need for: but wanting and believing do not make a sound business case, and that’s why we spent the first few weeks of the project in a ‘discovery’ phase.

    Our plan was to find out as much as we could about journalists, our prospective users — and particularly just how they go about using FOI in their work. Ultimately, though, we were seeking to understand whether journalists really would want, or need, the product as we were imagining it.

    So we went and talked to people at both ends of the FOI process: on the one hand, journalists who make requests, and on the other, the information officers who respond to them.

    Since we’re planning on making Alaveteli Professional available to partners around the world, it also made sense to conduct similar interviews outside the UK. Thanks to links with our Czech partner, running Informace Pro Všechny on Alaveteli, that was a simple matter. A recent event at the Times building in London also allowed us to present and discuss our findings, and listen to a couple of interesting expert presentations: Matt Burgess of Buzzfeed talked about some brilliant use of FOI to expose criminal landlords, and listed FOI officers’ biggest complaints about journalists. Josh Boswell of the Sunday Times was equally insightful as he ran through the ways that he uses FOI when developing stories.

    These conversations have all helped.

    The life of an investigative journalist is never simple

    Alaveteli Professional process diagram drawn by Mike Thompson

    The insights our interviewees gave us were turned by Mike Thompson (formerly of mySociety, and brought back in for this phase) into a simple process model showing how journalists work when they’re pursuing an investigation using FOI.

    After conceiving of a story that requires input from one or more FOI request, every journalist will go through three broad phases: research; request and response; and the final data analysis and writing. The more complicated cases can also involve refused requests and the appeals process.

    For a busy working journalist, there are challenges at every step. Each of these adds time and complexity to the process of writing a story, which is an anathema to the normal daily news cycle. FOI-based stories can be slow, and timing unpredictable — editors do not particularly like being told that you’re working on a story, but can’t say when it will be ready, or how much value it will have.

    During the research phase diligent journalists will make a time-consuming trawl through resources like authorities’ own disclosure logs and our own site WhatDoTheyKnow (or its equivalents in other countries), to see if the data they need has already been released.

    Where a ‘round robin’ request is planned, asking for information from multiple authorities — sometimes hundreds — for information, further research is needed to ensure that only relevant bodies are included. In our two-tired council system, where different levels of authority deal with different responsibilities, and not always according to a consistent pattern, that can be a real challenge.

    Wording a request also takes some expertise: get that wrong and the authorities will be coming back for clarification, which adds even more time to the process.

    Once the request has been made it’s hard to keep on top of correspondence, especially for a large round robin request. Imagine sending a request to every council in the country, as might well be done for a UK-wide story, and then dealing with each body’s acknowledgements, requests for clarifications and refusals.

    When the responses are in journalists often find that interpretation is a challenge. Different authorities might store data or measure metrics differently from one another; and pulling out a meaningful story means having the insight to, for example, adjust figures to account for the fact that different authorities are different sizes and cater for differently-dispersed populations.

    Sadly, it’s often at this stage that journalists realise that they’ve asked the wrong question to start with, or wish that they’d included an additional dimension to the data they’ve requested.

    What journalists need

    As we talked through all these difficulties with journalists, we gained a pretty good understanding of their needs. Some of these had been on our list from the start, and others were a surprise, showing the value of this kind of exploration before you sit down to write a single line of code.

    Here’s what our final list of the most desirable features looks like:

    An embargo We already knew, anecdotally, that journalists tend not to use WhatDoTheyKnow to make requests, because of its public nature. It was slightly sobering to have this confirmed via first person accounts from journalists who had had their stories ‘stolen’… and those who admitted to having appropriated stories themselves! Every journalist we spoke to agreed that any FOI tool for their profession would need to include a way of keeping requests hidden until after publication of their story.

    However, this adds a slight dilemma. Using Alaveteli Professional and going through the embargo-setting process introduces an extra hurdle into the journalist’s process, when our aim is, of course, to make the FOI procedure quicker and smoother. Can we ensure that everything else is so beneficial that this one additional task is worthwhile for the user?

    Talking to journalists, we discovered that almost all are keen to share their data once their story has gone live. Not only does it give concrete corroboration of the piece, but it was felt that an active profile on an Alaveteli site, bursting with successful investigations, could add to a journalist’s reputation — a very important consideration in the industry.

    Request management tools Any service that could put order into the myriad responses that can quickly descend into chaos would be welcome for journalists who typically have several FOI requests on the go at any one time.

    Alaveteli Professional’s dashboard interface would allow for a snapshot view of request statuses. Related requests could be bundled together, and there would be the ability to quickly tag and classify new correspondence.

    Round-robin tools Rather than send a notification every time a body responds (often with no more than an acknowledgement), the system could hold back, alerting you only when a request appears to need attention, or send you status updates for the entire project at predefined intervals.

    Refusal advice Many journalists abandon a request once it’s been refused, whether from a lack of time or a lack of knowledge about the appeals process. WhatDoTheyKnow Professional would be able to offer in-context advice on refusals, helping journalists take the next step.

    Insight tools Can we save journalists’ time in the research phase, by giving an easy representation of what sort of information is already available on Alaveteli sites, and by breaking down what kind of information each authority holds? That could help with terminology, too: if a request refers to data in the same language that is used internally within the council, then their understanding of the request and their response is likely to be quicker and easier.

    Onwards to Alpha

    We’re currently working on the next part of the build — the alpha phase.

    In this, we’re building quick, minimally-functional prototypes that will clearly show how Alaveteli Professional will work, but without investing time into a fully-refined product. After all, what we discover may mean that we change our plans, and it’s better not to have gone too far down the line at that point.

    If you are a journalist and you would like to get involved with testing during this stage and the next — beta — then please do get in touch at alaveteli-professional@mysociety.org.


    Image: Goodwines (CC by-nc-nd/2.0)

  9. Tweet if you’d like to bring Alaveteli Professional to Kenya

    We’d love to bring our Alaveteli Professional project to Kenyan journalism.

    So with Article 19 East Africa, we’ve applied to innovateAFRICA, which is seeking disruptive digital ideas to improve the way that news is collected and disseminated.

    As of this year, Kenyan citizens are enjoying a new right to know, thanks to their Freedom Of Information Act, pending since 2007 and finally passed this year.

    Alaveteli Professional will provide Kenyan journalists with a toolset and training to help them make full use of FOI legislation, so they can raise, manage and interpret requests more easily, in order to generate high-impact public interest stories.

    But the project will also bring benefits to all Kenyans. By helping journalists and citizen reporters to make full use of the Act, it will ultimately make it easier for everyone to hold power to account.

    How you can help

    Now here’s the bit you need to know about: please tweet using the hashtag #innovateAFRICA explaining why you think Alaveteli Professional in Kenya is an important digital solution.

    This will demonstrate that you agree that Alaveteli Professional is worthy of innovateAFRICA’s support — every tweet helps to give our application more traction.

    Tweets from everyone are welcome, but yours will have extra leverage if you’re a mySociety partner, a Kenyan journalist or activist who would use the project, a funder or a digital innovator yourself.

    Please use your 140 characters to help us bring better FOI capabilities to Kenya! And don’t forget that hashtag: #innovateAFRICA.

    Thank you.

     

    Image: Innovate Africa

     

  10. Alaveteli Professional – learning more about journalistic use of Freedom of Information

    In the last few weeks, we’ve started conducting background research interviews for our new project, Alaveteli Professional. Alaveteli Professional will be a companion service to Alaveteli, our Freedom of Information platform – initially it will be aimed specifically at journalists, but it should be of interest to anyone who uses Freedom of Information in their work.

    Why are we doing this project?

    Alaveteli Professional is an unusual project for mySociety. Our mission is to create digital tools that empower citizens in their interactions with the state, and people in power. Usually that means that we create tools which we intend to be used by as broad a range of people as possible – we think a lot about how to design and build for people in their role as citizens, which is a role we all experience. But with Alaveteli Professional, we’re focusing on journalists, a specific professional group. Why is that?

    Citizen empowerment doesn’t always happen by direct interaction with institutions. Feeling empowered and capable of affecting what happens in your community requires knowing what’s going on in your community. Although models of journalism are changing, whether you’re getting your news from The Times, or from Buzzfeed, whether it’s funded by a paywall or by crowdsourcing, it’s hard to imagine a future in which ordinary people can be well-informed, without specialists doggedly asking questions of power, putting information from different sources together, and helping make sense of what’s going on.

    Alaveteli-powered sites like WhatDoTheyKnow have been successful in giving ordinary people a simple way to ask questions of government and to share the responses with everyone automatically online. But we know that the way the sites work doesn’t always match the needs of someone who’s working on assembling a bigger story that they may want to break elsewhere. We’d love to see the work put into Alaveteli so far also go to serve the goal of informing people through high quality public interest stories in media platforms with a long reach.

    That’s why we were delighted to get funding for the project from the Google Digital News Initiative, which aims ‘to support high quality journalism and encourage a more sustainable news ecosystem through technology and innovation’.

    What we’re doing

    The initial research for the project has been an interesting and exciting process, and not just because it has meant actually ‘leaving for work’ in the morning, rather than spending the day entirely in the virtual world of remote working. For me, one of the real joys of working on digital tools is the opportunity to spend some time in different domains of life and think about how they work.

    We’ve been talking to media professionals who use Freedom of Information requests in their jobs, trying to understand what parts of the process are painful or unnecessarily time consuming. We’re also talking to FOI officers, and other people who’ve thought deeply about journalistic use of FOI, in an effort to understand the ecosystem of people and motivations – and answer questions of who is doing what and why. It’s been a real pleasure to explore these questions with people who’ve been incredibly generous with their time and ideas.

    The process of making a Freedom of Information request can sometimes seem quite similar to an adversarial legal system – with the requester pitted against an institution that’s reluctant to release information, and FOI law defining the obligations, exemptions, and public interest tests that set the landscape in which the two sides are in conflict. But as with any other domain, the more you dig into it, the more interesting complexity you find in both sides, and in the interaction between the two.

    There are freelance journalists working against the clock to turn around a story they can sell, but also data journalism groups in larger institutions making frequent requests as part of ‘business as usual’, and pushing out stories to their regional colleagues. As you would expect, there’s competition between journalists and media institutions, but also surprising opportunities for collaboration and shared resources. There’s a significant amount of collaboration between requesters and authorities – in some cases producing nuanced national public-interest data sets that neither could generate alone. There’s a lot of diversity in the authorities that are subject to Freedom of Information law – from tiny schools and parish councils to huge central government departments, police and health authorities. There’s also still variation in how different authorities store similar data and how they respond to FOI requests.

    What’s next?

    At this point, we’re trying to get the best sense we can of both the details and the big picture. We’re also starting to ask where we could reduce friction, encourage responsible practices, save time in such a way that it benefits the system as a whole, and increase the chance of ordinary people becoming better informed about what is being done with their money and in their name by institutions. It’s an exciting part of the project, as we start to discard some of the preconceptions we had about what might be useful, and get more confident in the value of others. I’m looking forward to starting to put those ideas into practice in the form of simple prototypes that we can put back in front of people.

    Image: Dean Hochman (CC by 2.0)