We’ve just released version 0.32 of Alaveteli, our open source platform for running Freedom Of Information sites. Here are some of the highlights.

Making correspondence threads easier to navigate

Thanks to our designers, it’s now possible to collapse individual messages in a correspondence thread in order to focus on just the parts you’re trying to read. Plus you can quickly collapse (or expand) all the messages in the thread using the “Collapse all” and “Expand all” links from the “Actions” menu.

Alaveteli Pro users gain the additional benefit of a redesigned sidebar which allows for easier navigation of lengthy correspondence and avoids having to scroll to the top of the request thread to update its status. See Martin’s full explanation here.

Better password security

We’ve started enforcing stricter password length constraints wherever a password is set or updated to help users keep their accounts secure. And we’re also using a stronger encryption method for storing password data, using bcrypt rather than the older SHA1 algorithm to obscure the actual password. (Be sure to run the rake task documented in the release upgrade notes to upgrade secure password storage for all existing users.)

You can read more about what this does and why it’s important if you’re interested in the technical details behind this upgrade.

Authorities not subject to FOI law

We’ve adopted WhatDoTheyKnow’s foi_no tag for authorities to indicate that although the authority is listed on the site, it is not legally subject to FOI law. This could be for advocacy purposes – if it’s felt an authority should be covered by legislation – or where the authority has agreed to respond on a voluntary basis.

Adding the foi_no tag now causes an extra message to appear under the authority’s name on their page and on all related requests, and removes language about legal responsibilities to reply from the messages sent to users.

To improve the UI, we’ve made a similar change for authorities with the eir_only tag to make it clearer that such authorities are only accepting requests about the environment.

(Don’t worry admins, you don’t need to remember all this – we’ve updated the documentation on the edit page to reflect the new functionality!)

Improvements for site admins

We’ve made it easier for admins to ban users who sign up to post spam links in their profile. There’s now a “Ban for spamming” button which is available on the user edit page or as soon as you expand the user’s details in the listing rather than having to manually edit user metadata.

We’ve also made it harder to leave requests flagged as vexatious (or “not_foi”) in an inconsistent state. Previously the site just assumed that vexatious requests would always be hidden. Now the admin interface enforces the hiding of vexatious requests by showing warnings when a request is set as vexatious while it’s visible on the site, and prevents the updated request from being saved until a valid state is selected.

Announcements

And last but not least – introducing the new Announcements feature!

Easier popup banner management

Site admins will be relieved to hear that they can now update the popup banner message on the site without needing to schedule developer time.

This feature supports multi-language sites so if you set the announcement for your main (default) language, it will appear across all language versions that you have not added a specific translation for.

Admin-only announcements

You can set announcements that will only be seen by fellow administrators when they visit the summary page. (If you’re running a Pro site, you can also have announcements that will only be seen by your Pro admins.)

Pro announcements

Announcements for Pro users appear as a carousel at the top of their dashboard. So far we’ve used it on WhatDoTheyKnow Pro to publicise new features, offer discount codes, and encourage people to share their published stories with us.

The full list of highlights and upgrade notes for this release is in the changelog.

Thanks again to everyone who’s contributed!

What happens when your site is the target of a major spam attack? That wasn’t something we were particularly keen to find out —  but it’s a scenario we’re now fully acquainted with. That’s all thanks to a recent concerted assault on our Freedom of Information site WhatDoTheyKnow.

All is calm again now, and hopefully, as a user of the site, you’ll have noticed very little. Yes, you’ll now have to complete a recaptcha when creating a new request*, and you might have discovered that the site was inaccessible for a couple of hours. Beyond that, everything is pretty much as it was.

From our point of view, though, it was an emergency situation that meant that several of us had to put down what we were doing and join in with some quick decision-making.

Alert, alert!

It was around 12:30 on a Wednesday afternoon when Richard, one of the volunteers who helps to run WhatDoTheyKnow, noticed unusual activity on the site.

WhatDoTheyKnow was created to help people send requests for information to public authorities — a service for the wider good. Unfortunately, at this point, it was also doing something quite the opposite of good: it was providing the means for unknown sources to send those same authorities hundreds of spam messages.

We’d like to apologise to those who were on the receiving end: clearly, spam is a nuisance for everyone who receives it and we’re unhappy to have played any part in its perpetuation.

We also had a secondary concern. It seemed likely that recipients would mark these incoming emails as spam. When enough people had done that, email providers would see us as an insecure source, and block all our messages, valid or otherwise, potentially preventing the WhatDoTheyKnow system from running efficiently.

A little fire-fighting? That’s actually situation normal

Spam is an obvious example of the site being abused, but it’s perhaps worth mentioning that we work hard on many levels to ensure that WhatDoTheyKnow is only used for its core purpose: the requesting of information under the FOI Act.

And note that we’ve always been careful to protect against abuse. WhatDoTheyKnow does already have several measures in place as standard: we only allow one account per email address; we verify that email addresses are genuine; and we cap the number of requests that users can make each day (a restriction that we only override for users who are demonstrably making acceptable use of our service). We reckon that these measures very much helped to reduce the impact of the attacks.

Action

After a quick discussion between the volunteer team, trustees and mySociety staff, we took the site offline to give us time to work on a solution while stopping any more spam from being sent.

Of course, we then removed all the spam requests and comments from the site and banned the accounts that had made them. We also contacted the affected bodies to let them know what had happened and to assure them that we were taking steps to deal with it.

When we brought the site back up, a couple of hours later, we did so cautiously and with new restrictions and safeguards in place.

Spam ‘requests’ had been sent over a period of about 13 hours. There were in the region of 800 made, though only about 500 actually got sent to authorities. Additionally, around 368 spam comments were left on existing requests. These relatively small numbers lead us to believe that they were being made manually.

Time to breathe… or nearly

Once we’d discovered the issue, dealing with it and getting the site back up and running took us 2.5 hours.

Job done — so now we could sit back and relax, eh? But no: the next day we discovered that a couple of other sites running on the Alaveteli platform,  AskTheEu and New Zealand’s FYI, were being subjected to the same attacks.

So we rolled out the changes we’d made on WhatDoTheyKnow to make them available to all Alaveteli users. And then, finally, we could get back to the everyday work we’d been doing before — making our sites better for you, and the other nice non-spamming people who use them.

—

* We’ll be looking at removing it as soon as we can, though, as recaptcha doesn’t offer a very accessible experience for many disabled people. Meanwhile, we can manually remove the recaptcha for specific accounts, so if you’re struggling with it, contact the team to implement this exemption.

—
Image: Mark Granitz (CC by-nc-nd/2.0)