So, there are now over 600 petitions in the petitions system, and we’re getting a steady stream of appeals from our users to add categories.

I’m posting to ask how you all think we should handle this. It seems to me that there are a few options:

• Ask petition creators to pick one very basic top level category of no more than 10 or so, taken from a hierarchical taxonomy like the one the BBC uses.
• Ask petition creators to pick the top level and the subsequent sub-levels to be more specific.
• Go all web 2.0 and simply ask people to tag their petitions with some key words

More than just thinking about the overall philsophy I’d also appreciate thoughts on design. When you come to the homepage, how should the category system be presented to you? Tricky stuff, and I’d really appreciate your thoughts.

Since the petition system went out properly on Wednesday, we’ve been absolutely buried in an avalanche of changes, fixes, feature additions and massive massive amounts of email. I thought that you might be interested as to what sort of stuff has happened in the first two days:

• Email has taken over our lives. Matthew has responded to over 200 emails since yesterday morning, and I was up at 4am last night just trying to cope with the rate of incoming of mail. Francis, who’s now in Canada, then heroically took up the baton and responded to mail all (UK) night! Many if not most of these mails are giving us suggestions, as well as bug reports, problems with email and bits of praise and the odd conspiracy theory.
• Changes made to cope with expats and overseas military personnel.
• Phoned Hotmail to stop their system from eating 95% of the confirmation messages being sent to Hotmail accounts!
• Redesigned the automated mails no10 get telling them there’s a new petition (they’ve had over 500 of these mails, so they need to be clear and easy to read!)
• Made the rejected petitions system more granular, so that if a petition has to be rejected, and part of it has to be hidden (say, if it is libellous), then it only hides that bit, not the whole thing. Maximum transparency is the goal, you see.
• New options added to sort the list of all petitions in different ways, by number of signatures being the most asked for.
• Limited the length of “more info” fields so people can only write long rants, rather than really really long rants 🙂
• Special cased people with AOL accounts, so that their, erm, nonstandard email clients can actually cope with the confirmation links.
• Made several fixes to the processes involved in sending out confirmation mails.
• Made RSS changes and improvements.
• Updated various bits of text, like providing examples of what “party political” means. The BBC initially wrote that this meant no pledges mentioning controversial issues like Iraq, which was grabbing quite the wrong end of the stick about the nature of the rules. Now we have some complaining emails saying we’re being too liberal!
• Compiled a big list of user suggestions and fixes on the wiki.
• Made the rejection criteria in the Ts&Cs actually match the ones in the admin interface.
• Installed a stats packages to watch what’s going on.
• Added facility to search petitions
• Improved/fixed logging
• Added link and text pointing to the open source code.

I’ve probably missed some – I’m sure Matthew, Chris, Francis and Ben will let me know!

I’m very pleased to announce that the petitions system we’ve built for 10 Downing Street has gone live today.

I’m very grateful for the hard and often inspired work put into this by Chris Lightfoot and Matthew Somerville, as well as the civil servants who have helped to build a petitions system which I believe is in a real class of its own.

The most notable features are:

1. Petitions are accepted and published, regardless of the political slant of the petition. However, if they break the Ts&Cs (a petition that doesn’t actually ask for any action, for example) then they are put on a special rejected petitions page: they don’t just vanish. We think this transparency feature is probably unique.

2. The site is being launched in beta, and will change over time. This might seem too commonplace to note for many of you, but it reflects a willingness to see a public IT service evolve in response to users, not simply fulfil a contract agreed in advance. mySociety exists partly to spread good practice in the public sector, and we think this is a nice example of that in action.

3. The code, including Chris’s amazing high-load optimised engine, is all open source.

Any questions? Come into our chat channel at www.irc.mysociety.org or mail us at hello@mysociety.org.

We just had our irregular weekly meeting, which we do most Mondays using a conference call. I thought I’d just write up what we’re all up to this week.

• I’m continuing to test the ePetitions site for 10 Downing Street, and developing an interesting branded version of PledgeBank for CAFOD (more when it launches).
• Matthew is going to look at various things that need doing on PledgeBank and WriteTothem. For PledgeBank more chivvying emails, I think something like this ticket but not exactly. For WriteToThem, various bits of code to do with how we handle error cases.
• Chris is making more pretty maps for the Department for Transport.
• Tom is working out in detail how we’re doing to spend the money from DCLG which has finally come through. It’s mentioned in this post, look for “e-Innovations Product and Marketisation strand via Kirklees MBC”. Which means, we’re being paid to do proper marketing and sales of branded version of our services, such as WriteToThem, PledgeBank, and Neighbourhood Fixit. He’s also chasing up some interesting people met at a conference in Eastern Europe (Bratislava, I think?) last week.

Please ask questions in the comments – for example, if you’d like us to post about particular things on this blog.

Much of my August seems to have been absorbed with maintenance tasks.

For example, Chris and I spent a few days tightening up WriteToThem’s privacy. I made sure the privacy statement correctly describes what happens with backup files, and failed messages. I reduced the timeouts on how long we keep the body of failed messages. I made sure we delete old backup files of the WriteToThem database. I wrote scripts to run periodically to check that no bugs in our queueing demon can accidentally mean we keep the body of messages for longer than we say. I added a cron job to delete Apache log files older than a month for all our sites. As AOL know to their cost, the only really private data is deleted data.

Earlier in the month, I handled some WriteToThem support email for the first time in ages. We get a couple of hundred messages a week, which Matthew mainly slogs through. It’s good for morale to do it, as we get quite a lot of praise mail. It is also hard work, as you realise how complicated even our simple site and the Internet are, and it leads to fixing bugs and improving text on the site. I made a few improvements to our administration tools, and things like the auto-responder if people reply to the questionnaire, to try and reduce the amount of support email, and make it easier to handle.

I did some more work on the geographically cascading pledges (like this prototype one), but I’m still not happy with them. In the end, I realised that it is the structure of wording of the pledge that is the key problem. Our format of “If will A but only if N others will B” just isn’t easily adapted to get across that the pledge applies separately in different geographically areas. Working out how to fix that is one of the things we’ll brainstorm about in the Lake District (see below).

The last couple of days I’ve been configuring one of our new servers who is called Balti, and getting the PledgeBank test harness working on it. Until now, it has only been run on my laptop. This is partly heading towards making a proper test harness for the ePetitions site, running on a server so we properly test nothing can be broken before deploying a new version.

Matthew has wrapped up the TheyWorkForYou API now, and is working on Neighbourhood Fixit next. Chris has been doing lots more performance work for the ePetitions site. And he’s been making some funky monitoring thing to detect PostgreSQL database lock conflicts, which we get occasionally and are hard to debug

Tom’s in Berlin at the moment, he gave a talk last night, and I think has been to see some people from Politik Digital. As we’ve been discussing on the mySociety email list, there’s an EU grant we’re likely to apply for in collaboration with them.

On Friday, we’re all going to the Lake district for a week, with some of the trustees and volunteers intermittently. We very conveniently and cheaply all work from home, so it’s good and necessary to meet up for a more sustained period of time at least once a year. Last year we were in Wales.

No lolling about in the sun for us, as we follow an endless chain of projects through the hot months. Inured to hasslebot, we’ve not been posting to this blog much. Instead, busy working on, or soon about to work on:

• The ePetitions site for Number 10
• On a syndicated version of PledgeBank for someone’s large global warming campaign later in the year, and another for a fundraiser for a Brazilian NGO
• Making more maps (like these) for the Department for Transport
• Adding an API to TheyWorkForYou, paid for by an award from the Department for Constitutional Affairs
• Meetings endless meetings. I’ve given up trying to track Tom meeting people, and just assume at all times he is in an important meeting.
• Supporting all our existing sites – customer support emails, nursing parliament screen scrapers, fixing up WriteToThem contact details, making sure our servers don’t break.

And that’s without mentioning Neighbourhood Fix-it and the call for proposals. Later in the year. Have I missed anything?

Have a good weekend!

I haven’t posted here for a while – in late May / early June I was on holiday in China for a month. After 18 months of hectic mySociety with only a break for a week, I needed it!

Since I got back, I’ve been busy on the no. 10 petitions site, which in the tag-team coding way that we do, Matthew has now taken over for a bit. And I’m doing something I’m hesitatingly calling “geocascading pledges”.

I’m not quite sure what to call it yet. It lets you make one pledge, but whose target counts separately in different towns. When someone signs up, they also choose a place. Rather than having to make a separate pledge for everywhere in the world, you can just make one.

Here’s an example pledge, which is also real (more on that later), so do sign it if you like. And then post your thoughts in the comments – it needs a bit of polishing yet.

mySociety has been asked to do some work on an online petition system for Number 10 Downing street. mySociety is delighted to have been asked to bring our experience to the task, and aims to develop a new tool as easy to use, popular and trustworthy as our other non-partisan democractic web sites. In particular we are pleased to have jointly designed a system in which moderation is extremely transparent, and which we hope can be used as an exemplar of good practice elsewhere.

Francis and Matthew have just started coding, and we plan to have a public beta running later in the year. All the code that is generated by mySociety (and which isn’t specific to the No10 site, like logos) will of course be available under open source licences, and we will be delighted to talk to other organisations who are interested in petition systems themselves.

Author: Matt Hall

What NEED does this meet?

The recent Power report into parlimentary reform recommended that any petition with more than 2 million signatories should be automatically given a referendum. The power of petitions on the internet seems to be something that could have a real impact. This idea attempts to make the gathering of petition signatories easier.

This would benefit those seeking to engage with local politics and also serve to highlight national causes, bringing likeminded people together online.

What is the APPROACH?

How about a page like pledgebank where people can post petitions and sign up to them online. People could suggest petitions, with arguments for their point of view and what exactly the petition would address.

Subsequently, petitions would be viewable online and sortable by area or issue and providing email addresses when siging-up means that mailing lists for signatories could be easily compiled for updates on the progess of the petition/issue.

What are the BENEFITS to people?

This would make it easier to engage with local politics and also mean that gathering large petitions on important single issues becomes far less labourious.

What is the COMPETITION?

I’ve not heard of anything exactly like this, but it has obvious similarities to pledgebank. In this case, however, the aim of the site is slightly different. rather than “i’ll do this if x other people do it with me” it’s more of “let’s let those in charge know that x people feel strongly about this”, which would seem nicely complimentary.

What BUDGETS & LOGISTICS are required?

Given that pledgebank is built and running, the amount of time and effort involved in constructing petitionbank would not appear to be that great.