PledgeBank is now available in Esperanto. Getting this translation up and running was very instructive. During the process, we found numerous strings missed from the translation file, a variety of interesting bugs, and had to make various improvements all over the place (things that are the same in English whether something is singular or plural are not so in other languages, and the fonts that we use for PDF posters don’t contain the glyphs needed, to name but two examples).

From a technical point of view, we now have a suite of scripts that bring our main .po file up to date with what’s currently on the site (including coping with oddities like email templates, JavaScript text), merge this in with the translations that already exist (so nothing is lost), and update everything on the site, giving stats of how much has been translated in a particular language. Very handy.

Many thanks go to Tim Morley, the translator, who now finds himself having to respond to any Esperanto contact emails we get (sorry!) 🙂

Spanish also went live recently (thanks Hugo!), Russian and Ukranian are both near completion, German is on its way, and there are others we’ve heard about too, like Greek and Polish.

PledgeBank is quite an unusual site. Many international websites simply need translation (e.g. Debian in Chinese), there aren’t any data items which vary between regions. Others have multiple international markets, with a special website tweaked for each one (for example Amazon in Canada, which has some French and English text on every page).

PledgeBank is slightly different. First of all the interface needs translating into other languages, like Debian. And we don’t quite have markets like Amazon. Partly this is because we don’t yet know what our markets are, so we just make sites for every country and language combination. We have pledges, which have both a local area and a language associated with them. We’ve also got global pledges.

All this means that sometimes pledges and text in multiple languages gets shown on one page. For example, if your browser is configured for the Brazilian language, and you are in Brazil, then www.pledgebank.com will look like this. At the time of writing there is only one Brazilian pledge, so below it we show some global pledges in English as examples.

We use some software called GNU gettext to do our translations. Obviously, I’m not telling the truth – people do the translation, gettext just substitutes the translations into the pages. It’s a great piece of software, simple, old, well used and supported, with good tools for translators to update translations with.

For some time there’s been a bug in PledgeBank. On certain pages the language can change back and forth several times, and gettext would start returning translations for earlier languages rather than the current one. I’m setting the LANG environment variable to tell it what language to use. After much debugging and an email to GNU, it turns out that this is to do with gettext’s cache. The cache was behaving differently on FreeBSD and Linux, which was confusing me even more.

To clear the cache you rebind the text domain, that is call textdomain(textdomain(NULL)), after changing the environment variable. This makes everything work happily everywhere. And the main point of this post is to get that nugget into search engines, so anybody else with the same problem has a hope of finding out..

For pretty well the last week now I’ve been adding various new messages and alerts to PledgeBank. First up were a set of emails which go to creators. Two of these are “chivvying” ones, which remind the creator to do publicity, and give them some advice.

Quite a few people make pledges and don’t realise that they have to publicise them themselves. Ambient traffic to PledgeBank will get you some sign ups, especially at the start of last week when we had lots of media attention, but pledge creator’s also have to do some of their own work. So we chivvy the creator once after a few days if they don’t have many sign ups.

Secondly, even if you’ve got some publicity and some signups, sometimes you just haven’t got a high enough rate to reach your target on time. I’ve added some code which checks signup rate on average over the last week (for pledges at least a week old). If things were to carry on at that rate until the deadline, and the pledge still wouldn’t succeed, it sends an email with ideas about publicity/marketing of your pledge. Now we just have to hope this is useful rather than annoying…

Another thing we noticed is that lots of pledge creators hadn’t sent a message to their signers after their pledge succeeded. Admittedly this was partly because of a bug in our announce message code (there was no action= in one of the forms, which caused problems on some browsers). So I also added a successful pledge reminder email which is sent a few days after success.

All the stuff above was done using the nice “message” tables which Chris made a while ago. They make it easy to send messages which go out only once, and to creators, signers and late signers as appropriate. However, for the next set of emails, some new code was needed. These are optional “alerts”. For example, to email you when new comments are added to a pledge, and when there is a new local pledge in your area. So, the last day or two I’ve been coding that.

So, it’s my turn to write something here. Well, this’ll be short. As Francis mentions below, PledgeBank launched today, and everything’s gone reasonably smoothly, with the exception of some tedious PHP bug we haven’t tracked down yet. With any luck the new version of PHP will fix it, so there won’t be hours of painful debugging to do. (I could bore you for hours with my opinions of PHP — actually, I could probably shorten that quite a lot if I were allowed to swear, but this is a family-friendly ‘blog — but let’s just say that fixing PHP problems Is Not My Favourite Job.)

Instead I’ll say what I’m doing right now, which is beginning to add geographical lookup to PledgeBank. At the moment we ask pledge authors for a country (though it can either be “UK” or “global” at the moment), and, if they’re in the UK, a postcode. The idea is to “georeference” (i.e. look up the coordinates of) the postcode, though we don’t actually do that yet. So I’m modifying the database a bit to store coordinates (as a latitude/longitude, so that we don’t have to write a separate case for every wacky national coordinate grid) and generalise the notion of “country” so that we can let non-Brits actually put in their own countries when they create pledges.

Other things we’ve discovered today:

• People are confused by the “(suspicious signer?)” link next to signatures on each pledge page — several people thought that we were reporting our suspicions of the signer. You probably think that’s stupid, but if so that’s only because you’re familiar with sites that have this sort of retroactive moderation button everywhere. Actually it’s us that’s being stupid and we’re going to remove it until we have a better way to implement it — at the moment we think we’re mostly on top of the occasional joke/abusive signature.
• People are confused by the pledge signature confirmation mail, which currently reads (for instance),
  

  Please click on the link below to confirm your signature on the pledge at the bottom of this email.

  http://www.pledgebank.com/L/…

  The pledge reads:

  ‘Phil Booth will refuse to register for an ID card and will donate £10 to a legal defence fund but only if 10,000 other people will also make this same pledge.’

  — the PledgeBank.com team

  We got several emails from people saying “your site has got my name wrong — I’m not Phil Booth”. The point is that Phil Booth wrote the pledge, so it’s in his name; the email reflects that. But that’s not obvious to the signer, and since the only name in the body of the email isn’t theirs, they think it’s got it wrong and complain. (To be fair, only three out of ~1,100 did, but that’s still bad.) This is the sort of problem we need user testing to spot: none of us saw anything wrong with the text when we were testing it. So we need to reword that.

• We’ve had several people email to say that they’d like to do versions of PledgeBank in their own countries, and we’d like to hear from anyone interested in localising the mySociety projects who has time, expertise or even just opinions to donate. If that’s you, please get in touch!

And probably some other stuff, but I said this post would be short….

Last night I should have gone to bed early, but these things being how they are I stayed up late having tea with my housemate and his friend. I wanted to get up early, because I knew a few things needed tidying before we started getting media coverage, so I set my alarm. I haven’t done that for work for years! So I’m a bit sleepy.

The most important early thing I did was make the front page featured pledges appear in a random order, for more fairness and serendipity. Late last night Chris had added code in to fuzzily find pledges which somebody has typed in. It uses the database to look for the number of common three letter substrings, so if you type in “http://www.pledgebank.com/suirname” it gives a nice error page leading you to go to “http://www.pledgebank.com/Suriname”. It’s pretty good, and all I had to do was tidy up the text a bit, and add it to the search page as well.

By that time everyone else was up, and the no2id people were publicising their pledge. We were all on IRC, and tailing various logfiles. There were quite a few minor tidy ups for us to make to the launch pledges that were made over the weekend, changing text and signup numbers for the creators a bit.

Someone spotted that the “all pledges” page had the wrong calculated count for one of the pledges. This was very odd, as it was right for all the others. I downloaded a fresh dump of the database to my local machine, where everything was fine. Meanwhile, Chris noticed the PHP server was crashing. After more investigation, we found a subtle bug was creating a corrupt PHP variable. Calling “gettype” on it caused the PHP process to stop with an error, and calling number_format crashed the whole thing. We’re still not sure quite what PHP bug caused this, and need to investigate it more. But we found a simple workaround which stopped it causing any more trouble.

You always find all the bugs when your traffic goes up! That’s why staged beta getting larger and larger, of which today is in many ways the next phase, is the way to go.

Slightly late; I was “hassle”d yesterday, but was at my school’s 21st anniversary, with Terry Waite, some other past students, all the current students, teachers past and present, and a lot of balloons. So I’ve been working some of today instead, which worked out quite well, as it was beautifully sunny yesterday and pouring with rain today.

I fixed a number of bugs in various places, including one that meant all pledges would expire a day early and various problems with the reporting abuse process. I also renamed NotApathetic’s “best of” page to “busiest”, as they’re not the same thing. 😉 The PledgeBank RSS feed of new pledges will hopefully be available from the live site soon (I’ve added the HTML to make the little orange icon appear in Firefox) – if you can’t wait until then.

I think Tom wants me to work next on user-defined flyers, which will involve adding to the poster generation code all the code we removed when we moved from using text in the POST to fetching it from the database. 🙂 Not sure of the details involved, so await direction. Looks like it might involve learning RTF generation in Python, though; hope that’s possible…

So far today fixed a couple of bugs in PledgeBank. One to do with ‘.’ (fullstop) characters at the end of confirm URLs in emails. Some email clients assume that is a sentence end and don’t include them in the URL, so it doesn’t work. In the end I’ve removed all punctuation from the tokens, and added test stuff to make sure nothing comes back.

The other problem was that delivery of success and failure messages was broken. The bin/frequenupdate script that sent these was trying to send them for unconfirmed pledges, and causing errors. This was caused by a chain of bugs. First of all, we weren’t tracking errors from the script, they were just queuing up in a mail spool nobody was looking at. Secondly, frequentupdate needed to have “AND confirmed” added to some of the SQL queries. Thirdly, the test script never made an unconfirmed pledge, so couldn’t detect this. Hopefully all now fixed.

Next up something to do with missing “Date:” headers… The new version of Exim doesn’t by default add them when you submit via SMTP. Now mails without “Date:” appear at the bottom of the inbox in some people’s mail clients so they never see them, and some spam filters block them. We could fix this by reconfiguring Exim, but probably better to add it to EvEl our email comoponent, so we can never lose it again.

This morning I woke up quite late because I didn’t sleep well (and had been building Public Whip’s How To Vote). Today I’m not really working for mySociety, as I’m going to London this afternoon, but Tom had a couple of important bits for me to do. There was a bug in the admin page, and a missing link on the confirmation page. I fixed these up, improved the test suite a bit, and deployed to the main site.

PledgeBank is now ready for people to start using it in earnest. We’re still in testing, as we’re sure there’ll be lots of changes needed to it as it is used in the real world. But all the basic features, and fancier ones such as SMS and the auto-generated flyers, work. Tom’s just been on the radio, and he’s making lots of specific example pledges like this one about Shropshire. So, the hunt is now on.

My next jobs are to tidy up outstanding tickets which we have already fixed, and fix any bugs in there. The next feature we’re adding to the site is comments, so people can discuss the pledge.

The script that confirms new entries to NotApathetic (NA) started to fail around 6.30pm last night, due to some FastCGI/exit gubbins. So I’ve just written a short script that confirms every post that tried to be confirmed since then but failed. Sadly, there are more errors in the scripts, and FastCGI will suspend scripts that fail too often, so I’m currently having to restart Apache every so often until I fix them all. Or what I’ve just done, and wrap the script in an eval to catch errors, it’ll do for now.

Other than that, I will be writing a NA tag generator, and hopefully some PledgeBank stuff, as I thought I’d be working on that today, and have a list of things to do.