1. The story of Pledgebank

    Pledgebank  homepageThese days, when you think of mySociety’s major projects, you’d be forgiven for passing over the vision in purple that is Pledgebank.

    And yet, it’s among mySociety’s longest-running sites, and one that we had big plans for. It was a truly international project, too, with users in many countries.

    It even, as we’ll see, spawned one of the UK’s major transparency organisations.

    But all good things come to an end, and as we announced in a recent post, we’ll shortly be closing Pledgebank down.

    Before we do, it seems a good moment to record some of its history.

    The Pledgebank concept

    In November 2004, we announced mySociety’s second official project:

    The purpose of  PledgeBank is to get people past a barrier which strikes down endless good plans before they can are carried out – the fear of acting alone. It allows anyone to say “I’ll do X if other people also do X”, for example “I’ll write to my councillor if 5 other people on my street do the same”.

    However, there is no scale to big or too small, it could equally be used to say “I’ll start recycling if 10,000 other people in Britain also start”.

    Pledgebank officially launched on 13 June 2005. We’d opened a trial version of the site to a few users first, with early pledges including anti-ID card campaigning, carbon offsetting, and community river cleaning. People were interested. It was off to a good start. As the Guardian reported, even Brian Eno was a user.

    By that September, mySociety Director Tom was describing Pledgebank as our most popular site yet, and as of January 2006, there had been more than 200 successful pledges. In July 2006 the site won the New Statesman New Media award.

    Finding a niche for Pledgebank

    So that was all going swimmingly, and as time passed, we started building on the basic Pledgebank model.

    There were location-specific Pledgebanks, like Pledgebank London which urged folk to do a good deed for their city. Both the then PM Tony Blair and Mayor of London Ken Livingstone helped launch it, pledging to become patrons of a sports club.

    And, like FixMyStreet, we sold Pledgebank as white-label software for councils, allowing them to organise, for example, community snow clearance, and Royal Wedding street parties.

    Did we miss something?

    Here at mySociety, we’re not all about making the big bucks. But that doesn’t stop us from occasionally wondering why we never evolved Pledgebank into a lucrative service like Kickstarter or Groupon, both of which are founded on the very same idea: that there’s potential power in a pledge.

    Whether you back a project on Kickstarter, or put in for a hot stone massage on Groupon, you’re basically undertaking to buy something. But while Pledgebank did allow fundraising pledges, it didn’t take a cut of the moneys raised.

    At one point we did look into using an escrow service, but we decided in the end that each pledge organiser could sort out collection of any payments. And thus, we never quite became Kickstarter. Oh well.

    Simple concepts have many possibilities

    Pledgebank might have been founded on a simple concept, but, like so many simple concepts, it turned out that there were endless features we could add to it.

    At launch, SMS text messages were an important part of the site, and one that we spent considerable time and effort on. It was 2005, remember, and as we often said in our blog posts at the time, many people either weren’t online or had no desire to be. We wanted the site to cater for them too.

    And almost immediately after launch we added another feature: the ability to subscribe, so you’d receive an email when someone set up a pledge that was near you, geographically. This was ideal for those pledges with a local aspect, such as saving an ancient tree, or getting together to clean up a community.

    Then there was the international aspect. Pledgebank was mySociety’s first in-house project to be translated.

    In true mySociety style, the translation was crowdsourced and ultimately overseen by our diligent volunteer Tim Morley. As I write, just prior to the site’s closure, it is available in 14 languages, from Simplified Chinese to Belarusian, and including Esperanto.

    And it was taken up, enthusiastically, in many countries. Even now, we still sometimes have to deploy Google Translate in order to reply to Pledgebank’s user support emails.

    A site to change the world

    Over its lifetime, Pledgebank has been the starting point for many people to make the world a better place, in ways both large and small.

    Before we say goodbye all together, let’s take a look at some of the surprising, sometimes amazing, things it helped bring about.

    The smaller pledges were sometimes just as interesting:

    …and many more. Over time, Pledgebank became an archive of inspirational, utopian, and sometimes plain eccentric pledges. It brought thousands of people together in common causes, and multiplied the power of a single person’s desire to do good.

    We’d love to hear how you used Pledgebank: let us know in the comments below.

     

  2. Testing SMS code

    Today I’m expanding the test harness to thoroughly test out all the SMS code. Adding in SMS creates all sorts of edge cases to check out, and indeed quite a few common cases. For example, it needs to make a pledge with both SMS and email signers, and test SMS conversion to email. And do both failure and success for all those cases. It gets even more complicated when you allow for SMS delivery reports.

    To do this, I’ve got small bits of script which pretend to be C360, our SMS aggregator. This is much cheaper than using real SMS messages every time I run the test script 🙂 The pretend scripts talk C360’s API, and fake telling our scripts as if incoming SMS messages arrive. They also store up outgoing SMS from our scripts, and check all the correct messages have been sent out to users.

    At the moment Chris is making some crazy Perl stuff to override the time() function in the SMS daemon, so when the test harness fakes the advance of time, it notices.

    If I get bored of all this, I’ll tidy up and test the new comments code Chris started the other day.

  3. About to have breakfast

    This morning I woke up quite late because I didn’t sleep well (and had been building Public Whip’s How To Vote). Today I’m not really working for mySociety, as I’m going to London this afternoon, but Tom had a couple of important bits for me to do. There was a bug in the admin page, and a missing link on the confirmation page. I fixed these up, improved the test suite a bit, and deployed to the main site.

    PledgeBank is now ready for people to start using it in earnest. We’re still in testing, as we’re sure there’ll be lots of changes needed to it as it is used in the real world. But all the basic features, and fancier ones such as SMS and the auto-generated flyers, work. Tom’s just been on the radio, and he’s making lots of specific example pledges like this one about Shropshire. So, the hunt is now on.

    My next jobs are to tidy up outstanding tickets which we have already fixed, and fix any bugs in there. The next feature we’re adding to the site is comments, so people can discuss the pledge.

  4. New PledgeBank and minor irritants

    So, another day, another new version of the PledgeBank source code live on http://www.pledgebank.org/. Actually, the changes are mostly underneath the surface, so you shouldn’t notice any specific differences, unless we’ve broken something, in which case whinge to team@pledgebank.com, as usual. That said, the new posters and SMS signup to pledges are now live, so you should now go out into the world and do Good Virtuous Stuff with them.

    This is supposed to be the developers’ ‘blog, so a couple of technical things which have annoyed me today. (presumably you all read my actual web log and therefore expect me to write about things that annoy me):

    • You can’t use a variable quantity in a limit clause in a subselect in PostgreSQL. “A what? In a what? What?“, I hear you cry. Well, this did come up in real life. When I was upgrading the PledgeBank code, there were various changes to the database schema which had to be made first. One of them was to change the way that the success of a pledge (i.e., what happens when it reaches its target) is recorded. Previously we had two boolean columns, like this:
      create table pledges (  -- obviously SQL tables should have singular names, but
                              -- in this case nobody asked me...
          -- ...
          success boolean
              not null
              default false,  -- indicates that the pledge has succeeded
          completionnotified boolean
              not null
              default false,  -- indicates that the creator and signers have been
                              -- told that the pledge succeeded
          -- ...
      );

      Now, this is messy and not enough to describe how the site actually works. Specifically, there are some types of messages which should be sent to creators and signers, some which should be sent only to signers who signed before the pledge succeeded (in between success and the deadline, you can still sign the pledge), some which should only be sent to non-SMS recipients, etc. etc. So instead we now have a table of messages with flags indicating where they should go to and so forth. A side-effect of this is that the above structure is replaced with this:

      create table pledges (
          -- ...
          whensucceeded timestamp,    -- indicates when pledge succeeded
          -- ...
      );

      Now, there are Real Pledges on the live site, so unlike the development site we can’t just drop the database in an update; instead, we have to port all the data over to the new data model. So what you’d like to write is, obviously,

      begin work;
      alter table pledges add column whensucceeded timestamp;
      update pledges
      set whensucceeded = (
          select signtime
          from signers
          where pledge_id = pledges.id
          order by signtime
          limit 1 offset pledges.target
      )
      where success;
      alter table pledges drop column success;
      commit work;
          -- I love using a proper database!

      Sadly, you can’t. The two arguments in the limit statement in the subselect have to be constant. (No, I don’t know if/where this is documented. I, uh, asked on IRC.) This sucks. In the end I just set whensucceeded to the current time for currently-successful pledges; it’s not right, but it’ll do.

    • Python (2.3, on FreeBSD) either sets O_NONBLOCK on sockets by default, or fails to clear it when creating a socket. Result: program crashes with EAGAIN down in the FastCGI library every so often. Outstanding!

    Poll: Should we turn on comments on this ‘blog? (Does anyone read it, anyway?) Mail me at chris@mysociety.org with your answers….

  5. Progress: more SMS and email

    So, just a very quick update. Actually Tom was supposed to write this, but he’s off schmoozing at some event for people much cooler than us programmers, so I’ve volunteered. Since I last wrote about SMS, we’ve arranged to do our SMS stuff in partnership with C360, who’ve very kindly offered us a much better deal than the other aggregators we’d talked to. Unfortunately that required a rewrite of the SMS interface, which took up a bit of time. But that’s now done and seems to be working. I’d ask you to test it, but (a) we already think it’s working OK, and (b) because of The Rules on the promotion of “premium rate” (i.e., reverse-billed) SMS services, we can’t advertise the service publically without also advertising a point of contact for users. Because that has to go on the flyers which can be printed out to advertise pledges, we need to arrange a short-and-snappy postal address (PO Box or similar) which won’t take up too much space. Next week, hopefully.

    So, that was yesterday and this morning’s work. Today it’s back to the mailing lists which pledge creators will use to send information about pledges out to the signers. But before that, I need to make some more coffee. Toodle-pip.

  6. Pledgebank and email

    Right now I’m working on the email code for Pledgebank.

    There are (ignoring SMS) three communications problems in Pledgebank: firstly, how does the pledge setter get to communicate with the pledge signers when the pledge is complete (so that the setter can tell the signers what to do / how to organise themselves, or whatever); secondly, how do the signers communicate amongst themselves before the pledge completes (to discuss tactics for getting more people signed up, or whatever); and thirdly, how can other people get more information or clarification on the pledge.

    Ideally, we’d like to be able to do all of the above while keeping individual signers’ email addresses private.

    (As an aside, I should say that the proper English words for “pledge creator” and “pledge signer” are “pledgee” and “pledger”. I fought a valiant struggle to have these adopted as the Official Pledgebank Terminology, but inevitably we all got confused as to which was which, and reluctantly abandoned them in favor of “creator” and “signer”, which sound a bit buzzword to me. Ho-hum.)

    Anyway, there are lots of different ways that we could allow the creators and signers to communicate. After a long discussion we decided that we’re going to implement,

    • an announcements-type mailing list for the pledge creator to use,
    • an (optional to join) discussion list for signers and the pledge creator, and
    • a web-log-style comments page for each pledge.

    which seemed like a sufficient set of features which would avoid having too many confusing configuration options.

    (Another way to look at this is to look at this from a permissions perspective:

    Who can send messages? Who can read messages?
    Creator Signers Anyone Signers Anyone
    announcements about the pledge Y N N Y N
    discussion about the pledge Y Y N Y N
    questions about/clarifications of the pledge Y Y Y Y Y

    For the first two functions we need to limit access to what’s said to particular groups of people. That means that either we have to have a login system on Pledgebank, which we’re avoiding, to keep things as simple as possible; or to use email, which is practical because creators and signers have to confirm their email addresses in order to pledge. No access restrictions are needed for comments and clarifications, and it’s useful to show comments publically — so as to avoid ever answering the same question twice — so public web comments are appropriate here.)

    So, this week’s task is to get the email stuff all working. Getting email sending from web applications right can be tricky, especially if you care about reliable delivery. As you can imagine, this is a whole heap of fun….

  7. Tom’s Stuff

    Ooh, where to start. Looking into the regulations concerning what information you have to put on a leaflet when it involves SMSes that cost money (i.e how much small print do you need). Going through old emails to see if there are any vital opportunitities for mySociety which I’ve failed to seize lurking in my inbox. And, erm, standing fully clothed in the bath trying to explain to a plumber why water keeps leaking into the kitchen.

  8. A short message

    So, my script actually decided to hassle Matthew into writing a post here, but he’s off enjoying himself at EtCon so I’m stepping into the breach myself.

    One of the features PledgeBank will support is allowing users to sign up to a pledge by SMS (as well as through the traditional web form plus email confirmation route). So, that’s what I’ve been working on lately.

    The idea is that the user sees a poster or flyer describing the pledge, with the instruction, “text ‘PLEDGE {name-of-pledge}’ to {number} to sign up!”. We then sign them up and send them another SMS when the pledge completes.

    Now, the last time I had anything to do with SMS, making any progress required writing a giant C program which talked to an “SMS message centre” via the universal computer protocol (so called because it’s not universal and isn’t used for talking to computers — in fact, I think it’s a hangover from the days of numeric-only pagers). Since then, this stuff has become a lot easier; there are lots of companies which do the talking-to-the-phone-company stuff for you, and let you transmit and receive SMSs over HTTP. The one we’re using for testing, MX Telecom, seem pretty typical; you submit messages to send via HTTP POST, and get told about incoming messages and delivery reports via — slightly disappointingly — a GET to your own servers. (Further disappointment: while sending is authenticated — via a plaintext username and password — receiving isn’t, other than by source IP address. Ho-hum.)

    For those who don’t use them, SMS messages can be regarded as being like email, but much more expensive, slower, and not as reliable. Oh, and you only get to send 160 characters in each message, unless you want to use characters outside the ISO-8859-15 code page (“Latin1 with added ‘?'”), in which case you get only 70 characters. The one feature SMS has which email doesn’t — other than cost — is delivery reports. Decent aggregators will send you a message which tells you whether a message you’ve sent has been delivered, buffered in the network waiting for the recipient’s phone to be switched on, rejected because the user pays to receive SMSs and has run out of credit, or various other rare conditions.

    It’s also possible to send “premium rate” reverse-billed SMS messages, where the punter pays to receive them. Sadly you don’t get the option of sending these from your regular phone, but if you pay BIG CASH MONEY to an SMS aggregator you can do so. These messages typically cost the recipient 25p or 50p (there are a whole set of billing bands). For the cheaper messages, the telephone company keeps most of the money, the aggregator takes a bit, and eventually you get 4–6p.

    Sending a normal (“bulk”) SMS message costs somewhere from 5p to 10p, in small quantities.

    So so far our plan looks like this:

    • User texts us, asking to be signed up
    • We send them a reverse billed message containing a URL which they can use to convert their SMS subscription into an email one
    • If we get a successful delivery report for that message, or if they visit the URL we sent them, we sign them up
    • Later on, when the pledge completes, and if we don’t have their email address, we let the pledge creator send them another SMS telling them what to do

    One obvious problem here is that, on the face of it, we lose money on every transaction here (and only letting the pledge creator send one message to the signers is pretty sucky too). Based on the estimates we’ve been given, we can break even as long as ~20% of SMS signers convert to email subscriptions. But it would be nice to be able to run these things by SMS alone, since not everybody has the web or email (or wants to use them).

    Anyway, from a code point-of-view, this all now works — I could bore you for hours about all the nasty cases which occur when somebody signs up by email, and by SMS, and then tries to convert their SMS subscription to email and so forth, but anyway, we’ve solved that one and this post is already too long — so all we have to do now is get the best deal we can for SMS messages so that this becomes practically useful….