1. How to publish local FixMyStreet reports onto your Facebook page

    If you run a Facebook page or group for a local community, you might like to add a FixMyStreet feed. This will publish recent reports, made within the geographic area that you define, as posts on your Facebook page, like this:

    FixMyStreet on a Facebook page

    Adding a FixMyStreet feed to your Facebook page is not difficult, but you do need to be an administrator for the page you want it on.

    Also, it is a multi-step procedure. In other words, you might like to fetch yourself a cup of tea before following along with the instructions below.

    Here’s what to do:

    1. Visit FixMyStreet.com and locate the area you want a feed for

    Does your Facebook page deal with a particular city or town, or an area within that town?

    You’ll probably want to publish the FixMyStreet reports that are made within that area. The wider the area you choose, the more reports you will be publishing, so think carefully about what your followers will actually want to see on your Facebook stream.

    Once you’ve decided, locate that area on FixMyStreet by putting a postcode or place name into the box on the homepage. It doesn’t need to be precise; you just need to locate any spot within the area that you want to cover.

    find area on FixMyStreet

    You’ll be taken to a page showing all recent reports for the surrounding area.

    Local area on FixMyStreet

    Don’t worry if this isn’t the exact area that you want your feed to cover, so long as you’re at a point within that area – we’re going to refine that in the next step.

    You can click and drag the map or zoom in and out if you’re not quite within the area that you want to be.

    2. Create your feed

    At the very foot of the FixMyStreet page, below the list of reports on the left, you’ll see a little icon marked “get updates”. Click on this.

    get updates from FixMyStreet

    You now have the choice of several options. You can get a feed for:

    – All problems reported within 2km, 5km, 10km or 20km, or within a population of roughly 200,000 people

    – All problems reported to your local council

    – All problems reported within the ward of your council

    Choose which option most closely matches the area that your Facebook page deals with, and click the green button marked ‘Give me an RSS feed’.

    Pick your FixMyStreet feed

    3. Grab the URL

    Your feed page will look something like this:

    FixMyStreet RSS feed

    It’s basically just the data from FixMyStreet, with none of the site’s styling or functionality around it. This is what we need Facebook to grab and publish on your page.

    You will need the URL (web address) of this page in a short while – just keep it open while you go through the next steps.

    4. Log into Facebook

    You don’t need to be logged in as the page that you want the feed on; it’s fine just to be logged in as yourself, the administrator of that page.

    5. Connect an RSS action to your Facebook page In order to publish FixMyStreet reports, you’ll be using what’s called an RSS feed – a stream of data that can be picked up and published anywhere else. In this case, the data stream is found on what you saw in step 3: the ‘bare bones’ FixMyStreet page; and the target for publication is your Facebook page.

    Facebook itself does not provide a way to publish RSS feeds, so we’re going to use a service called IFTTT.

    IFTTT stands for ‘If This, Then That”, and it’s a really nifty, free service that basically allows you to say: “Every time [something] happens, do [something else].

    We are going to use it to say “Every time a new post appears in the RSS feed that I specify, publish it to my Facebook page”.

    Here’s what to do.

    a) Sign up for an IFTTT account, if you don’t already have one.

    IFTTT homepage

    b) Click on ‘my recipes’ and then ‘create a recipe’:

    create an IFTTT recipe

    c) Click the word ‘this’:

    ifthis

    d) Search for the word ‘feed’ and then select the orange RSS symbol:

    feed

    e) Click ‘new feed item’:

    new feeditem

    f) Input the URL of your FixMyStreet feed (the one we kept open earlier, in step 3) and click ‘create trigger’:

    input url

    g) We’ve set up the first half of our ‘recipe’—the ‘IF THIS’. You can see it as the orange RSS feed sign in the sentence now.
    So next we’re going to tell the recipe what to do when that feed updates.
    Click the word ‘that’:

    that

    h) Search for Facebook and select ‘Facebook pages’:

    facebook

    IFTTT will take you through the steps of linking with Facebook and choosing which page to publish to. Just make sure you say ‘yes’ to everything.

    i) Choose how you would like updates to display – I think a link post looks most suitable

    This format allows you to add a message to every item it publishes: probably a good idea, because it helps give context to these posts that are going to appear in your Facebook stream.

    FixMyStreet reports are often written in the first person, so if they appear without a title or explanation, they may look as if they are posts from you yourself – take a look at the example at the top of this post and you’ll see what I mean.

    format

    j) Input some text if required, eg “Here’s a new report from x area”:

    fill in

    k) Click ‘create action’ and you’re done. Note that your feed will not start publishing out until the next report is made on FixMyStreet.

    Do let us know in the comments below if you go ahead and install this functionality – plus any tips you might have.

    By the way, you can use this method to publish any RSS feed to your Facebook page, so you could also publish anything from blog posts to YouTube videos, so long as you can find the RSS source, which is usually signaled by that little orange icon: rss

    Footnote

    A user, Alan, has kindly been in touch with this message:

    I connected to Facebook Pages, and assigned a Facebook Page I manage (a trial site). Then, later, I couldn’t figure out how to change to another Facebook Page I manage. After much delving, I found that people can change the Facebook Page to where the feed should go on this page.
    Also of note: I have also had a dabble with zapier.com which seems pretty good at doing the same thing. It offers a range of subscription plans ranging from free.

     

  2. Open Data Day resources

    Image by OpenSourceWay

    Note (June 2016): This post is now slightly out of date. FixMyTransport is no longer running, though all of the other APIs and tools listed are still available.

    There is also one significant addition which developers should find useful: EveryPolitician, which provides data on all current politicians around the world (and, in the future, we hope, all past ones too). See more here.


     

    Much of what we do here at mySociety relies on Open Data, so naturally we support Open Data Day. In case you haven’t come across this event before, here’s the low-down:

    Open Data Day is a gathering of citizens in cities around the world to write applications, liberate data, create visualizations and publish analyses using open public data to show support for and encourage the adoption open data policies by the world’s local, regional and national governments.

    If you’re planning on being a part of Open Data Day, you may find some of mySociety’s feeds, tools and APIs useful. This post attempts to put them all in one place. (more…)

  3. Upcoming business on TheyWorkForYou

    TheyWorkForYou has, until now, only covered things that have already happened, be that Commons main chamber debates since 1935, Public Bill committees back to 2000, or all debates in the modern Northern Ireland Assembly.

    From today, we are taking the UK Parliament’s upcoming business calendar and feeding it into our database and search engine, which means some notable new features. Firstly, and most simply, you can browse what’s on today (or the next day Parliament is sitting), or 16th May. Secondly, you can easily search this data, to e.g. see if there will be something happening regarding Twickenham. And best of all, if you’re signed up for an email alert – see below for instructions – you’ll get an email about any matching future business along with the matching new Hansard data we already send. We currently send about 25,000 alerts a day, with over 65,000 email addresses signed up to over 111,000 alerts.

    Mark originally wrote some code to scrape Parliament’s business papers, but this sadly proved too fragile, so we settled on Parliament’s calendar which covers most of the same information and more importantly has (mostly) machine-readable data. Duncan and I worked on this intermittently amidst our other activities, with Duncan concentrating on the importer and updating our search indexer (thanks as ever to Xapian) whilst I got on with adding and integrating the new data into the site.

    I’ve also taken the opportunity to rejig the home page (and fix the long-standing bug with popular searches that meant it was nearly always Linda Gilroy MP!) to remove the confusingly dense amount of recent links, bring it more in line with the recently refreshed Scottish Parliament and Northern Ireland Assembly home pages, and provide more information to users who might not have any idea what the site covers.

    Signing up for an email alert: If you want to receive an email alert on a particular person (MP, Lord, MLA or MSP), visit their page on TheyWorkForYou and follow the “Email me updates” link. If you would like alerts for a particular word or phrase, or anything else, simply do a search for what you’re after, then follow the email alert or RSS links to the right of the results page.

  4. Embedding FixMyStreet Google map in a blog

    On Twitter about 15 minutes ago, @greenerleith asked: “Has anyone worked out how to display the most recent #fixmystreet reports on a local map widget that can be embedded? #hyperlocal”

    Like this? 🙂

    It’s very simple to do:

    1. Go to FixMyStreet, and locate any RSS feed of the latest reports you want (for the above map, I used Edinburgh Waverley’s postcode of EH1 1BB; you could have used reports to a particular council, or ward, using the Local alerts section). Copy the URL of the RSS feed.
    2. Go to Google Maps, paste the RSS feed URL into its search box, and click Search Maps.
    3. Click the “Link” link to the top right of the map, and copy the “Paste HTML to embed in website” code.
    4. Paste that code into your blog post, sidebar, or wherever (you can alter the code to change its size etc.).
    5. Done. 🙂

    The latest reports from FixMyStreet, superimposed on a Google Map, embedded in your blog. Hope that’s helpful.

  5. FixMyStreet RSS

    FixMyStreet has a lot of RSS feeds. There’s one for every one-tier council (170), one for every ward of every one-tier council (another 5,044), two for every two-tier (county and district) council (544), and two for every ward of every two-tier council (20,296) – two per two-tier council because you might want either problems reported to one council of a two-tier set-up in particular, or all reports within the council’s boundary.

    Then there’s an RSS feed every 162m across Great Britain in a big grid, returning all reports within a radius of that point, the radius by default being automatically determined by that point’s population density, but customisable to any distance if preferred. That’s, at a very rough approximation assuming Great Britain is a rectangle around its extremities, which it’s not, 19 million RSS feeds, lots of which will admittedly be very similar. 🙂

    Every single one of those feeds can be subscribed to by email instead if that’s preferable to you, and are all accessible through a simple interface at http://www.fixmystreet.com/alert.

    However, none of these RSS feeds was suitable for the person who emailed from a Neighbourhood Watch site and said that all they had was a postcode and they wanted to display a feed of reports from FixMyStreet. Given you could obviously look up a FixMyStreet map by postcode, it did seem odd that I hadn’t used the same code for the RSS feeds. Shortly thereafter, this anomaly was fixed, and if you now go to a URL of the form http://www.fixmystreet.com/rss/pc/postcode you will be redirected to the appropriate local reports feed for that postcode (I could say that adds another 1.7 million RSS feeds to our lot, but given they’re only redirects, that’s not strictly true). And after a couple more emails, I also added pubDate fields to the feeds which should make displaying in date order easier.

    It’s great to see our RSS feeds being used by other sites – other examples I’ve recently come across include Brent Council integrating FixMyStreet into their mapping portal (select Streets, then FixMyStreet), or the Albert Square and St Stephen’s Association listing the most recent Stockwell problems in their blog sidebar. If you’ve seen any notable examples, do leave them in the comments.

  6. acts_as_xapian

    One of the special pieces of magic in TheyWorkForYou is its email alerts, sending you mail whenever an MP says a word you care about in Parliament. Lots of sites these days have RSS, and lots have search, but surprisingly few offer search based email alerts. My Mum trades shares on the Internet, setting it to automatically buy and sell at threshold values. But she doesn’t have an RSS reader. So, it’s important to have email alerts.

    So naturally, when we made WhatDoTheyKnow, search and search based email alerts were pretty high up the list, to help people find new, interesting Freedom of Information requests. To implement this, I started out using acts_as_solr, which is a Ruby on Rails plugin for Solr, which is a REST based layer on top of the search engine Lucene.

    I found acts_as_solr all just that bit too complicated. Particularly, when a feature (such as spelling correction) was missing, there were too many layers and too much XML for me to work out how to fix it. And I had lots of nasty code to make indexing offline – something I needed, as I want to safely store emails when they arrive, but then do the risky indexing of PDFs and Word documents later.

    The last straw was when I found that acts_as_solr didn’t have collapsing (analogous to GROUP BY in SQL). So I decided to bite the bullet and implement my own acts_as_xapian. Luckily there were already Xapian Ruby bindings, and also the fabulous Xapian email list to help me out, and it only took a day or two to write it and deploy it on the live site.

    If you’re using Rails and need full text search, I recommend you have a look at acts_as_xapian. It’s easy to use, and has a diverse set of features. You can watch a video of me talking about WhatDoTheyKnow and acts_as_xapian at the London Ruby User Group, last Monday.

  7. Bees

    We’re busy as bees, lots of things happening, increasingly many of which are commercial, and we can’t talk about until they’re released.

    Commercial? But you’re a charity! Yes – but just as Oxfam have a trading subsidiary company which runs the second hand clothes shops, we have a trading subsidiary company that sells services relating to the websites that we make (structural details here).

    Everything from other small charities to large media companies are buying our services – which range from customised versions of FixMyStreet, through to strategic consulatancy. If you’ve got something that you think we might be able to help with, email Hello@mysociety.org – easier to talk to than us geeks.

    Meanwhile we’re cracking on with our free services for the public, which are increasingly funded by this commercial work.

    TheyWorkForYou recently launched a Scottish version, thanks to volunteer Mark Longair, and Matthew. More goodies in store as the Free Our Bills campaign unfolds. We’ve started a sprint to get a photo for every MP’s page. If you work for or are an MP or have copyright of a photo of one that we’re missing, then email it to us.

    WhatDoTheyKnow is getting lots of polishing – the new site design that Tommy has been working on is nearly ready. Today I just turned on lots of new email alerts and RSS feeds, so you can get emailed, for example, when a new request is filed to a particular public body, or when a request is successful.

    Our super ace volunteers have been busy adding public authorties to the site, and we now have 1153 in total. We’re getting a steady trickle of good requests (pretty graph) coming in. Blogs such as Blind man’s buff and confirm or deny are sorting the wheat from the chaff. Do blog about and link to any interesting requests that you see!

    Other things in the works are a much needed revamp of www.mysociety.org, some interesting things on GroupsNearYou, and no doubt squillions of other things. I’ll let Matthew post up anything I’ve missed 🙂

  8. Lessons from mySociety conversion tracking

    Matthew and I have been sitting next to each other today looking at the outputs of his lovely new custom built conversion tracking system, designed to ensure that the optimal number of users who just come to one of our services as a one off get signed up to something else longer lasting.

    I’ve been banging on for ages about how government should seize on cross selling people who’ve just finished using one online service into using another of a more democratic nature, so it seems worth spelling out some of the lessons.

    First, there’s some interesting data from the last few weeks, since our newest conversion tracking infrastructure has been running in its nice new format.

    One of the adverts randomly served to users of WriteToThem (after they’ve finished sending their letter) encourages them to sign up to TheyWorkForYou email alerts – the service people use to get emailed whenever their MP speaks in Parliament. The advert features a slogan of encouragement, and a pre-populated email form containing the user’s email, and a ‘Subscribe me’ button. This advert was shown to 2328 users last month, of whom 676 became TheyWorkForYou email subscribers, which is a pretty cool 29.04% conversion rate. However, we also showed another advert for the same service, to the same WriteToThem users, which also had the same button and text, but which hid the form (and their address). That was shown to 2216 users of whom 390 signed up, a more modest 17.6%. So the impact of simply showing an email box with the users email address in it, versus hiding it, was worth 10% more users. Why? Go figure!

    So now we’ve canned the advert that hides the address form, and instead we’re comparing two different adverts both of which feature the pre-populated signup form, but which use different words. It’s probably too early to judge, but the new ad appears to have a very similar conversion rate suggesting it might be hard to squeeze many more subscribers out of this page. We’ll keep trying though!

    Another thing we learned of interest was that monthly subscribers to email alerts on TheyWorkForYou were down year on year in the month before we added this new advertising & conversion tracking system, even though the total number of visitors were clearly up on the same month last year. This appears to suggest that two things are happening. First RSS is catching on, so some users who would previously have got email alerts are subscribing to RSS feeds instead. Second, it suggests that the TheyWorkForYou user audience might have been getting more saturated with regulars – proportionally fewer new users coming (although more visitors in absolute terms) so fewer people signing up to get alerts. The cross marketing and conversion tracking seems to have reversed that trend, which is awesome.

    We also advertise several different services to people who just finish signing up to get email alerts on TheyWorkForYou itself. We’ve just noticed that a full 25% of people shown the advert to sign up for HearFromYourMP proceed to sign up. We’ve therefore just decided to dump other adverts shown on TheyWorkForYou (such as advertisements for other sorts of TheyWorkFor you email alert) and concentrate on just cross selling HearFromYourMP. A back of the envelope calculation suggests that by just advertising this one site from the completion page we should get an extra 10,000 subscribers to HearFromYourMP this year on top of the organic growth. Not bad for a few minutes analysis, and a number likely to make a fair few more MPs post messages to their patiently waiting constituents.

    One last interesting thing (at least to me) is how some more demanding services are a much harder sell than others to users. So asking people to make new groups on GroupsNearYou.com or report a problem in a street on FixMyStreet tend to result in more traditional online marketing scale conversion rates of 0.1% to 2%. Still worth doing, and so we compare different versions of those ads too, to try and eke up those rates for these sites that arguably have more tangible, direct impacts on people and communities.

    It will be a challenge for mySociety’s future to work out how to trade off impact against scale of service use – are 10 HearFromYourMP subscribers worth one pothole that doesn’t get fixed? Answers on a postcard…

  9. Assorted news updates

    Just a quick post to keep those of of you interested in mySociety in the loop with our activities at the moment.

    New Things You Can Use Now

    1. Email or RSS alerts when people report problems in your ward or your council via FixMyStreet. Ideal for councillors, people on resident’s associations, or anyone just concerned about what’s breaking and being fixed in the area right near their home.

    Have a go – it’s ace when the mail comes dropping in from just down your road.

    2. The Queen on TheyWorkForYou

    Is this the first monarch with her own RSS feed? Would anyone really care if she was?

    New Projects Coming Up

    We have three major projects under way at the moment, and unusually
    only two of them involve us building websites.

    1. The Freedom of Information Filer and Archive website is under construction. Aiming to make it easier to make freedom of information requests, and easier for people to find what other people have found out, this is being build mainly by Francis. We’re having lots of discussions about design and features right now, and if you have anything to contribute please either get in touch or leave your ideas on the wiki page.

    2. Local Email Groups Near You – an attempt to record the location of hyper local email groups and local forums and websites and to share that information on lots of other sites. Why go blindly hunting for advice on a plumber if there’s already an email list that covers your street? This is going to be a rare international project for us, so if you’re outside the UK and interested in community Internet usage, please get in touch.

    3. The 90 Day Project – mySociety’s first lobbying exercise, trying to encourage parliament to take some steps to improve the way it publishes information, and to improve the tools that MPs have to handle mail from their constituents.

    There’s lots and lots more too, but we can’t blow all our surprises in one go, can we?

  10. “Now, here, you see, it takes all the running you can do, to keep in the same place.”

    Whereas new sites are lovely, and I talk about Neighbourhood Fix-It improvements further down, there’s still quite a bit of work that needs to go into making sure our current sites are always up-to-date, working, and full of the joys of spring. Here’s a bit of what I’ve been up to recently, whilst everyone else chats about database upgrades, server memory, and statistics.

    The elections last week meant much of WriteToThem has had to be switched off until we can add the new election results – that means the following aren’t currently contactable: the Scottish Parliament; the Welsh Assembly; every English metropolitan borough, unitary authority, and district council (bar seven); and every Scottish council. The fact that the electoral geography has changed a lot in Wales means there will almost certainly be complicated shenanigans for us in the near future so that our postcode lookup continues to return the correct results as much as possible.

    Talking of postcode lookups, I also noticed yesterday that some Northern Ireland postcodes were returning incorrect results, which was caused by some out of date entries left lying around in our MaPit postcode-to-area database. Soon purged, but that led me to spot that Gerry Adams had been deleted from our database! Odd, I thought, and tracked it down to the fact our internal CSV file of MLAs had lost its header line, and so poor Mr Adams was heroically taking its place. He should be back now.

    A Catalan news article about PledgeBank brought a couple of requests for new countries to be added to our list on PledgeBank. We’re sticking to the ISO 3166-1 list of country codes, but the requests led us to spot that Jersey, Guernsey and the Isle of Man had been given full entry status in that list and so needed added to our own. I’m hoping the interest will lead to a Catalan translation of the site; we should hopefully also have Chinese and Belarussian soon, which will be great.

    Neighbourhood Fix-It update

    New features are still being added to Neighbourhood Fix-It.

    Questionnaires are now being sent out to people who create problems four weeks after their problem is sent to the council, asking them to check the status of their problem and thereby keep the site up-to-date. Adding the questionnaire functionality threw up a number of bugs elsewhere – the worst of which was that we would be sending email alerts to people whether their alert had been confirmed or not. Thankfully, there hadn’t yet been any such alert, phew.

    Lastly, the Fix-It RSS feeds now have GeoRSS too, which means you can easily plot them on a Google map.