Mobile operators altering (and breaking) web content

We had a complaint that FixMyStreet maps weren’t displaying on someone’s computer. We hadn’t had any other complaints, and we quickly narrowed it down to the fact that the person was on the internet using a tethered T-Mobile phone.

T-Mobile (and Orange, and quite possibly others) are injecting JavaScript and altering content served over their networks. Their reason for doing this, according to their websites (T-Mobile, Orange), is to compress images and video sent to your browser, so as to speed up your browsing. Seeing it in action, they also inline some CSS and JavaScript, though not all, and remove comments from external files.

However, their implementation breaks things. In this particular instance, the T-Mobile JavaScript comment stripper appears to be searching for “/*” and “*/” and removing everything inbetween. This might work in most cases; however in the jQuery library, we find a string containing “*/*”, and later down the file, another string containing “*/*”. T-Mobile remove everything between the things it thinks are comment markers, even though they’re actually contained within strings, causing the jQuery library to be invalid JavaScript and stopping anything using jQuery from running.

Their decision to inline lots of the CSS also seems a bit odd – sure, on a mobile this might be quicker, but even ignoring tethering nowadays plenty of mobiles have caches too and having the CSS download once and be cached would seem better than adding weight to every page download. But I’m sure they’ve studied their decision there, and it doesn’t make any difference to the actual browsing, as opposed to the comment removal.

To turn off this feature on your mobile phone or broadband, visit accelerator.t-mobile.co.uk or accelerator.orange.co.uk on your connection and pick the relevant option – if anyone knows of similar on other networks, do leave updates in the comments.

From a FixMyStreet point of view – whilst FixMyStreet functions just fine without JavaScript, I had made the (perhaps incorrect) decision to put the map inside a <noscript> element, to prevent a flash of map-oddity as the JavaScript map overlaid the non-JS one. However, this meant in this circumstance the map did not work, as JavaScript was enabled, but jQuery was unable to be loaded. I haven’t decided whether to change this behaviour yet; obviously it would help people in this situation as the map would still display and function as it does for all those without JavaScript, but for those with JavaScript it does look a bit jarring as the page loads. Any suggestions on a better approach welcome :)

14 Comments

  1. Inlining the CSS will presumably significantly lower the latency of the request, which mobile networks have an incentive to do, at the expense of increasing the download size, which they also have a small incentive to do.

  2. Another example of the way mobile phone operators really don’t understand their role. They provide pipes, nothing more. As soon as they start to ‘add value’, as they see it, they screw up. I have had sites blocked by their proxy because, in their opinion, it was inappropriate, it turns out there was some swear words, how long before they block twitter on those grounds?

    If you’re listening T-Mobile et al: just be a pipe, keep the costs down and pass on the savings. Then we’ll be happy and you will make money.

  3. As an ex-MNO employee – involved in the transcoding brouhaha a few years ago – I’d like to weigh in.

    From a “how do I stop this happening” POV, there are three options.
    a) SSL
    b) Ask T-Mobile and all the other providers to “whitelist” you
    c) Set various headers so the proxies don’t mess with your content (no-transform http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.5 and hope they honour them)

    Now, I don’t want to go too deeply into whether a service provider who charges per MB has any business in compressing content before sending it to a customer – but as people start expecting faster delivery of content through crowded spectrum, this is something we’ll see more of.

    The majority of customers (according to what I saw) like transcoding. It speeds things up and it costs them less. There may be the odd glitch – but it’s the exception and a price worth paying.

    It is always worth coding defensively – in this case (if I’ve understood correctly) seeing what happens if a user is browsing without JS.

  4. Richard: Good idea, I might try that (and suggest it to jQuery if it works!).

    Terence: I’m not disagreeing with anything you say (although as I say they are actually *increasing* the size of the downloaded HTML, not decreasing), and I personally don’t have a problem with transcoding as long as it’s optional and, most importantly, as long as it’s coded correctly. In this case, it’s clearly not, as it’s removing actual JavaScript code that it thinks is a comment. If MNOs had public bug reporting sites, I’d happily report it to them…

    The site works fine without JavaScript (and has done since 2007); the issue is someone browsing with JavaScript, but part of that JavaScript is then broken by the mobile operator in transit.

  5. Isn’t this an unauthorised interception? Can they legally read and change data requested by a user from a third party?

  6. We used to go with having the content in a noscript and then also document.write-ing it if they had javascript but it wasn’t up to the job. Replicating the content is annoying but if it’s just a single img tag or something simple like that it shouldn’t be a huge deal.

  7. I’ve just wasted two days chasing this down, made particularly more difficult because the broken jQuery stays in the browser cache with an aggressive cacheing browser like Chrome – and the site consequently remains busted even when the user is no longer on T-mobile’s 3G network.

  8. I’m with Pete B. This is blatantly illegal wire-tapping just the same as what Phorm attempted. Imagine if they did the equivalent with voice calls – replacing the words someone says to you with a brief summary, indistinguishably portrayed in the person’s own voice among untampered bits of their conversation. You wouldn’t accept that from a phone company, and nobody should take it from an ISP either.

  9. This also breaks the speed-up hack of shipping (g-zipped) JS over the wire as a comment, then unpacking it as required in order not to have to inflate it all at once if you don’t need all the features to begin with.

  10. O2 is guilty of creating breakages too. This is not related to tethering but just to regular access of web content over 3G (which to me is even more alarming)

    I have witnessed first hand (on the iPhone) scripts like jQuery / jQuery mobile breaking, and even base 64 encoded images stripped out from
    CSS stylesheets (oddly, inline base 64 encoded images in HTML go unaffected). This not only breaks functionality of websites, but stops pieces of content from being displayed altogether. A site will remain this way until you completely empty the cache, quit the browser, and re-visit the website over WIFI.

    This is a terrible state of affairs. I wish operators would stop trying to make such poor and badly thought out optimisations.

  11. Friday 22.06.2012

    T-Mobile remove any graphics and website names,
    including today, when they took off all my email envelopes and replaced them with their “clear gif” icon. Which looks like a small page, the top right corner is bent over the front of the page, containing a light blue dot, a light blue triangle, and below this, in the bottom left corner’s a pink triangle and under the triangles is a torn line across their icon. T-Mobile has done this on your website, the first time I’ve visited it today, as they’ve removed Tim Green’s picture.

    This can’t be legal as it’s breech of copyright and they don’t have anyone’s permission, except their own. I asked T-Mobile about 2 years ago why they do this and was told. “Graphics take up too much broadband
    and fair use policy.” I call their lies. “Unlimited Internet and fair use policy” what they really are. “Limited Internet and unfair use policy,” there are numerous online complaints about them ranging from: Stopping people getting online, disconnecting their Internet calls, saying people can browse between Midnight-04:00am, when this is virtually impossible, I know as I’ve tried many times in the past. They also steal from
    people’s bank accounts and off others PAYG accounts.
    I’ve read about the former and the latter they did to me, taking £7.50 from my PAYG when I’d 6 months free Internet, for all it’s worth, thinking I wouldn’t notice. I’d signed up for free for them to let me know when my balance went under £1.00, January 2011 they let me know but my balance was a lot more that that and I was suspicious. It took 3 calls to their 0845 number before it was sorted out, the first man wasn’t interested regarding my balance and then gave me a load of crap as to why it was usually so difficult to get online, blaming the area I live and the time I tried to get online, both were lies. The next thing he blamed was my phone saying. “It’s the lower end of the broadband market,” I said. “Why are T-Mobile still selling them then as I’ve seen them on your website?” to which he replied. “I don’t know.” The second call the woman said. “I’ll call you back in about an hour,” I only half believed her, thinking ‘no she won’t’ and no she didn’t. the third call the man sorted out as I was offered 6 free months Internet again which I chose, ending August 2011 and since then I’ve bought their. “MONTHWEB” FOR £5.00 and bought another on 01.06.2012, as £1.00 had been taken I was again suspicious, when I asked why the woman said. “There was a huge broadband connection and it’s £1.00 a day,” T-Mobile haven’t done it since, and my month’s web she said.”Will finish on the 21.06.2012,” WRONG as it’ll be 01.07.2012.

    I wouldn’t trust any firm who told their Scottish call centre staff. “Move to the Philippines or lose your job,”
    this was the heading to an online article.

    I Googled. “T-Mobile Internet Complaints,” and saw numerous complaints, including in the USA, I’ve Googled other variations and have read many too from the UK.

    T-Mobile need to be thoroughly investigated by the Police never mind the Ombudsman.