Introducing Railgun

CloudFlare Railgun™

As a CloudFlare Optimized Partner, we are thrilled to offer the CloudFlare Railgun™ technology to all our customers. Railgun is CloudFlare’s latest performance optimization technology that gives you significant improvements in site load times. To activate Railgun, log into your control panel and select Railgun “On”. Read on for additional details and instructions:

railgun-map

Railgun ensures that the connection between our network and the CloudFlare network is as fast as possible. Railgun achieves a 99.6% compression ratio for previously uncacheable web objects by using techniques similar to those used in the compression of high-quality video. The average website can expect a 1.43x performance increase.

railgun-details

When a request is made to a CloudFlare server for a web page that is not in cache CloudFlare makes an HTTP connection to the origin server to request the page. It’s that HTTP connection that Railgun accelerates and secures.

Even highly dynamic websites change slowly

Railgun works by recognizing that uncacheable web pages do not change very rapidly. For example, during an experiment, the CNN.com homepage HTML was captured once, and then again after 5 minutes and then again after one hour. The page sizes were 92,516, five minutes still 92,516 and one hour later 93,727.

CNN sets the caching on this page to 60 seconds. After one minute, it is necessary to download the entire page again. However, looking inside the page, not much has changed. In fact, the change between versions is on the order of 100s of bytes out of almost 100k. Here’s a screenshot of one of the binary differences between the CNN home page at five minute intervals. The yellow bytes have changed, the rest have not.

railgun-data

Experiments at CloudFlare have revealed similar change values across the web. For example, reddit.com changes by about 2.15% over five minutes and 3.16% over an hour. The New York Times home page changes by about 0.6% over five minutes and 3% over an hour. BBC News changes by about 0.4% over five minutes and 2% over an hour.

Although the dynamic web is not cacheable, it is also not changing quickly. That means that from moment to moment there’s only a small change between versions of a page. CloudFlare Railgun uses this fact to achieve very high rates of compression. This is very similar to how video compression looks for changes from frame to frame; Railgun looks for changes on a page from download to download.

Railgun Technology

Railgun consists of two components: the sender and the listener. The sender is installed at every CloudFlare data center around the world. The listener is a software component that we have installed on our network for customers.

The sender and listener establish a permanent TCP connection that’s secured by TLS. This TCP connection is used for the Railgun protocol. It’s an all binary multiplexing protocol that allows multiple HTTP requests to be run simultaneously and asynchronously across the link. Or in other words, a persistent connection is opened between CloudFlare’s network and ours and eliminates the overhead of TCP.

TV election debates create ‘plausible tarts’, warns historian

Peter Hennessy said that he had “grave reservations” about the broadcasting of the leadership debates, which took place in Britain for the first time before the 2010 ­general election.

Warning that they meant future leaders would be selected for being “plausible tarts” rather than simply “decent”, he said a culture of “celebritocracy” could blight politics.

Lord Hennessy, a cross-bench peer, added that Clement Attlee, the post-war Labour prime minister, would never be elected today because his “considerable virtues” would be eclipsed by his “charisma and presence of a gerbil”.

“The problem with the debates is that the structure of them, the nature of the celebrity in Britain in parts of the media these days, mean that to shine in leadership debates before a general election, you need the characteristics of a plausible tart,” he said.

“Now, being a plausible tart is about 10 per cent of the requirement of being prime minister — no more. It’s not the most important bit.

“My fear is that we’ll be stuck with these leadership elections and when parties choose leaders in the future, the plausible tart bit will play too powerfully in their choice of leader and therefore rule out the decent but non-tarty people.”

“They’re only happy in their own company.

Speaking of the perils of 24-hour news, Lord Hennessy also criticised the habit of parties to rush into responses and “collapse in tittishness” by posting reactions to everything on Twitter.

“The cumulative power of celebritocracy and soundbitery means that it’s an ever narrower spectrum in which you operate,” he said.

“Clement Attlee wouldn’t get through any selection process now because his answers would be monosyllabic.

“He was a man of considerable virtue but it would be unimaginable today, because he had all the charisma and presence of a gerbil. It’s inconceivable now. If there was such a character around, he or she wouldn’t make it.”

via Ways With Words 2013: TV election debates create ‘plausible tarts’, warns historian – Telegraph.

HP laserjet P1102W wifi setup fail

Here’s an unofficial but quite professional screencast showing how it should work:

First I try to connect to the printer network:

Connect to P1102W network
Connect to P1102W network

Then I try opening the print setup:

Then Print Utility
and nothing happens apart from the OSX lit up click lamp on the button.which I tried despite the fact that the open printer utility fails (it should launch a web page hosted by the printer itself).

This is the start of the problem: it never seems to know where it is, or if it does it won’t volunteer this information using the Bonjour protocol by default, but it’s address does appear in tcpdump when connected to its own “device” network:

The official way to find this out is via by printing out a test page. The address assigned is always on the 169 subnet and keeps changing. When I apply the changes to the wireless alone or the wireless and what are the correct network settings together it goes in to a hanging mode where the printout shows it has attached itself to my network but its address is still on the 169 subnet and it’s using its own dhcp.

One last thing to try might be to set my own network to the same as this but as there are at several other devices on 192 I’m reluctant to even try that out.

Configuration printouts
configuration print outs to pdf
tcpdump
Tom-Coadys-MacBook-Pro:~ tomxcoady$ sudo tcpdump -i en1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en1, link-type EN10MB (Ethernet), capture size 65535 bytes
16:37:56.902414 ARP, Request who-has 169.254.87.142 tell 0.0.0.0, length 46
16:37:57.780097 IP tom-coadys-macbook-pro.local.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 142.87.254.169.in-addr.arpa. (45)
16:37:57.780429 IP6 tom-coadys-macbook-pro.local.mdns > ff02::fb.mdns: 0 PTR (QM)? 142.87.254.169.in-addr.arpa. (45)
16:37:58.781839 IP tom-coadys-macbook-pro.local.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 142.87.254.169.in-addr.arpa. (45)
16:37:58.782079 IP6 tom-coadys-macbook-pro.local.mdns > ff02::fb.mdns: 0 PTR (QM)? 142.87.254.169.in-addr.arpa. (45)
16:37:58.892288 ARP, Request who-has 169.254.87.142 tell 0.0.0.0, length 46
16:38:00.882314 ARP, Request who-has 169.254.87.142 tell 0.0.0.0, length 46
16:38:01.948848 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:38:02.871973 ARP, Request who-has 169.254.87.142 tell 0.0.0.0, length 46
16:38:03.212434 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:38:05.382180 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:38:06.852627 ARP, Request who-has 169.254.87.142 tell 169.254.87.142, length 46
16:38:08.846282 ARP, Request who-has 169.254.87.142 tell 169.254.87.142, length 46
16:38:09.568499 IP6 fe80::f6ce:46ff:fe4f:e3ce > ff02::1:ff4f:e3ce: HBH ICMP6, multicast listener reportmax resp delay: 0 addr: ff02::1:ff4f:e3ce, length 24
16:38:09.570396 IP6 :: > ff02::1:ff4f:e3ce: ICMP6, neighbor solicitation, who has fe80::f6ce:46ff:fe4f:e3ce, length 24
16:38:09.571402 IP6 fe80::f6ce:46ff:fe4f:e3ce > ff02::2: ICMP6, router solicitation, length 8
16:38:09.767804 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:38:09.822698 IP tom-coadys-macbook-pro.local.mdns > 224.0.0.251.mdns: 0 PTR (QM)? e.c.3.e.f.4.e.f.f.f.6.4.e.c.6.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (90)
16:38:09.822795 IP6 tom-coadys-macbook-pro.local.mdns > ff02::fb.mdns: 0 PTR (QM)? e.c.3.e.f.4.e.f.f.f.6.4.e.c.6.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (90)
16:38:09.837470 IP6 fe80::f6ce:46ff:fe4f:e3ce.dhcpv6-client > ff02::1:2.dhcpv6-server: dhcp6 solicit
16:38:13.499378 IP6 tom-coadys-macbook-pro.local.mdns > ff02::fb.mdns: 0 [5a] [15q] PTR (QM)? _afpovertcp._tcp.local. PTR (QM)? _smb._tcp.local. PTR (QM)? _rfb._tcp.local. PTR (QM)? _adisk._tcp.local. PTR (QM)? _see._tcp.local. PTR (QM)? _daap._tcp.local. PTR (QM)? _appletv-pair._tcp.local. PTR (QM)? _appletv._tcp.local. PTR (QM)? _00000000-03d9-f72b-4ca8-f84b129d4330._sub._appletv-v2._tcp.local. PTR (QM)? _00000000-03d9-f72b-4ca8-f84b129d4330._sub._home-sharing._tcp.local. PTR (QM)? _touch-remote._tcp.local. PTR (QM)? _raop._tcp.local. PTR (QM)? _airplay._tcp.local. SRV (QM)? HP LaserJet Professional P1102w._riousbprint._tcp.local.._http._tcp.local. TXT (QM)? HP LaserJet Professional P1102w._riousbprint._tcp.local.._http._tcp.local. (585)
16:38:13.547869 IP6 fe80::f6ce:46ff:fe4f:e3ce > ff02::2: ICMP6, router solicitation, length 8
16:38:14.546318 IP 169.254.87.142 > 239.255.255.250: igmp v2 report 239.255.255.250
16:38:14.547427 IP6 fe80::f6ce:46ff:fe4f:e3ce > ff02::c: HBH ICMP6, multicast listener reportmax resp delay: 0 addr: ff02::c, length 24
16:38:14.762537 IP 169.254.87.142.upnp-discovery > 239.255.255.250.upnp-discovery: UDP, length 1281
16:38:14.782386 IP 169.254.87.142.upnp-discovery > 239.255.255.250.upnp-discovery: UDP, length 1281
16:38:14.815985 IP 169.254.87.142 > 224.0.1.60: igmp v2 report 224.0.1.60
16:38:14.816952 IP6 fe80::f6ce:46ff:fe4f:e3ce > ff02::116: HBH ICMP6, multicast listener reportmax resp delay: 0 addr: ff02::116, length 24
16:38:14.818064 IP6 fe80::f6ce:46ff:fe4f:e3ce > ff02::fb: HBH ICMP6, multicast listener reportmax resp delay: 0 addr: ff02::fb, length 24
16:38:14.852390 IP6 fe80::f6ce:46ff:fe4f:e3ce.upnp-discovery > ff02::c.upnp-discovery: UDP, length 1281
16:38:18.164147 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:38:19.799281 IP 169.254.87.142.mdns > 224.0.0.251.mdns: 0 [6q] [6n] ANY (QM)? NPI4FE3CE.local. ANY (QM)? NPI4FE3CE.local. ANY (QM)? HP LaserJet Professional P1102w._pdl-datastream._tcp.local. ANY (QM)? HP LaserJet Professional P1102w._http._tcp.local. ANY (QM)? HP LaserJet Professional P1102w._http-alt._tcp.local. ANY (QM)? HP LaserJet Professional P1102w._printer._tcp.local. (361)
16:38:19.803338 IP6 fe80::f6ce:46ff:fe4f:e3ce.mdns > ff02::fb.mdns: 0 [6q] [6n] ANY (QM)? NPI4FE3CE.local. ANY (QM)? NPI4FE3CE.local. ANY (QM)? HP LaserJet Professional P1102w._pdl-datastream._tcp.local. ANY (QM)? HP LaserJet Professional P1102w._http._tcp.local. ANY (QM)? HP LaserJet Professional P1102w._http-alt._tcp.local. ANY (QM)? HP LaserJet Professional P1102w._printer._tcp.local. (361)
16:38:20.371449 IP tom-coadys-macbook-pro.local.mdns > 224.0.0.251.mdns: 0*- [0q] 38/0/0 (Cache flush) TXT "", PTR _afpovertcp._tcp.local., PTR Tom CoadyM-bM-^@M-^Ys MacBook Pro._afpovertcp._tcp.local., TXT "model=MacBookPro5,1", (Cache flush) TXT "", PTR _smb._tcp.local., PTR Tom CoadyM-bM-^@M-^Ys MacBook Pro._smb._tcp.local., (Cache flush) TXT "", PTR _ssh._tcp.local., PTR Tom CoadyM-bM-^@M-^Ys MacBook Pro._ssh._tcp.local., (Cache flush) TXT "", PTR _sftp-ssh._tcp.local., PTR Tom CoadyM-bM-^@M-^Ys MacBook Pro._sftp-ssh._tcp.local., (Cache flush) SRV Tom-Coadys-MacBook-Pro.local.:548 0 0, (Cache flush) SRV Tom-Coadys-MacBook-Pro.local.:445 0 0, (Cache flush) SRV Tom-Coadys-MacBook-Pro.local.:22 0 0, (Cache flush) SRV Tom-Coadys-MacBook-Pro.local.:22 0 0, (Cache flush) TXT "sys=waMA=00:23:DF:7C:A1:46,adVF=0x200,adDT=0x3,adCC=1", PTR _odisk._tcp.local., PTR Tom CoadyM-bM-^@M-^Ys MacBook Pro._odisk._tcp.local., (Cache flush) TXT "", PTR _rfb._tcp.local., PTR Tom CoadyM-bM-^@M-^Ys MacBook Pro._rfb._tcp.local., TXT "LKDC:SHA1.9ABB9D91F95F0388A329FF682FF826A158AB6AD7", (Cache flush) SRV Tom-Coadys-MacBook-Pro.local.:49152 0 0, (Cache flush) SRV Tom-Coadys-MacBook-Pro.local.:5900 0 0, (Cache flush) TXT "txtvers=1" "userid=47837200-6C10-44B1-9012-60659F49E5F4" "name=Tom Coady" "version=2", PTR _see._tcp.local., PTR [email protected] CoadyM-bM-^@M-^Ys MacBook Pro._see._tcp.local., (Cache flush) SRV Tom-Coadys-MacBook-Pro.local.:6942 0 0, (Cache flush) TXT "txtvers=1" "DbId=EC2F25162B4B504E" "CtlN=Tom CoadyM-bM-^@M-^Ys Library" "OSsi=0x1F6" "DvSv=2577" "DvTy=iTunes" "iV=196612" "Ver=131074", PTR _touch-able._tcp.local., PTR 89503FFFDFF532F5._touch-able._tcp.local., (Cache flush) TXT "txtvers=1" "DbId=EC2F25162B4B504E" "CtlN=Tom CoadyM-bM-^@M-^Ys Library" "OSsi=0x1F6" "msg=" "DvSv=2577" "DvTy=iTunes" "iV=196612" "Ver=131074", PTR _remote-jukebox._tcp.local., PTR 89503FFFDFF532F5._remote-jukebox._tcp.local., PTR _home-sharing._tcp.local., PTR Tom CoadyM-bM-^@M-^Ys Library._home-sharing._tcp.local. (1448)
16:38:42.133646 IP6 fe80::f6ce:46ff:fe4f:e3ce.dhcpv6-client > ff02::1:2.dhcpv6-server: dhcp6 solicit
16:38:43.756270 ARP, Request who-has 192.168.1.254 tell 192.168.1.85, length 28
16:38:52.339011 ARP, Request who-has 192.168.1.254 tell 192.168.1.85, length 28
16:38:57.015542 IP tom-coadys-macbook-pro.local.50887 > 169.254.87.142.http: Flags [S], seq 3051841237, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 997133271 ecr 0,sackOK,eol], length 0
16:38:57.018243 ARP, Request who-has tom-coadys-macbook-pro.local tell 169.254.87.142, length 46
16:38:57.018295 ARP, Reply tom-coadys-macbook-pro.local is-at 00:23:6c:83:16:a4 (oui Unknown), length 28
16:38:57.020959 IP 169.254.87.142.http > tom-coadys-macbook-pro.local.50887: Flags [S.], seq 975060428, ack 3051841238, win 8688, options [mss 1460,nop,nop,sackOK,nop,nop,TS val 194040 ecr 997133271], length 0
16:38:57.021017 IP tom-coadys-macbook-pro.local.50887 > 169.254.87.142.http: Flags [.], ack 1, win 65535, options [nop,nop,TS val 997133271 ecr 194040], length 0
16:39:01.145555 ARP, Request who-has 192.168.1.254 tell 192.168.1.85, length 28
16:39:01.145676 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:39:01.160689 ARP, Request who-has 10.105.116.1 tell 10.105.119.19, length 28
16:39:14.061850 IP6 fe80::f6ce:46ff:fe4f:e3ce.dhcpv6-client > ff02::1:2.dhcpv6-server: dhcp6 solicit
16:39:17.024395 IP tom-coadys-macbook-pro.local.50887 > 169.254.87.142.http: Flags [P.], seq 2577:3018, ack 46055, win 65535, options [nop,nop,TS val 997133471 ecr 210780], length 441
HP LaserJet Professional P1102w._printer._tcp.local. (1239)
16:39:28.829295 IP tom-coadys-macbook-pro.local.netbios-dgm > 169.254.255.255.netbios-dgm: NBT UDP PACKET(138)
16:39:29.798610 IP tom-coadys-macbook-pro.local.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 255.255.254.169.in-addr.arpa. (46)
16:39:29.798948 IP6 tom-coadys-macbook-pro.local.mdns > ff02::fb.mdns: 0 PTR (QM)? 255.255.254.169.in-addr.arpa. (46)
16:39:30.801447 IP tom-coadys-macbook-pro.local.mdns > 224.0.0.251.mdns: 0 PTR (QM)? 255.255.254.169.in-addr.arpa. (46)
16:39:30.801681 IP6 tom-coadys-macbook-pro.local.mdns > ff02::fb.mdns: 0 PTR (QM)? 255.255.254.169.in-addr.arpa. (46)
16:39:33.460831 IP tom-coadys-macbook-pro.local.50887 > 169.254.87.142.http: Flags [F.], seq 3370, ack 61883, win 65535, options [nop,nop,TS val 997133635 ecr 214780], length 0
16:39:33.461030 IP tom-coadys-macbook-pro.local.50888 > 169.254.87.142.http: Flags [F.], seq 3124, ack 67503, win 65535, options [nop,nop,TS val 997133635 ecr 214780], length 0
16:40:40.881671 IP tom-coadys-macbook-pro.local.50893 > 169.254.87.142.http: Flags [F.], seq 763, ack 5567, win 65535, options [nop,nop,TS val 997134308 ecr 230690], length 0
16:40:42.840454 IP tom-coadys-macbook-pro.local.netbios-ns > 169.254.255.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
16:40:42.895153 IP tom-coadys-macbook-pro.local.netbios-dgm > 169.254.255.255.netbios-dgm: NBT UDP PACKET(138)
16:40:45.547211 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:40:53.592388 ARP, Request who-has 10.105.116.1 tell 10.105.119.19, length 28
16:40:53.592431 ARP, Request who-has 192.168.1.254 tell 192.168.1.85, length 28
16:40:53.592556 IP 0.0.0.0.bootpc > broadcasthost.bootps: BOOTP/DHCP, Request from 00:23:6c:83:16:a4 (oui Unknown), length 300
16:40:53.607519 ARP, Request who-has 10.105.116.1 tell 10.105.119.19, length 28
16:40:53.607545 ARP, Request who-has 192.168.1.254 tell 192.168.1.85, length 28
16:40:53.622628 ARP, Request who-has 10.105.116.1 tell 10.105.119.19, length 28
16:40:53.622650 ARP, Request who-has 192.168.1.254 tell 192.168.1.85, length 28

Wikileaks evades hackers with shift to Amazon EC2

Wikileaks, the site that has infuriated the US government by releasing thousands of US diplomatic cables, is being hosted by one of the symbols of that country’s internet success – Amazon.

The site came under a “Distributed Denial of Service” (DDOS) attack on Sunday night from an unidentified hacker, forcing it to seek a new location for its computer files. And it found it though Amazon’s “Elastic Cloud Computing” (EC2) service, which enables businesses to hire its servers and store their data there.

DDOS attacks typically force sites off the net unless they have enormous bandwidth at their disposal or highly effective countermeasures. Wikileaks, being small and struggling for funds, is neither.

But EC2 allows companies to pay for their usage as it mounts up, rather than upfront.

While Amazon is American, not all of its servers are hosted there – and it could cause a major incident if the US government were to take action against a company on the basis that it might be hosting material the government finds embarrassing.

via Wikileaks evades hackers with shift to Amazon | Media | guardian.co.uk.

One example of Twitter’s excellent UX

Brilliant example of Twitter’s idiot proof design :

Sorry, ignore the highlighted words – I was using that as a lazy way to post this blog using the excellent Press This WordPress bookmarklet, and the tweet itself was lazily created the same way. FWIW I resent the tweet as the first attempt resulted in a fairly dire server busy error. Maybe its that unreliability that inspired the sanity check loop?