Talking #censusfail on The Project

I went on the Project (amongst others) to talk about the poor old ABS and the census website failure. I experience mixed emotions watching their non-apology press conference today. On the one hand, big IT projects are hard they go wrong and I can well imagine the hell that is the lives of the engineers involved over the last and coming few days. However, I was nothing but amused at the rather ham-fisted arse-covering by the minister Michael McCormack and the Australian Statistician David Kalisch. Firstly, they blamed denial of service (DoS) attacks and an “abundance of caution” by themselves for the site being unavailable. At the same time, they admitted that DoS attacks were expected and they had a plan. It was really a “confluence of factors” that took the site down. These included:

• A router hardware failure
• A failure in the geolocation system used for blocking foreign IPs during a DDoS (is this the same as above?)
• A “false positive” by which I can only guess they meant an intrusion alarm was triggered, probably as the site was failing under the load
• A big DDoS attack that came at 7:30 as Australians were finishing dinner.

None of these are a one-in-a-million events. This isn’t an unfortunate series of coincidences, just an admission that the site was poorly architected and had several single points of failure. One router failing should not bring down the site. You build in redundancy. The geolocation should have had a backup, or choose a more reliable service. A faulty intrusion detection system is nobody’s fault but your own.

Finally, the 7:30 DoS attack has been greeted with much skepticism. The digital attack map didn’t report anything and by the ABS’ own admission they load tested up to about a million censuses an hour. I would have expected several times that in the after-dinner rush, slanted towards the early end.

No doubt time will tell but I will offer a sincere apology if this was really the work of the PLA cyber-warfare unit, avenging the honour of Sun Yang.

Here’s the Project clip.