Pet Tech Pro Website Outage June 22, 2021

Incident Report for Pet Tech Pro

Postmortem

Which customers were affected?

All customers with Pet Tech Pro sites.

Why did this happen?

All of the Pet Tech Pro customer sites are stored in Amazon Web Services in Sydney. The “DNS” that points to those sites is provided by a third party DNS provider in Australia. It appears that company was targeted by a “Denial of Service” attack (essentially a flood of fake queries) which caused their DNS system to fail under the load, thereby impacting on Pet Tech Pro (and many other thousands of people’s websites).

What is “DNS”?

DNS or “Domain Name Services” is basically how the internet finds websites and directs people to them. It’s a bit like the old fashioned street directory books. So when you type in www.pettechpro.com.au to your browser it is the “DNS” system that locates where the Server is containing the website you want to visit then tells your computer exactly how to locate it. Obviously this DNS is critical, if it fails then you are not able to access the website.

How will we prevent this from happening in the future?

As this DNS platform is currently managed by a third party, we are not in a position to state which processes in the recovery or failover failed, but we can confirm that the DNS system is progressively moving away to the “AnyCast network” which will avoid such outages in future as it uses multiple locations to provide the DNS service and is less susceptible to a Denial of Service attack.

In addition, yesterday we made an alteration to the DNS settings for all Pet Tech Pro customer sites which will mean that any future outage will only affect customers if the outage lasts longer than 24 hours. The change we made basically tells the internet to ‘remember’ our DNS details for 24 hours therefore if any DNS outrages occur our sites will still be accessible to our customers.