Intermittent Website Availability Issues

Incident Report for CodeShip

Postmortem

Please see the full incident report published on the Codeship blog: http://blog.codeship.com/dnsimple-ddos-outage/

Posted Dec 10, 2014 - 16:33 UTC

Resolved

We've included Cloudflare as an additional DNS provider and have monitored for the last hour. So far everything is running fine and we will keep monitoring
Posted Dec 02, 2014 - 04:10 UTC

Monitoring

Codeship's primary DNS provider continues to experience issues despite implementation of mitigation against the denial of service attack. Codeship has implemented a new DNS service provider, and name resolution issues should be resolved as the change propagates across nameservers, worldwide. We continue to carefully monitor this issue.
Posted Dec 02, 2014 - 03:14 UTC

Identified

Codeship's DNS provider has implemented new hardware and networking to overcome their ongoing denial of service attack. They report some name servers are coming back online, but they are still dealing with a partial DNS outage. They are continuing to work to resolve it and restore full availability.
Posted Dec 02, 2014 - 00:47 UTC

Update

Codeship's DNS provider continues to report an outage due to an ongoing denial of service attack. Although Codeship's service is currently operating normally, this DNS outage prevents customer requests to be properly routed over the Internet to Codeship's servers. Our DNS vendor is working diligently to mitigate the issue.
Posted Dec 01, 2014 - 21:12 UTC

Investigating

Codeship's DNS provider is reporting problems due to a DDoS attack. Engineers are currently working to mitigate the issue.
Posted Dec 01, 2014 - 20:04 UTC