CodeShip Pro Build Hooks
Incident Report for CodeShip
Postmortem

During a credential rotation exercise, authorization was temporarily revoked between two components of our webhook processing system. We quickly identified this failure and set about to restoring authorized connectivity between these components. During this outage all of the webhook's were safely being captured and were eventually processed once connectivity was restored. Resulting in delay but not loss of any builds within our system. We will be reviewing our credential rotation strategy in response to this incident. We sincerely apologize for this delay to your workflow.

Posted Nov 29, 2018 - 12:53 UTC

Resolved
Build are processing webhooks normally on the CodeShip Pro platform. We apologize for any impact this has had on your workflow.
Posted Nov 29, 2018 - 11:36 UTC
Update
We have identified and fixed the issue processing webhooks. We are currently replaying build hooks which were temporarily stalled by the system problem. We apologise again for any impact this has had on your workflow. Our engineers are continuing to monitor the outage.
Posted Nov 29, 2018 - 11:26 UTC
Investigating
We are currently experiencing issues with webhooks being processed on the CodeShip Pro platform. We apologize for the impact this is having on your workflow. Our engineers are actively investigating.
Posted Nov 29, 2018 - 11:10 UTC
This incident affected: CodeShip Pro (Builds).