Stuck Builds on Codeship Basic
Incident Report for Codeship

We published a post mortem for this incident over at our blog. Please see Postmortem: Codeship Basic Platform Build Processing Incidents for more details.

TL;DR A change to our Codeship Basic machine pool configuration uncovered an issue with build container created_at timestamps, leading to pipeline failures due to programmatic termination of long-running processes by the Codeship Basic build machine agent.

Posted 2 months ago. Jun 16, 2017 - 10:20 UTC

Resolved
Builds are processing as planned. We keep monitoring the build queues and will make further changes to improve stability.
Posted 2 months ago. Jun 05, 2017 - 21:31 UTC
Monitoring
We've resolved the issue and are monitoring our systems. There still might be builds affected by this and we are identifying those and taking steps to unstuck those builds.
Posted 2 months ago. Jun 05, 2017 - 18:16 UTC
Update
We are taking necessary steps to keep impact on users as minimal as possible and are further looking into the root cause for this.
Posted 2 months ago. Jun 05, 2017 - 15:58 UTC
Investigating
We are seeing builds stuck in queue erroneously and are investigating further.
Posted 2 months ago. Jun 05, 2017 - 14:01 UTC