blacksmith - US-West storage cluster degraded – Incident details

All systems operational

US-West storage cluster degraded

Resolved
Partial outage 30 %
Started 6 days agoLasted 21 minutes

Affected

Incremental Docker Builders

Partial outage from 6:16 AM to 6:25 AM, Degraded performance from 6:25 AM to 6:37 AM

us-west Storage Cluster

Partial outage from 6:16 AM to 6:25 AM, Degraded performance from 6:25 AM to 6:37 AM

Docker Container Cache

Partial outage from 6:16 AM to 6:25 AM, Degraded performance from 6:25 AM to 6:37 AM

us-west Storage Cluster

Partial outage from 6:16 AM to 6:25 AM, Degraded performance from 6:25 AM to 6:37 AM

Stickydisks

Partial outage from 6:16 AM to 6:25 AM, Degraded performance from 6:25 AM to 6:37 AM

us-west Storage Cluster

Partial outage from 6:16 AM to 6:25 AM, Degraded performance from 6:25 AM to 6:37 AM

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    We've applied a fix and are seeing failure rates and latency starting to come down.

  • Investigating
    UTC
    Investigating

    We are seeing some failures with sticky disk availability and increased latency with the US-West storage cluster.