blacksmith - Degradation in US West storage cluster – Incident details

eu-west Storage Cluster under maintenance

Degradation in US West storage cluster

Resolved
Degraded performance
Started about 1 year agoLasted 3 hours 35 minutes

Affected

Incremental Docker Builders

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    Fix has been deployed, and we are currently monitoring the state of the cluster.

  • Identified
    UTC
    Identified

    We're about to deploy a fix, this will result in a few minutes of downtime and all stickydisk artifacts being cleared.

  • Monitoring
    UTC
    Monitoring

    We are still seeing signs of degradation in bursts as the storage cluster trends towards recovery. We will continue monitoring the situation and provide updates here.

  • Resolved
    UTC
    Resolved

    This incident has been resolved. We had a sudden surge of usage that required us to scale up our storage cluster.

  • Investigating
    UTC
    Investigating
    We are currently investigating this incident.