blacksmith - Actions Cache degradation in us-west – Incident details

All systems operational

Actions Cache degradation in us-west

Resolved
Degraded performance
Started about 2 months agoLasted about 7 hours

Affected

Incremental Docker Builders

Degraded performance from 3:23 PM to 8:25 PM, Operational from 8:25 PM to 9:08 PM, Degraded performance from 9:08 PM to 9:55 PM

us-west Storage Cluster

Degraded performance from 3:23 PM to 8:25 PM, Operational from 8:25 PM to 9:08 PM, Degraded performance from 9:08 PM to 9:55 PM

Docker Container Cache

Degraded performance from 3:23 PM to 8:25 PM, Operational from 8:25 PM to 9:08 PM, Degraded performance from 9:08 PM to 9:55 PM

us-west Storage Cluster

Degraded performance from 3:23 PM to 8:25 PM, Operational from 8:25 PM to 9:08 PM, Degraded performance from 9:08 PM to 9:55 PM

Actions Cache

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    UTC
    Identified

    We are running a maintenance operation on our cache storage cluster to resolve the degradation. For roughly 15 to 20 minutes, caching will fall back to the GitHub Actions cache and may be slower; jobs will otherwise run normally. We will update once complete.

  • Update
    UTC
    Update

    We are continuing to investigate a network issue with a storage server in our US West region since around 14:50 UTC on July 6. Customers may experience slowness in US West. We are working with our upstream network provider on a fix and will post updates here.

  • Investigating
    UTC
    Investigating

    We are currently investigating this incident.