blacksmith - Job adoption delays – Incident details

All systems operational

Job adoption delays

Resolved
Degraded performance
Started 4 months agoLasted about 4 hours

Affected

Blacksmith Managed Runners

Degraded performance from 3:09 PM to 6:52 PM

US West

Degraded performance from 3:09 PM to 6:52 PM

us-west ARM

Degraded performance from 3:09 PM to 6:52 PM

us-west x86

Degraded performance from 3:09 PM to 6:52 PM

EU Central

Degraded performance from 3:09 PM to 6:52 PM

eu-central ARM

Degraded performance from 3:09 PM to 6:52 PM

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Update
    UTC
    Update

    We've identified and promoted a few more fixes for the delayed job adoption. We are still monitoring.

  • Monitoring
    UTC
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    UTC
    Identified

    We've identified a load-related issue in our control plane that is causing delays in job adoption. We're working on a mitigation.