blacksmith - Degraded US East AWS connectivity – Incident details

All systems operational

Degraded US East AWS connectivity

Resolved
Degraded performance
Started 6 days agoLasted about 1 hour

Affected

Blacksmith Managed Runners

Degraded performance from 1:11 AM to 1:58 AM

US East

Degraded performance from 1:11 AM to 1:58 AM

us-east x86

Degraded performance from 1:11 AM to 1:58 AM

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    We have identified a small subset of runners that are exhibiting this network stalling and have isolated them. We're continuing to monitor recovery.

  • Investigating
    UTC
    Investigating

    We are currently observing network stalling in our US East region with runner connectivity against AWS's us-east-1 region. Uploads and downloads to and from ECR and S3 may experience occasional stalling. We are investigating.