blacksmith - Flakiness in `ubuntu/22.04/prod jammy/main arm64` mirror causing apt installs to occasionally fail – Incident details

All systems operational

Flakiness in `ubuntu/22.04/prod jammy/main arm64` mirror causing apt installs to occasionally fail

Resolved
Degraded performance
Started about 2 years agoLasted about 2 hours

Affected

Blacksmith Managed Runners

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    We've not seen the flakiness in a while but are continuing to monitor the situation. There are a few relevant issues in https://github.com/microsoft/linux-package-repositories/issues which further motivate this being an issue with the upstream mirror.

  • Investigating
    UTC
    Investigating

    We are noticing some tests failing while setting up the job and installing the required dependencies with the error File has unexpected size (43334 != 43043). Mirror sync in progress?

    This usually indicates an upstream mirror is down, but we are investigating the issue.