Status | Blacksmith - Notice history

Blacksmith Managed Runners - Operational

100% - uptime
Sep 2024 · 100%Oct · 99.95%Nov · 99.99%
Sep 2024
Oct 2024
Nov 2024

API - Operational

100% - uptime
Sep 2024 · 100%Oct · 100.0%Nov · 99.99%
Sep 2024
Oct 2024
Nov 2024

Website - Operational

100% - uptime
Sep 2024 · 100%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

Github → Actions - Operational

Github → API Requests - Operational

Github → Webhooks - Operational

Notice history

Oct 2024

Increased queue times because of GitHub not assigning jobs to our runners
  • Resolved
    Resolved

    GitHub has resolved the incident on their end, we are seeing normal queue times once again.

  • Update
    Update

    GitHub is still continuing to investigate delays to status updates to Actions Workflow Runs, Workflow Job Runs, and Check Steps.

  • Monitoring
    Monitoring

    GitHub has declared an incident, we are continuing to monitor the situation - https://www.githubstatus.com/incidents/9yk1fbk0qjjc

  • Investigating
    Investigating

    We are currently noticing higher than normal queue times because of GitHub not assigning jobs to some of our runners. We are noticing excessive queueing on ubuntu-latest as well.

Sep 2024

actions/checkout failing with upstream service unavailable
  • Resolved
    Resolved

    This is now resolved.

  • Monitoring
    Monitoring

    We're no longer seeing issues with checkout actions and GitHub has closed their incident: https://www.githubstatus.com/incidents/1g9v7rry4z86. We're still monitoring this issue.

  • Identified
    Identified

    GitHub has started an incident on their end: https://www.githubstatus.com/incidents/1g9v7rry4z86. We're still monitoring this issue on our end.

  • Investigating
    Investigating

    Customers are reporting checkout actions failing with Error: Name or service not known (internal-api.service.iad.github.net:443. We are currently investigating this incident.

Github `actions/checkout` is failing with `fatal: protocol error: bad pack header` when trying to pull the repository
  • Resolved
    Resolved

    This incident has been resolved by GitHub, operations should be back to normal.

  • Identified
    Identified

    GitHub has opened an incident, we will continue monitoring the status of this - https://www.githubstatus.com/incidents/q3xqwmcxzkqq

  • Investigating
    Investigating

    Both GitHub hosted runners and Blacksmith hosted runners are seeing an error when trying to checkout the repository in a GitHub workflow. This is indicative of an outage or bad deploy on GitHub's end, we are monitoring the situation.

Flakiness in `ubuntu/22.04/prod jammy/main arm64` mirror causing apt installs to occasionally fail
  • Resolved
    Resolved

    This incident has been resolved. This has been fixed upstream.

  • Investigating
    Investigating

    We are noticing some tests failing while setting up the job and installing the required dependencies with the error File has unexpected size (43334 != 43043). Mirror sync in progress?

    There is an open issue tracking the problem with the upstream mirror https://github.com/microsoft/linux-package-repositories/issues/174, we will continue to monitor the situation.

Sep 2024 to Nov 2024

Next