blacksmith - Notice history

100.0% uptime

eu-central Storage Cluster - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

us-west Storage Cluster - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

eu-west Storage Cluster - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

us-east Storage Cluster - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

API - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 99.99%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024
100.0% uptime

https://blacksmith.sh - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

Github → Actions - Operational

Github → API Requests - Operational

Github → Webhooks - Operational

Codesmith - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

Dashboard - Operational

100.0% uptime
Sep 2024 · 100.0%Oct · 100.0%Nov · 100.0%
Sep 2024
Oct 2024
Nov 2024

Notice history

View current status

Nov 2024

Oct 2024

Increased queue times because of GitHub not assigning jobs to our runners
ResolvedPartial outage4 hours 13 minutes
  • ResolvedUTC

    GitHub has resolved the incident on their end, we are seeing normal queue times once again.

  • UpdateUTC

    GitHub is still continuing to investigate delays to status updates to Actions Workflow Runs, Workflow Job Runs, and Check Steps.

  • MonitoringUTC

    GitHub has declared an incident, we are continuing to monitor the situation - https://www.githubstatus.com/incidents/9yk1fbk0qjjc

  • InvestigatingUTC

    We are currently noticing higher than normal queue times because of GitHub not assigning jobs to some of our runners. We are noticing excessive queueing on ubuntu-latest as well.

Sep 2024

actions/checkout failing with upstream service unavailable
ResolvedPartial outage1 hour 53 minutes
  • ResolvedUTC

    This is now resolved.

  • MonitoringUTC

    We're no longer seeing issues with checkout actions and GitHub has closed their incident: https://www.githubstatus.com/incidents/1g9v7rry4z86. We're still monitoring this issue.

  • IdentifiedUTC

    GitHub has started an incident on their end: https://www.githubstatus.com/incidents/1g9v7rry4z86. We're still monitoring this issue on our end.

  • InvestigatingUTC

    Customers are reporting checkout actions failing with Error: Name or service not known (internal-api.service.iad.github.net:443. We are currently investigating this incident.

Github `actions/checkout` is failing with `fatal: protocol error: bad pack header` when trying to pull the repository
ResolvedDegraded performance57 minutes
  • ResolvedUTC

    This incident has been resolved by GitHub, operations should be back to normal.

  • IdentifiedUTC

    GitHub has opened an incident, we will continue monitoring the status of this - https://www.githubstatus.com/incidents/q3xqwmcxzkqq

  • InvestigatingUTC

    Both GitHub hosted runners and Blacksmith hosted runners are seeing an error when trying to checkout the repository in a GitHub workflow. This is indicative of an outage or bad deploy on GitHub's end, we are monitoring the situation.

Flakiness in `ubuntu/22.04/prod jammy/main arm64` mirror causing apt installs to occasionally fail
ResolvedDegraded performance6 hours 20 minutes
  • ResolvedUTC

    This incident has been resolved. This has been fixed upstream.

  • InvestigatingUTC

    We are noticing some tests failing while setting up the job and installing the required dependencies with the error File has unexpected size (43334 != 43043). Mirror sync in progress?

    There is an open issue tracking the problem with the upstream mirror https://github.com/microsoft/linux-package-repositories/issues/174, we will continue to monitor the situation.

Previous

Sep 2024 to Nov 2024

Next