GitHub’s Nearly 8-Hour Outage: How One Bottleneck Triggered a Retry Storm

A regional bottleneck became a worldwide developer crisis on August 17, 2026, when GitHub’s recovery machinery collided with a client retry bug. The result was nearly eight hours of errors across code review, automation, authentication and Copilot—and a sharp reminder that “the cloud” can still have a very physical choke point.

For a Port St. Lucie software shop, managed service provider, ecommerce operator or healthcare office with a development vendor, GitHub may sit far outside the building—but it can still be threaded through the business. A pull request may gate a security fix. An Actions workflow may deploy a customer portal. A webhook may update a ticket or kick off an integration. When GitHub stumbled, those invisible conveyor belts slowed, stopped or became uncertain.

What happened: a long afternoon for developers

GitHub’s official incident report places the disruption from 13:28 to 21:15 UTC on August 17. Issues, Pull Requests, APIs, Actions and Copilot suffered elevated errors or latency. Enterprise identity and provisioning functions—including SAML/OIDC authentication, SCIM and Team Sync—were affected too. GitHub also called out Actions workflows in GitHub Enterprise Cloud with Data Residency when they depended on public workflow definitions hosted on GitHub.com.

The outage did not behave like one clean on/off switch. Most services recovered by 16:36 UTC as the Central US datacenter recovered. Actions remained degraded until about 18:03. Copilot’s Token Service was the stubborn tail, not fully recovering until 21:02. GitHub formally resolved the overall incident at 21:15.

1. 13:28 UTC — GitHub’s retrospective start time. Users begin encountering service trouble.
2. 14:58 UTC — GitHub reports roughly 20% errors for web/API traffic and roughly 50% for archives and raw repository content; enterprise authentication and synchronization are affected.
3. 16:36 UTC — Most services have recovered as Central US stabilizes.
4. 18:03 UTC — The remaining Actions degradation ends.
5. 21:02 UTC — Copilot Token Service fully recovers after an extended retry-driven surge.
6. 21:15 UTC — Incident resolved after 7 hours, 47 minutes.

Independent coverage broadly matches that picture. Cyber Security News, publishing while the event was unfolding, reported the 20% and 50% figures, enterprise authentication impact, component chronology and a Downdetector complaint spike near the start. DevOps.com later described thousands of user reports and the disruption to core developer workflows. Those reports corroborate the visible impact; GitHub’s completed report is the controlling source for cause.

The root-cause chain: one limit, then a cascade

GitHub says the immediate failure was load-balancer network saturation in its Central US datacenter during a new traffic peak. The first technical domino was an Istio service-mesh sidecar pod reaching its concurrency limit. Autoscaling did not respond correctly because the policy watched the host service but not the sidecar’s capacity. In plain English: the system watched the engine while missing the narrowing exhaust pipe.

The failure cascaded until four HAProxy nodes exhausted their flow limits. That degraded the gateway authentication path, producing widespread authentication latency and failures. GitHub says optimistic retry logic then worsened the overload on internal load balancers. Pausing HAProxy on those nodes produced immediate broad recovery, while some failing traffic was moved from Central US to Northern Virginia.

That reroute helped—but it also exposed a latent retry bug in Visual Studio Code. Delayed responses from one internal endpoint caused the client to amplify traffic by about ten times. Normal Copilot Token Service volume of 7,000–9,000 requests per second surged to 70,000–100,000. A failed token operation could create many extra requests and loop, keeping pressure high after much of GitHub had recovered.

Engineers reduced gateway retries, temporarily blocked inbound Copilot token requests at load balancers with HTTP 403 responses, then gradually restored traffic site by site. GitHub also says scraping attacks against codeload endpoints complicated recovery. That is important context, but not a license to rewrite the story as a breach: GitHub identified load-balancer saturation, missed sidecar capacity and runaway retries as the causal chain. It did not say a cyberattack caused the outage, and there is no public evidence in the reviewed sources establishing one.

What broke—and what that means in the real world

Why the retry storm matters more than the headline

Retries are supposed to make software resilient. A transient request fails, the client waits and tries again. But retries without strict limits, exponential backoff and randomness can synchronize thousands of clients into a digital stampede. The service receives more work precisely when it has the least capacity. Recovery becomes its own outage.

This incident illustrates three layers of resilience. First, capacity controls must measure the constrained component, not merely its neighbor. Second, failover must account for how clients react to delay and error codes after traffic moves regions. Third, recovery procedures need a way to shed load safely rather than inviting every caller to hammer the door again.

GitHub’s announced follow-ups map directly to those layers: sidecar-aware autoscaling; audits of Istio request, concurrency and scaling limits; review of retry limits and backoff across gateways and clients; a fix for the VS Code behavior; and better load-balancer capacity and regional-failover monitoring. These are GitHub’s planned actions, not proof that each change is already deployed.

What Port St. Lucie teams should do now

1. Check the status page before debugging your own network. Bookmark GitHub Status and subscribe to updates. Compare the affected component and time window with your failed job.
2. Pause deployments deliberately. If source control, Actions, packages or authentication is unstable, stop automatic promotion. Retry only after understanding whether a job is safe to repeat; a timeout does not prove nothing happened.
3. Reconcile webhooks and automation. Verify delivery records, replay only missing events and make handlers idempotent so duplicates cannot create duplicate tickets, invoices, releases or configuration changes.
4. Keep local clones—and know their limits. A current local clone lets developers inspect history and continue some work offline. For critical repositories, consider a governed read-only mirror. Protect credentials and do not create an unmanaged source-code sprawl.
5. Design against retry storms. Use bounded retries, exponential backoff, jitter, circuit breakers and concurrency caps. Track total attempts, not only final failures, so amplification is visible.
6. Build runner and dependency resilience. Where the risk justifies it, maintain self-hosted or alternate runner capacity. Pin dependency versions and use a controlled cache or artifact repository so a raw-content failure does not unexpectedly change or halt a build.
7. Communicate business status, not just vendor status. Tell staff which releases, integrations and customer promises are affected. Record decisions and run a post-incident reconciliation after the vendor turns green.

A second incident—and one mistaken alert

On August 18, GitHub tracked a separate, minor incident from 07:40 to 11:42 UTC involving a communication issue between Actions services. Customers using Larger Runners could have trouble loading runner groups and related permission pages. GitHub applied a mitigation, resolved the incident and said a detailed root-cause analysis would follow. It should not be folded into the previous day’s root-cause narrative without evidence connecting them.

Another Actions incident was listed from 09:36 to 10:23 UTC, but GitHub later stated that it had been posted in error. According to that correction, Actions workflows were not impacted and workflow runs continued normally. This is a useful operational lesson of its own: status alerts can be corrected, so teams should preserve timestamps and confirm what actually executed before mass-retrying jobs.

FAQ

The bottom line

GitHub’s August 17 outage was not simply “the website went down.” It was a systems story: one unobserved sidecar limit, four saturated proxy nodes, a strained authentication path, regional rerouting and retries that turned recovery traffic into a second wave. For Treasure Coast organizations, the takeaway is not to abandon a vital platform. It is to know where that platform sits in your operations, reduce unsafe automation, and practice what happens when a shared dependency becomes unreliable for an afternoon.