"I panicked, what happened?" Cloudflare outage causes global internet chaos.

Written by: Zhao Yuhe

Source: Wall Street Watch

On Tuesday morning Eastern Time, internet infrastructure service provider Cloudflare reported that its global network was experiencing anomalies, leading to access failures such as “internal server error” on numerous websites, including the social media platform X. Users were unable to access many websites and services, including retail, e-commerce, social media, financial services, and transportation-related platforms. The company later claimed that the issue had been resolved in less than four hours.

During the downtime, some functions of X were interrupted, and many websites also faced access issues. According to data from the outage tracking platform Downdetector, in addition to X, a large number of sites were affected, with the number of related reports continuing to rise. Users encountered Cloudflare-related error messages when accessing websites such as X, ChatGPT, DoorDash, IKEA, and the Metropolitan Transportation Authority (MTA) of New York City.

Subsequently, a female spokesperson from Cloudflare stated that around 6:20 AM Eastern Time, there was an abnormal spike in traffic for one of their services, which caused errors in the traffic passing through the company's network.

Another spokesperson for Cloudflare, Jackie Dutton, said in a statement that the issue was caused by an automatically generated configuration profile used to manage threat traffic, and the fix took less than four hours. The company stated that core remediation measures have been deployed but cautiously noted that the system “still needs time to fully stabilize.”

Dutton said:

“The number of entries in this document exceeded the expected size, triggering a software system crash responsible for handling a portion of Cloudflare's service traffic.”

The statement claims that there is no evidence to suggest that this incident is related to a cyber attack or malicious activity.

The impact of the outage is extremely widespread. Downdetector reported on the platform that during the Cloudflare outage, “reports of various affected services exceeded 2.1 million,” indicating that this incident has become one of the more serious infrastructure-level disruptions in recent years.

After the incident, Cloudflare's stock price plummeted by 7% at the opening on Tuesday, but the decline later narrowed.

The digital asset industry has also reacted. Binance co-founder and former CEO Zhao Changpeng posted on X, stating: “Blockchain kept working,” implying that the decentralized system was unaffected by this incident.

As of 12:15 PM Eastern Time, Cloudflare stated that the system is gradually recovering, but some regions around the world may still experience access errors, performance degradation, or login issues. The company will continue to update the repair progress on the status page.

Over-reliance on a few companies

In recent years, multiple issues with digital infrastructure providers have caused global internet usage to come to a standstill. Amazon Web Services (AWS), CrowdStrike Holdings Inc., and Microsoft Corporation have all experienced similar incidents, highlighting the extent to which the global internet relies on a few companies for services.

The services of Cloudflare and AWS are almost “invisible” to the average user, but their tools support a large number of websites and services that consumers use daily.

Last month, the AWS outage caused parts of the internet to grind to a halt, leaving millions of users unable to access websites and applications, disrupting retail sales, and interrupting social media and financial services, affecting many businesses as well. Last year, a vulnerability in a tool used by the cybersecurity company CrowdStrike led to widespread crashes of computer systems globally, resulting in thousands of flight delays and cancellations, and throwing the operations of government agencies and large enterprises into chaos.

Graeme Stewart, an expert from the California cybersecurity company Check Point Software, stated that such incidents highlight the internet's excessive reliance on a handful of infrastructure providers.

He said:

“Many organizations still rely all critical services on the same path, and there is no truly effective backup. Once this path encounters an issue, there is no contingency plan. This is the problem we have been observing.”

Alan Woodward, a professor of cybersecurity at the University of Surrey, stated that Tuesday's outage once again highlights the internet's heavy reliance on “a few players.” He described Cloudflare as “the biggest company you've never heard of.”

People have no choice but to rely on these few large companies.

Chief Technology Officer apologizes

Cloudflare's Chief Technology Officer Dane Knecht apologized for the incident. He wrote on X:

“When the Cloudflare network experiences issues that affect the massive traffic relying on us, we let down our customers and the entire internet. The problem itself, the impact caused, and the time taken to resolve it are all unacceptable. We have begun taking steps to ensure that similar situations do not occur again, but I know today has indeed caused trouble for everyone. Customer trust is the most important to us, and we will do everything we can to regain that trust.”

Cloudflare has experienced similar outages multiple times in the past few years.

In July 2019, a vulnerability in Cloudflare's software caused excessive consumption of computing resources in some network modules, taking thousands of websites relying on Cloudflare (including Discord, Shopify, SoundCloud, and Coinbase) offline for up to 30 minutes. In June 2022, Cloudflare experienced a failure that affected traffic to its 19 data centers, causing multiple major websites and services to go down for about an hour and a half.

Cloudflare's software is used by hundreds of thousands of companies worldwide as a buffer between enterprise websites and end users, protecting websites from traffic attacks or outages caused by traffic spikes.

Last year, a faulty software update released by cybersecurity company CrowdStrike caused millions of devices running Microsoft Windows to crash, resulting in widespread chaos across various industries such as aviation, banking, and healthcare.

The outage of CrowdStrike was caused by an error running at the lowest level on customer computers within its product. Cloudflare's role is to protect internet infrastructure, such as websites and platforms, so when Cloudflare goes down, many popular websites may become inaccessible or experience issues. Cloudflare is primarily responsible for “keeping websites always online and fast enough,” while CrowdStrike focuses on protecting computers and servers from attacks.