Search This Blog

Powered by Blogger.

Blog Archive

Labels

Showing posts with label Azure outage. Show all posts

Recent IT Meltdown: CrowdStrike Update Causes Global Chaos, Predicted Hours Earlier on Reddit

 

Only a few times in history has a single piece of code instantly wreaked havoc on computer systems globally. Examples include the Slammer worm of 2003, Russia’s NotPetya cyberattack targeting Ukraine, and North Korea’s WannaCry ransomware. However, the recent digital catastrophe over the past 12 hours wasn't caused by hackers, but by the software meant to protect against them.

Two major internet infrastructure issues converged on Friday, causing widespread disruptions across airports, train systems, banks, healthcare organizations, hotels, and television stations. The trouble began on Thursday night with a widespread outage on Microsoft's cloud platform, Azure. By Friday morning, things worsened when CrowdStrike released a flawed software update, causing Windows computers to reboot repeatedly. Microsoft stated that the two failures are unrelated.

The cause of one disaster was identified: a faulty update to CrowdStrike’s Falcon monitoring product. This antivirus platform, which requires deep system access, aims to detect malware and suspicious activity. However, the update inadvertently caused the system to crash. Mikko Hyppönen of WithSecure noted that this is unprecedented in its global impact, although similar issues were more common in the past due to worms or trojans.

CrowdStrike CEO George Kurtz explained that the problem was due to a defect in the code released for Windows, leaving Mac and Linux systems unaffected. A fix has been deployed, and Kurtz apologized for the disruption. CrowdStrike’s blog revealed that the crash was caused by a configuration file update aimed at improving Falcon’s malware detection capabilities, which triggered a logic error leading to system crashes.

Security analysts initially believed the issue was due to a kernel driver update, as the file causing the crash ended in .sys, the extension for kernel drivers. Despite CrowdStrike clarifying that it wasn’t a kernel driver, the file altered the driver’s functionality, causing the crash. Matthieu Suiche of Magnet Forensics compared the risk of running security software at the kernel level to “open-heart surgery.”

Microsoft requires approval for kernel driver updates but not for configuration files. CrowdStrike is not the first to cause such crashes; similar issues have occurred with updates from Kaspersky and Windows Defender. CrowdStrike’s global market share likely contributed to the widespread impact, potentially causing a chain reaction across web infrastructure.

The outages had severe consequences worldwide. In the UK, Israel, and Germany, healthcare services and hospitals faced disruptions, while emergency services in the US experienced issues with 911 lines. TV stations, including Sky News in the UK, had to stop live broadcasts. Air travel was significantly affected, with airports using handwritten boarding passes and airlines grounding flights temporarily.

The incident highlights the fragility and interconnectedness of global digital infrastructure. Security practitioners have long anticipated such vulnerabilities. Ciaran Martin of the University of Oxford noted the event’s powerful illustration of global digital vulnerabilities.

The update’s extensive impact puzzled experts. CrowdStrike’s significant market share suggests the update triggered crashes in various parts of the web infrastructure. Hyppönen speculated that human error might have played a role in the update process.

As system administrators work to fix the issue, the larger question of preventing similar crises looms. Jake Williams of Hunter Strategy suggested that CrowdStrike’s incident might prompt demands for changes in how updates are managed, emphasizing the unsustainability of pushing updates without IT intervention.

Redditor Predicted CrowdStrike Outage Hours Before Global IT Chaos

A Reddit user, u/King_Kunta_, predicted vulnerabilities in CrowdStrike's systems just hours before the company caused a massive global IT outage. The user called CrowdStrike a "threat vector," suggesting it was susceptible to exploits that could lead to widespread damage. Initially, users dismissed the claims, but their tune changed dramatically after the outage occurred.

One commenter noted, "He tells us that CrowdStrike is a threat vector. A few hours later, every computer in the world with the CrowdStrike client installed goes blue screen. The single biggest global PC system collapse in history. Just uncanny."

Amidst the chaos, CrowdStrike's CEO George Kurtz reassured the public via X (formerly Twitter), stating, "Today was not a security or cyber incident. Our customers remain fully protected," and confirming that the issue was due to an update error, not a cyberattack.

Despite reassurances, many were left suspicious and impressed by the timing and accuracy of the Reddit post. One user aptly summed up the sentiment: "There’s no way the timing of this crazy post aligns so perfectly."