How a Content Update Shook the Industry and Potentially Reshaped Vendor Relationships
The recent IT outage caused by CRWD’s content update for Falcon has left a significant mark on the industry, affecting countless customers and raising questions about the reliability of such updates. This article delves into customer reactions, the recovery process, and the potential impact on CRWD and other vendors in the near and mid-term.
Customer Reactions and Recovery Challenges
The global IT outage has caused widespread disruption among CRWD’s customers. Conversations with numerous partners and an expert call reveal that many customers are still grappling with the aftermath. The primary hurdles in the recovery process include a lack of on-site IT support, inability to physically access devices, and difficulties obtaining encryption keys.
The absence of on-site IT personnel has been a major obstacle, as implementing CRWD’s workaround and subsequent update often requires physical device access. Devices with drive encryption faced additional issues, further slowing down the recovery process. Companies with robust backup and recovery procedures have fared better, but those with poor key management practices are still struggling.
Virtual servers and Virtual Desktop Infrastructures (VDIs) have been easier to recover due to their remote accessibility. CRWD has been proactive in reaching out to its customer base, offering assistance in the recovery of their IT systems.
Quantifying the Impact on CRWD
Estimating the precise impact on CRWD is challenging. Partners indicate that many deals are currently in a freeze or hold mode. Anecdotes suggest potential discounts as high as 40%, with nearly half of the deals in the pipeline for this quarter possibly being deferred. Our preliminary analysis suggests that while FQ2’25E Street estimates might remain safe, the FQ3’25E guide could be at risk. Consequently, CRWD might need to revise down its FY25E guide. We maintain our OUTPERFORM rating but lower our price target (PT) from $385 to $315 to account for the potential downside risk to Street numbers and guidance.
Root Cause Analysis and QA Failures
A pressing question among customers and investors is how CRWD could roll out an update without thorough testing. The official Root-Cause Analysis from CRWD is awaited, but experts theorize that the issue likely stemmed from skipped QA/testing or testing on Windows machines with different updates/patches than those used by most customers. Some Windows machines at certain patch/update levels were not impacted, suggesting that CRWD’s test machines might have been running different updates/patches.
Agent-Based Architecture and Vendor Liability
The incident has also raised questions about the reliability of CRWD’s “single-agent” architecture. While some believe this architecture might be to blame, experts argue that “agentless” products have their own weaknesses. Agentless products rely on APIs for communication, which can be disrupted, leaving workloads unprotected.
The responsibility of Microsoft (MSFT) in this incident has also been questioned. Partners suggest that MSFT might be forced to allow third-party companies like CRWD to update its Operating System as part of an Anti-Trust agreement with EU regulators. If true, this would absolve MSFT of responsibility for the outage.
Auto-Updates and Customer Behavior
The automatic application of the update to machines with “Auto-Update” enabled has sparked discussions about the need to test updates before applying them. While testing could prevent such disruptions, disabling “Auto-Update” could leave machines exposed to “zero-day” risks. Therefore, maintaining the “Auto-Update” feature is essential for timely protection against new vulnerabilities.
Despite the incident not being a security breach, it has disrupted business operations akin to a “Denial of Service” attack. Partners report that deal activities are mostly on hold, and CRWD may need to offer significant discounts to affected customers. However, quantifying these concessions is difficult due to limited activity in the immediate aftermath.
Lessons Learned and Future Considerations
This incident underscores the importance of robust Disaster Recovery processes. The disruption has affected various sectors globally, raising questions about the wisdom of consolidating IT environments on a limited number of vendors. Customers might now consider “dual-sourcing” or “multi-sourcing” to mitigate the risk of such disruptions.
Potential Beneficiaries
While CRWD deals with the fallout, other vendors might benefit. SentinelOne (S) and Palo Alto Networks (PANW) are potential alternatives for customers seeking next-gen endpoint security. However, selecting a new security vendor is a lengthy process, and immediate gains for other vendors are unlikely. The incident has also increased phishing attacks, potentially boosting demand for Email Security and Phishing Security solutions from companies like Abnormal Security, Proofpoint, and Mimecast. Backup and Recovery solutions such as Rubrik (RBRK) might also see increased interest.
The CRWD IT outage has had a profound impact on its customers and the broader industry. While CRWD is working diligently to assist customers and recover from the disruption, the incident raises critical questions about update testing, vendor reliability, and future IT strategies. The full extent of the impact on CRWD and potential benefits for other vendors will unfold in the coming months.
You might like this article:Breakthrough in Hemophilia A Treatment: Pfizer’s Gene Therapy Shows Promising Results
