On July 19, 2024 at 04:09 UTC (12:09am EDT) Crowdstrike, a cybersecurity company, released a patch to their Falcon Sensor product that caused Windows based computers and servers to throw the dreaded Blue Screen of Death (BSOD) making them unusable. This caused a worldwide computer disruption for airlines, hospitals, media companies, and other businesses worldwide.
What went wrong?
An update to CrowdStrike’s Falcon Sensor software designed for Windows machines triggered Blue Screen of Death (BSOD) errors, essentially crashing countless Windows systems.
Thankfully, CrowdStrike identified the problem quickly and deployed a fix. The more pressing problem is that in many cases the fix requires a technician to be in front of each computer needing the fix to be applied. While the situation caused significant disruption, it’s important to note this wasn’t a security breach or cyberattack, it appears to be lack of code testing and a distribution system that can’t automatically roll itself back if things go badly.
Here’s a quick recap:
Cause:
Faulty CrowdStrike Falcon update for Windows.
Impact:
BSOD errors causing widespread outages for businesses using Microsoft Azure and Windows systems. Slow recovery as there isn’t an automatic rollback solution in many cases.
Resolution:
CrowdStrike identified and resolved the issue with a fix. Technicians applying the fix as soon as practicable.
Lessons Learned:
Even reputable security software can have bugs. This incident highlights the importance of rigorous testing before deploying updates, especially those impacting critical systems. Companies who manufacture software need to be held accountable for distributing software that has not been rigorously tested.
For businesses, it serves as a reminder that robust disaster recovery plans need to be in place to minimize downtime during outages.
The Aftermath:
Airlines worldwide have lost check-in and booking services. Banks have been affected as they cannot receive payments. News stations in parts of the world went offline. Hospitals could not set appointments which affected critical care. 911 outages were also experienced. Retail outlets could not process payments or take customer orders. The tech world is still assessing the full impact of the outage. We can expect investigations and discussions focused preventing similar incidents in the future. Supply chain procedures need to be vetted more closely and companies held accountable for their actions.
This is a great time to review your incident response plans, disaster recovery plans, and backup strategies.
Contact us today for a consultation about your BDR and overall IT strategy. Reach us online or email us at cybersecurity@centriworks.com to get started.
— Todd Sheppard
CIO / Centriworks
CISSP, ITIL, GSEC, NET+, CDIA+
AI vs. YOU: Vishing, Smishing & Deep Fakes
Wed. August 28, 2024 11am EST
AI has become a favorite tool of cybercriminals. Join us for this webinar where you’ll hear valuable tips from cybersecurity industry experts about how to protect your data, your privacy, and your finances from AI-fueled attacks. We’ll look ahead into the future and focus on real-world threats happening now — don’t miss it!
Check out our
DATA DICTIONARY
video series!
We’re ready to help you work smarter.
Call us at (865) 524-1124 or use this contact form. Let us know what you’d like to know more about and one of our experts will be in touch with you soon.