Add Category
In short:
The global IT outage experienced on Friday came down to a single software update.
The update to the Falcon sensor program, run by US company CrowdStrike, caused a coding error that sent millions of Windows computers worldwide to a "Blue Screen of Death".
What's next?
The company says they fixed the issue within hours of it arising but some users are still experiencing issues with their devices.
Airports were left in chaos, supermarket check-outs started malfunctioning and journalists scrambled without the basic tools of the trade to report on an issue causing havoc worldwide.
One company and one tiny software update are at the centre of a global IT outage that engulfed millions of people, businesses and organisations on Friday.
The CrowdStrike outage is mostly resolved, but what actually caused the fault inside millions of the world's computers and devices?
A regular system update
CrowdStrike is a US cybersecurity company based in Texas that offers ransomware, malware and internet security products almost exclusively to businesses and large organisations.
On Friday, July 19 at 4:09am UTC (2:09pm AEST), they released a sensor configuration update on their Falcon program that targeted Windows systems, according to a statement published on Saturday on the company's blog.
Falcon sensor is a cybersecurity program that provides partially automated protection from malware, antivirus support, incident response and other security features.
The program is cloud-based, meaning it operates in conjunction with CrowdStrike's servers, without needing customers to install and manage extra equipment or software.
CrowdStrike said these types of updates to the Falcon program happen multiple times a day and have done so since the program's launch.
'Logic error' occurs
The company said the update, designed to target malicious system communication tools in cyber attacks, triggered a "logic error" that resulted in an operating system crash on Windows systems (Mac and Linux users were not affected).
A logic error means a mistake has been made in coding, and has caused a bug that can then cause the program to malfunction.
Ajay Unni, the chief executive of StickmanCyber, told the ABC the update was designed as a patch, meaning it was meant to improve the program rather than cause issues.
"So that patch didn't go as planned and that's what caused the outage."
Millions of Windows PC users reported seeing a "Blue Screen of Death" on their devices, with many computers going into a reboot loop.
CrowdStrike said anyone using Falcon on Windows version 7.11 or above may have been affected.
"The file is called a channel file, which needs to be deleted," Mr Unni said.
"If the systems are online, it can be deleted remotely … if the system is offline, we'll have to get on a phone call with your IT support," he said.
Not all users were able to delete the buggy file remotely, and some needed to delete it manually for the device to work again.
Channel File 291 was the impacted file, according to CrowdStrike.
Is the issue resolved?
CrowdStrike responded within an hour of the reported issue, saying it was aware and working on a fix.
By Friday 05:27am UTC (3:27pm AEST), CrowdStrike said it pushed out an update to replace the flawed configuration files.
Most users in Australia noticed the issue around 3pm, however, with many unable to get their devices working again for hours after the crash.
The company stated it was doing a "root cause analysis" on the problem.
The outage was crucially not the result of a cyber attack, CrowdStrike said.
Mark Jones, a senior partner and cyber expert at Tesserent, said the rollback of the configuration update seemed to be working, though deploying through entire systems like servers and multiple desktops would take hours.
"Depending on the environment, there might be potential issues that roll on from this," he said.