Overview:

The National Security Operations Centre (NSOC) is aware of the worldwide IT outage affecting Windows host consequent on a CrowdStrike Falcon Sensor update. CrowdStrike affirmed that this was not due to a security incident or cyber-attack, but rather a defect identified in a single content update specifically meant for Windows hosts.

Affected Product(s):

Microsoft Windows operating systems with CrowdStrike Falcon agent installed
 

Recommendations:

CrowdStrike has identified and isolated the problem and has issued a fix. The NSOC encourages individuals and public and private sector entities that are affected to go to the CrowdStrike Support Portal or contact the NSOC/JaCIRT for assistance with remediation. Additional information can be found on the CrowdStrike official blog here:
 

Current Action:

CrowdStrike Engineering has identified a content deployment issue contributing to these crashes and has reversed the changes. If hosts continue to experience crashes and are unable to stay online to receive Channel File Changes, follow these steps to implement a workaround:
 

Workaround Steps for Individual Hosts:

• Reboot the host to give it an opportunity to download the reverted channel file. If the host crashes again, then:
  • Boot Windows into Safe Mode or the Windows Recovery Environment
  • NOTE: Putting the host on a wired network (as opposed to Wi-Fi) and using Safe Mode with Networking can help remediation.
• Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
• Locate the file matching “C-00000291*.sys”, and delete it.
• Boot the host normally.
  Note: BitLocker-encrypted hosts may require a recovery key.

Workaround Steps for public cloud or similar environment including virtual:

Option 1:

• Detach the operating system disk volume from the impacted virtual server
• Create a snapshot or backup of the disk volume before proceeding further as a precaution against unintended changes
• Attach/mount the volume to a new virtual server
• Navigate to the %WINDIR%\\System32\drivers\CrowdStrike directory
• Locate the file matching “C-00000291*.sys”, and delete it.
• Detach the volume from the new virtual server
• Reattach the fixed volume to the impacted virtual server 

Option 2:

• Roll back to a snapshot before 0409 UTC.

Workaround Steps for Azure via serial:

1. Login to Azure console --> Go to Virtual Machines --> Select the VM
2. Upper left on console --> Click : "Connect" --> Click --> Connect --> Click "More ways to Connect" --> Click : "Serial Console"
3. Once SAC has loaded, type in 'cmd' and press enter.
   1. type in 'cmd' command
   2. type in : ch -si 1
4. Press any key (space bar). Enter Administrator credentials
5. Type the following:
   1. bcdedit /set {current} safeboot minimal
   2. bcdedit /set {current} safeboot network
6. Optional: How to confirm the boot state? Run command:
   1. wmic COMPUTERSYSTEM GET BootupState

The NSOC/JaCIRT will continue to asses the national impact of this outage with our partners and provide more information as it becomes available.

Resources:

https://www.crowdstrike.com/blog/statement-on-falcon-content-update-for… https://azure.status.microsoft/en-gb/status
https://supportportal.crowdstrike.com/s/article/Tech-Alert-Windows-cras…- Falcon-Sensor-2024-07-19
CrowdStrike Support Portal to open a case: https://supportportal.crowdstrike.com