“CrowdStrike, Broken Patch, or Cyber Attack?”
Photo by Soumil K umar Creative common license at pexels.com Person Typing on Computer Keyboard
By Scott Hamilton
Last week as I sat down to write my article I had the debate on whether to write about the major computer system outage that occurred last Friday, July 19, 2024. This was one of the most impacting computing system outages in recent memory. The last time I remember such an impact was from a bug that slipped into the openSSL libraries which were used for internet traffic encryption. This was a legitimate bug that slipped through the cracks of nearly every computer technician in the world, until a single hacker noticed something strange. You see, with this one, some new open source developer working on the project decided that it would be a good idea to clean up some of the many compiler warnings that happened when building the encryption library. One of the warnings he cleaned was critical to the function of the library. The warning was that the code was reading unallocated memory. Unallocated memory has random data within, but if you allocate memory it clears the memory to all zeros. In fixing this warning, he accidentally made every newly generated security key identical, because they were relying on the random memory read to generate the keys.
The openSSL library bug did not really break anything for a long time; it only caused problems on newly installed servers and websites. Anything that was already running was not impacted. The problem that occurred Friday took down systems at a global level. However, it left me questioning whether or not it was a real mistake by CrowdStrike, or a cover up of something more sinister. I plan to share my reasons and let you make a decision for yourself.
Did CrowdStrike release a bad patch causing a global outage, grounding flights, freezing electronic bank transactions, locking remote workers out of corporate networks and taking down major internet services? Or did someone wage cyber-warfare against the world?
I have a strong argument to suggest it was an actual attack and CrowdStrike is being used as a scapegoat to hide the real story. The first argument comes from knowing a lot about how major corporations and federal governments handle security patches. There is a method that is commonly referred to as an n-2 patch cycle used by 95 percent of all major corporate information technology (IT) departments. To put it in very easy to understand terms, it basically means that any security patch is tested at least twice before it is applied to a production computing system. This even goes for anti-virus software updates, and CrowdStrike software would definitely have been included in this n-2 process.
Here is how the process works. Every IT department has three stages of patch deployment. They have a set of servers, usually referred to as development servers, that receive the patch before any other systems. These serve the primary purpose of letting the IT department know what to expect when the patch is applied to the general systems. These systems are patched the day a patch is released (n-2). There is a time delay before the patch gets applied to the second set of servers, usually referred to as test systems, which are patched about a week after the patch release (n-1). These test systems are also an exact mirror of the production servers that face the general public. The patches are never applied to production systems until full tests have been completed on the test platform, about two weeks following the patch release (n). So as you can see, there are very solid checks and balances in place to prevent a patch from breaking production systems. So how did the CrowdStrike get past the n-2 deployment mechanism in place and take down so many important systems?
I argue that it was not a patch that took down these critical systems, but rather a cyber-attack against CrowdStrike, which has been cleverly disguised as a patch gone wrong. I have to admire CrowdStrike because of their rapid response to the incident, especially considering that they are also taking the blame for the mess. CrowdStrike teams worked around the clock to create automation scripts to repair the damage caused, and quickly posted manual instructions for repair within hours of the incident. I am not sure why CrowdStrike would agree to take the fall, being one of the top Cybersecurity companies in the world, other than to hide the fact that they were also impacted, and probably impacted first, giving them time to determine a fix before the rest of the world fell subject to the attack. If I had to guess, I would say CrowdStrike was likely warned of the security vulnerability days before the attack by the hacker group that performed it. There are hackers out there just trying to protect the public by exposing flaws in the top security companies’ systems, and those ethical hackers usually tell the target about the issue in advance, as well as sharing how to prevent it. They are in hopes that the companies do the right thing, but are always prepared to expose them by taking down the systems if they fail to make the vulnerability publicly known.
There is a second reason I lean toward cyber-attack and it has to do with the fact that the Chinese government bans the use of Microsoft products and the issue only impacted Microsoft systems, meaning that this global impact excluded China. It certainly gives you something to think about. Until next week, stay safe and learn something new.
Scott Hamilton is an Expert in Emerging Technologies at ATOS and can be reached with questions and comments via email to sh*******@te**********.org or through his website at https://www.techshepherd.org.
Please help support my site by purchasing related products from my Amazon Affliate Store: This weeks featured products are related to computer security:
