I got into a very insightful debate with somebody who will remain nameless in the beginning of this post, but will perhaps be revealed later. The debate focused on the role of context in threat detection.
Specifically, it is about the role of local context (environment knowledge, organization context, site details, etc) in threat detection. Can threat detection work well without such local context?
Now, some of you will say “yes, of course!” and will point at “success” (well, let’s not get into a fight over this) of anti-malware technology. After all, anti-malware tools promise to detect malware using vendor-created signatures that operate without any input from the customer about their environment (as a minor sidenote, if you “tune” AV then you do introduce that very local context). Note that for this discussion it does not matter that anti-malware will detect and then block (“prevent”) the threat (in other discussions, it definitely does).
The same line of thinking then affected intrusion detection as it was developing in the late 1990s. Intrusion detection systems (IDS) that had lots of signatures and so could detect something out of the box were “successful” (at least as a business) while those that expected customers to write signatures failed or had to evolve.
Then it was SIEM’s turn: SIEM vendors with lots of rules and reports were more successful (and now we have SOC Prime with lots of community rules). Next, it was security analytics tool sets with their “trained ML unicorns”: those with lots of pre-tuned algorithms seemed to be selling better. See the pattern yet? It seems like you can be successful with threat detection without any input from each specific client.
Now, let’s pause and think for a second! What if the industry was … well … if not wrong, but also not entirely right. What if truly successful threat detection must be a collaboration between the vendor and the customer?
In fact, it is easy to find examples of where canned and context-less threat detection does not work all that well. For this, let’s review how successful the detection technologies really are in regards to their use of local context data.
- Anti-malware mostly works (when it does) yet the ransomware epidemic continues and top-tier state-sponsored/-affiliated malware is almost never detected by traditional anti-malware tools. Along the same line, many initial loaders (that you may call “commodity”) aren’t well detected either, and it’s easier to obtain access to these as malicious tools than ever before. Finally, when used in large enterprises, AV is often tuned hence this local knowledge is in fact introduced.
- Network IDS and related technologies (like NDR) don’t really work or don’t work well without local context; at the very least, you will need to “tune” (i.e. add local context like “ignore this server, it always triggers that in legitimate traffic”). Untuned NIDS has long been a subject of many jokes, dating back to the 1990s, if not the 1980s.
- SIEM mostly does not work without a lot of local context, vendor-written SIEM rules never became “shoot and forget”, and you need to tweak them based on your environment and/or write your own rules. This is accepted by most sane SIEM vendors and customers.
- EDR would be a mixed bag, in this regard. Many EDR rules are naive pattern matching. Take a powershell execution with specific command line parameters. A rule may be tuned from 22,000 results all the way down to 17 because (say) PowerShell gets executed in a “suspicious” way all the time and local context (whitelist for system, process, application, etc) is needed. With ML-based EDR, the situation is … as far as I see… the same. Anomalies detected need local context to mean something.
(note that for attackers armed with “living off the land” techniques, the balance skews even further towards local context criticality for detection)
So, what can we learn from this? Threat detection today needs local context a lot more than people realize. Now, successful threat detection programs at elite enterprises, especially those that follow the “detection engineering” model all know this (this is why most/all of their detection logic is custom or customized, not OOB). But are they a rare exception rather than a trend?
And what does it mean for others? Well, you can hire “help” which here means an MSSP or an MDR (BTW, MDR label was born out of frustration with some MSSP threat detection offerings, so YMMV). However, please don’t automatically assume that “using an MSSP means that your local realities will be included in the detection process.” They will be — with quality MDRs and MSSPs, but you may also get canned off-the-shelf SIEM or even IDS alerts from some providers. You may need a combination of tools, services and — yes, still! — your own efforts.
Finally, this is where an ML unicorn will again emerge out of the bushes (or wherever they live…) and say “but we can just auto-learn local realities using my little machine brain.” And, presumably, “auto-learn” here will not mean “import from customer repository” (because many organizations simply lack such a thing, like they lack a current and correct list of assets). Well, can it happen? Sure, it can. In theory. Personally, it is easy for me to believe that it can happen, but I will also be the first to admit that I’ve never actually seen it happen … yet.
So, to summarize, we all need to think ….
- How well does threat detection really work without local context?
- How to best include local context in various detection tools and practices?
- How to select the vendor who will detect WITH you?
- How to practice detection jointly with the vendor or service provider rather than merely “consume” it?
P.S. Huge thanks to Brandon Levene for an idea for this post, for some of the examples and for a great discussion that almost became an argument :-)
P.P.S. I think this situation does not really change in the cloud; you need local cloud context to detect.
Originally posted at "Anton on Security"
Comments