Threat hunting is the process of proactively searching for dangerous or threatening activity within an IT environment. It moves beyond traditional detection technologies, as it proactively and iteratively scouts networks and other systems to detect and isolate advanced cyber threats that often evade standard security solutions.
Threat Hunting can be of many types: It can start from the weakness points identified as part of the organization’s risk assessment and unique vulnerabilities to its IT architecture. Structured Hunting, on the other hand, starts from a set of Indicators of Attack and specific tactics, techniques and procedures (TTPs) of specific Advanced Persistent Threat (APT) groups, before they can deal damage to the organization. Finally, hypothesis driven threat hunting starts with the hypothesis that the IT environment has already been compromised by an adversary. The objective is to determine how a threat actor might behave and deduce the attack TTPs that might be used. When the threat hunter has identified these TTPs, he can create detections that are automated to improve the detection capability of the organization. These can run in the background to detect real-life threats 24-7.
In this article we will explore why hypothesis driven threat hunting is crucial to the modern datacenter and cover a number of proven threat hunting strategies that you can use. We will also describe the threat hunting steps necessary for effective detection and cover some common pitfalls that you should avoid. For your convenience, please find an executive summary of this article below.
Threat hunting is a technique designed to augment automated cyber defense solutions, such as EDRs, NDRs, and other detection and prevention tools. It does so with the use of industry professionals who can counter threats that can slip past traditional defenses.
Contrary to what one might think, threat hunters are not uninterested in cyber defense automation tools. In fact, one of their roles is to identify how to improve and augment the automation tools, preventing hackers from exploiting gaps and weaknesses. Threat hunters do not replace automated threat detection processes. Instead, they identify and respond to threats that might already be active within the infrastructure.
Threat hunters must be vigilant, up to date, and technically proficient in order to stay ahead of hackers, who are always developing new tools and techniques. Threat actors may spend months or years searching for vulnerabilities in an organization’s systems that they have identified as financially or politically lucrative.
According to a survey published in VMware’s Carbon Black Global Threat Report (2020), 88% of CISOs and IT Managers believe that threat hunting is rapidly becoming essential and are including it as part of their cybersecurity strategy. Another 36% stated that they had found “significant evidence of malicious activity” thanks to the threat hunting programs they had implemented.
Moreover, Statista estimates the average down-time after a major cyberattack (such as a ransomware attack) to be around 3 weeks. This causes immense disruption to business continuity and operations. These facts clearly show that threat hunting can be a valuable addition to an organization’s cyberdefense strategy.
In order to implement a threat hunting program, it’s important to first perform an assessment via a threat hunting Maturity Model, such as the one provided by SANS. A model like this helps an organization assess its current quality and amount of data collected that can impact threat hunting capabilities and identify areas for improvement.
Threat Hunting Maturity Model (SANS Institute – sans.org)
Here are the typical requirements to achieve each of the maturity levels, providing organizations with an idea of their current standing.
| Maturity Level | Requirements |
|---|---|
| Initial | – Putting in place automated security alerts (SIEM tools, IDS, etc.) – Developing a dedicated team of incident response professionals |
| Minimal | – Collecting and centralizing logs from different domains within the environment (network, database, applications, etc.) – Using threat intelligence at the forefront of detection – Leveraging Indicators of Compromise (IoCs) from published reports and authorities. |
| Procedural | – Following published procedures in hunting for novel security incidents – Implementing a regular hunting schedule (weekly, monthly, etc.) – Having specialized hunters within the SOC team |
| Innovative | – Using a wide variety of data analysis frameworks and applying them in identifying suspicious activity – Developing or publishing original hunting procedures based on their experience within your environment – Using a dedicated threat hunting platform to streamline the process and strengthen cooperation within the team |
| Leading | – Using Data Science, Machine Learning, and Artificial Intelligence techniques within threat hunting procedures – Leveraging automated tools to get the output of threat hunting activities to improve alerts and detection efforts – Implementing a methodology to scale procedures so that they are applicable to a large scope within the organization. |
Hypothesis threat hunting breaks down into the following four steps:
A centralized platform to compile alerts and logs is critical to collect and process the required information. Threat hunting demands detailed data extracted from verbose logs, allowing for more concise and targeted analysis. The quality of data collection will allow the organization to formulate hypotheses and search historically for occurrences of impact.
This is a fundamental assumption that needs to be either confirmed or denied. It is a possible threat, weakness, or attack vector that we can examine and test for. For example, the assumption that a threat actor is going to use a specific technique, or exploit a particular vulnerability to damage resources or exfiltrate sensitive data. The hypothesis can be formulated after an analysis of the data collected in the previous step.
This is one of the steps where the threat hunter shows their expertise: by analyzing the behavior of a suspicious file, or by checking for Tactics, Techniques, and Procedures (TTPs) using logs and other data. Determining whether a threat or malicious activity is in progress or evaded defenses.
As threats are found by threat hunters and TTPs or IOCs are identified, they determine how to automate the detection. This automation can take the form of the development of a specific YARA rule within the existing Endpoint Detection and Response (EDR) system, or a new playbook within a SIEM.
These steps include not only escalation responses, but also the actions required to develop long term threat detection capability. Hard earned detection content can be used to fuel existing security controls and be utilized by traditional security analysts to watch for similar TTPs and behaviors in the future. In this way, threat hunting teams are not wasting time and resources by continuously conducting the same actions.
Many factors can result in a threat hunting program that is difficult to implement. One key factor is a lack of visibility often caused by lacking log centralization and threat detection tools. Good threat hunting requires full visibility and an advanced set of tools and techniques that must be built on a solid foundation of security measures, controls and tools. The absence of automated detection and response capabilities can also seriously hinder the efficiency and effectiveness of a threat hunting program. Finally, a lack of data collection capabilities can also reduce the efficiency of a Threat Hunting program, where a hypothesis is stated but no data can be collected to confirm or deny it.
Unfortunately, some organizations merely use threat hunting as a means to tick a box with regards to their security standards requirements. As a result they often hand over threat hunting responsibility to an existing, non-specialist team who usually have other workloads. Instead of setting a long-term strategy and regularly assessing the effectiveness of the program, the whole process becomes simply a box-ticking exercise.
Although this can occasionally lead to results, it is far from ideal for organizations attempt to improve their maturity and is unlikely to be as effective as a dedicated threat hunting team running as part of a wider and well managed security program.
As we have seen, threat hunting is most successful when placed within a security ecosystem where people, processes, and technology are already in place.
Threat hunting is critical in improving the security posture and safety of any organization. It is therefore vital that it is given the resources it really needs. This includes a dedicated threat hunting team, modern and appropriate tools, such as SIEM, EDR/NDR and automation tools (amongst others), as well as the funding and support from the key stakeholders. Threat hunting, similar to many other security disciplines, is an important ongoing exercise that requires regular input, support and appropriate reassessment when required.
