The Pivotal Role of Large Language Models in Extracting Actionable TTP Attack Chains

Enhancing Modern Threat Intelligence: The Pivotal Role of Large Language Models in Extracting Actionable TTP Attack Chains

Currently, the application of LLMs within the security landscape has achieved widespread adoption, becoming a standard practice across the industry. In the realm of threat intelligence, LLMs have distinguished themselves through their exceptional capabilities in extracting IOCs and summarizing cyberattack reports, significantly enhancing the efficiency and precision of threat intelligence processing. However, concurrently, the defensive mechanisms against cyber threats have evolved to be more proactive and dynamic, transcending the traditional reliance on IOC intelligence. The focus has shifted towards deeply mining TTPs (the pinnacle of the Pyramid of Pain). For in-depth defense evaluation systems, the capture of TTP intelligence that can be simulated and executed is of utmost importance. This rapid capture and transformation of modern threat intelligence is crucial for the timely identification and response to cyberattacks, enabling organizations to stay ahead of the curve in the face of evolving security threats.

Indeed, the current process of extracting and converting the vast amount of unstructured cyberattack reports released by researchers and vendors in the cybersecurity field into executable TTP intelligence (Tactics, Techniques, and Procedures, with highly abstract) is predominantly handled by security experts. This task is extremely time-consuming and labor-intensive. In the face of this challenge, the urgent need arises for an efficient, automated, and precise method to analyze attack reports, accurately extract the TTP attack chains used by attackers, and subsequently generate executable simulation attack scripts based on these insights. After a comprehensive review of existing technologies, we have specifically focused on the approach of leveraging LLMs in conjunction with prompt engineering techniques. Through in-depth evaluation, we have discovered that while this method holds immense potential, none of the LLMs currently available on the market are ready for immediate use when it comes to extracting TTP attack chains. Even when combined with various prompt engineering techniques or enhanced with background knowledge through RAG techniques, the accuracy of their outputs falls short of satisfactory standards, indicating significant space for improvement.

We have innovatively implemented an efficient and practical method that ingeniously leverages two avenues to provide contextual support for LLMs, significantly overcoming the challenges in extracting TTP attack chains and converting them into executable formats. Specifically, we have incorporated both a pre-defined, optional set of TTPs and the extensive TTP data from existing knowledge graphs as contextual inputs for the LLMs to process. Experimental results have demonstrated a notable improvement in both accuracy and practicality.

In this presentation, we will delve into sharing our groundbreaking advancements in the field of modern threat intelligence, focusing on the three historic phases of TTP intelligence extraction: the initial exploration in the Bronze Age, the refinement in the Silver Age, culminating in the leap forward into the Golden Age. Subsequently, we will highlight the innovative combination of LLMs with other technologies – specifically, the strategies of LLM + BERT for precise re-ranking and LLM + RAG (Retrieval-Augmented Generation with TTP knowledge graphs) – to achieve rapid capture and transformation capabilities for modern threat intelligence.

By:
Lorin Wu | Senior Security Researcher, 360 Digital Security Group
Porot Mo | Senior Security Researcher, 360 Digital Security Group
Jack Tang | Senior Security Researcher, 360 Digital Security Group

Full Abstract and Presentation Materials:
https://www.blackhat.com/asia-25/briefings/schedule/#enhancing-modern-threat-intelligence-the-pivotal-role-of-large-language-models-in-extracting-actionable-ttp-attack-chains-43631Black HatRead More