Malware analysis

Malware analysis is the study or process of determining the functionality, origin and potential impact of a given malware sample such as a virus, worm, trojan horse, rootkit, or backdoor. Malware or malicious software is any computer software intended to harm the host operating system or to steal sensitive data from users, organizations or companies. Malware may include software that gathers user information without permission.

Use cases

There are three typical use cases that drive the need for malware analysis:

• Computer security incident management: If an organization discovers or suspects that some malware may have gotten into its systems, a response team may wish to perform malware analysis on any potential samples that are discovered during the investigation process to determine if they are malware and, if so, what impact that malware might have on the systems within the target organizations' environment.
• Malware research: Academic or industry malware researchers may perform malware analysis simply to understand how malware behaves and the latest techniques used in its construction.
• Indicator of compromise extraction: Vendors of software products and solutions may perform bulk malware analysis in order to determine potential new indicators of compromise; this information may then feed the security product or solution to help organizations better defend themselves against attack by malware.

Types

The method by which malware analysis is performed typically falls under one of two types:

• Static malware analysis: Static or Code Analysis is usually performed by dissecting the different resources of the binary file without executing it and studying each component. The binary file can also be disassembled (or reverse engineered) using a disassembler such as IDA or Ghidra. The machine code can sometimes be translated into assembly code which can be read and understood by humans: the malware analyst can then read the assembly as it is correlated with specific functions and actions inside the program, then make sense of the assembly instructions and have a better visualization of what the program is doing and how it was originally designed. Viewing the assembly allows the malware analyst/reverse engineer to get a better understanding of what is supposed to happen versus what is really happening and start to map out hidden actions or unintended functionality. Some modern malware is authored using evasive techniques to defeat this type of analysis, for example by embedding syntactic code errors that will confuse disassemblers but that will still function during actual execution.
• Dynamic malware analysis: Dynamic or Behavioral analysis is performed by observing the behavior of the malware while it is actually running on a host system. This form of analysis is often performed in a sandbox environment to prevent the malware from actually infecting production systems; many such sandboxes are virtual systems that can easily be rolled back to a clean state after the analysis is complete. The malware may also be debugged while running using a debugger such as GDB or WinDbg to watch the behavior and effects on the host system of the malware step by step while its instructions are being processed. Modern malware can exhibit a wide variety of evasive techniques designed to defeat dynamic analysis including testing for virtual environments or active debuggers, delaying execution of malicious payloads, or requiring some form of interactive user input.
• Hybrid malware analysis: Hybrid analysis integrates static and dynamic techniques, often combined with memory forensics, to provide a more comprehensive understanding of malware behavior. This approach is used in community and commercial sandbox systems such as implemented on Hybrid Analysis.

Stages

Examining malicious software involves several stages, including, but not limited to the following:

• Manual Code Reversing
• Interactive Behavior Analysis
• Static Properties Analysis
• Fully-Automated Analysis

Standardized evaluation of sandbox-based analysis products has also emerged. In 2025, the Anti-Malware Testing Standards Organization (AMTSO) introduced the first Sandbox Evaluation Framework, aimed at providing consistent, use-case-driven testing criteria for sandbox malware analysis tools.

Artificial Intelligence vs. Humans

The use of artificial intelligence (AI) in malware detection has been an active area of research within the field of cybersecurity. One study compared the behavior of AI against human behavior to understand the similarities and differences in how these two parties performed. The results included:

• AI and humans have similar detection rates
• AI focused more on static analysis
• Humans focused more on dynamic analysis
• AI does not perform well with only dynamic analysis

Antivirus softwares have used these studies to come to a conclusion that a combination of both practices would be most ideal. Since AI can use static analysis at a much faster rate than humans in malware detection, the antivirus software will send the samples to the AI for an initial screening. If the sample is more complex and the AI cannot make a definitive conclusion, then the sample will be sent to a human to observe the sample in more detail. In this way, more samples can be correctly identified as malware or benign.