Windows Internals for Malware Analysis


User mode malware on Windows is ubiquitous and more are being found in the wild everyday. Malware analysts, reverse engineers, incident responders and forensics investigators take on the daunting task of hunting down compromised systems, identifying IOCs and taking apart malware. The one common theme amongst all Windows malware is that they abuse the Windows internals and APIs to perform nefarious tasks. Malware analysis requires a strong understanding of Windows internals, especially considering the rapid changes it is undergoing due to Microsoft's fast paced Windows servicing model.

This is not a typical malware analysis/reversing engineering course. Instead, this course covers Windows internals from the perspective of malware analysis and forensic investigations. Through a practical hands-on approach, attendees learn how malware leverages components, architecture, functionality, APIs and data structurers of the Windows operating system.

Every section is accompanied by instructor led demos and hands-on labs. These labs illustrate how malware subverts, abuses and exploits various subsystems of Windows OS to achieve its goals. Students study carefully selected samples which illustrate various phases of malware execution using tools like WinDBG, SysInternals, x64_DBG etc. All labs are performed on the 64-bit version of Windows 10. Students will receive the source code for all the labs amounting to thousands of lines we well documented C/C++, C#/.NET and PowerShell code.

Target Audience

Malware analysts, forensic investigators, incident responders, security researchers, system administrators. Anyone interested in understanding how modern malware works on Windows and is responsible for detecting, analyzing and defending against malware and other post-exploitation techniques.


Attendees must have a solid understanding of operating system concepts and have a working knowledge of Windows. This hands-on labs for this course do NOT involve any programming exercises. Prior experience with Malware analysis is desirable. Familiarity with Win32 API is desirable but not required.

Learning Objectives



System Mechanisms

This section introduces the course, hands-on labs, analysis tools and dives into the mechanisms available in modern Windows systems that are relevant to malware operations.

  • Tools Overview
  • System Architecture
  • User and Kernel Mode Execution
  • Processes, Threads & Jobs
  • System Calls and Native APIs
  • Window Messages
  • Sessions and Session Isolation
  • Malware Execution Stages

Processes and Threads

This section covers the details of various type of process in Windows, key process and thread data structures, methods of subverting thread execution and how process mitigation policies are used to reduce attack surface.

  • Normal, Minimal, Pico Processes
  • Protected Processes
  • App Container Processes
  • Services and Service Hosts
  • PEB & TEB
  • User Mode APCs
  • Thread Register Contexts
  • Process Mitigation Policies

Memory Management

The Windows virtual memory subsystem is leveraged by malware to inject and execute code inside processes. The section covers Windows memory management with emphasis on Malware code injection techniques such as reflective DLL injection, process hollowing, process doppelganging etc. and identifying malware artifacts in memory.

  • Process Address Space
  • Heaps and CRT Memory
  • Memory Protection
  • Thread Stacks
  • VM Read Write Operations
  • Process Injection Techniques
  • DEP & ASLR
  • Arbitrary Code Guard (AGC)

PE Files

Analyzing malware requires a good understanding of the Windows PE file (COFF) format and the functionality provided by the NTDLL loader to map PE files into memory. This section looks at 64-bit PE files from a malware perspective - packing, obfuscation, code caves, import table hashing, debugger subversion etc.

  • Loader functionality
  • PE Sections
  • Relocations
  • Address of Entry Point (AoEP)
  • TLS Callbacks
  • Import and Export Tables
  • Loader data structures
  • Control Flow Guard (CFG)


The Windows security subsystem controls access to various objects such as processes, threads, registry keys, files and directories that are common targets of malware. This section discusses the security subsystem including topics such as restricted tokens, impersonation, UAC Bypass, privilege escalation, non-admin abuse etc.

  • SIDs & Tokens
  • Privileges
  • Security Descriptors
  • DACLs & SACLs
  • Objects and Handles
  • Access Checks
  • UAC
  • Integrity Levels

Persistence and Auto-Start

To establish a permanent foothold on a system malware must not only make itself persistent but also hook into system auto-start vectors to regain execution. This section discusses the various mechanisms available in a Windows system for malware to achieve these objectives.

  • System ASEPs
  • Links and Shortcuts
  • DLL hijacking
  • Image hijacking
  • COM Object hijacking
  • Task Scheduler
  • PE binary trojaning
  • Autoruns blind spots

Script Hosts

Modern malware attempts to live of the land to stay under the radar of A/V products. This requires malware to leverage existing signed binaries on the system to achieve its goals. This section covers the various scripting environments available in Windows and how these are used my malware to circumvent writing PE files to disk.

  • File less Execution
  • Office Macros
  • PowerShell Malware
  • .NET Interfacing
  • WMI Persistence
  • Windows Remoting (WinRM)
  • Windows Scripting Hosts
  • Anti-Malware Scan Interface (AMSI)

Malware Self-Defense

This section covers user mode debuggers, virtualization software and endpoint security products, malware analysis sandboxes and how malware defends itself from them by employing anti-debugging / analysis / reversing techniques.

  • User mode debugging internals
  • Debugger artifacts
  • Anti-Debugging Techniques
  • Virtual Machine Architecture
  • VM Artifacts
  • Endpoint Security Product Architecture
  • PSP and A/V Evasion Techniques
  • Malware Analysis Sandboxes