Memory-Resident Malware and Fileless Attack Detection

Windows Internals, Memory Forensics, EDR, and Network Detection Engineering

Introduction: The Memory-Only Battlefield

1. Why “Fileless” Is a Misleading Term
2. What This Book Teaches You to Do
3. Defensive Ethics, Safe Research, and Controlled Simulation
4. The Reference Laboratory Architecture
5. How to Use This Book

Chapter 1: Windows Internals for Detection Engineers

1. User Mode, Kernel Mode, and Detection Visibility
2. Processes, Threads, and Execution Contexts
3. Virtual Memory, Address Spaces, and Paging
4. Heaps, Stacks, and Data Layout
5. PEB, TEB, and Process Metadata
6. Handles, Objects, Tokens, and Access Rights
7. Integrity Levels, Sessions, and Services
8. The Registry and Configuration State
9. System Calls and API Layers: A Defensive View
10. Chapter Summary

Chapter 2: PE/COFF Structures and Executable Image Behavior

1. PE File Format: Headers, Sections, and Metadata
2. Imports, Exports, Relocations, and TLS Callbacks
3. The Windows Loader: How Images Become Processes
4. Dynamic Linking and DLL Search Paths
5. Signed Versus Unsigned Modules and Authenticode
6. Memory-Mapped Images and Their Normal Appearance
7. Side-Loading Indicators and Module Provenance Anomalies
8. Chapter Summary

Chapter 3: Managed Execution, Scripting, and Automation Surfaces

1. The CLR, Managed Code, and Runtime Observability
2. PowerShell Execution: Policies, Logging, and Telemetry
3. Scripting Hosts: cscript, wscript, and Suspicious Invocation
4. WMI Architecture, Event Subscriptions, and Abuse Indicators
5. COM Objects, Activation Patterns, and Detection Signals
6. RPC, Named Pipes, and Inter-Process Communication Artifacts
7. Chapter Summary

Chapter 4: Windows Telemetry Infrastructure

1. Event Tracing for Windows (ETW): Providers, Sessions, and Events
2. The Windows Event Log: Channels, Sources, and Reliability
3. PowerShell Script Block Logging and Module Logging
4. AMSI: Architecture, Integration Points, and Limitations
5. Microsoft Defender Antivirus Telemetry and Interfaces
6. Sysmon-Style Extended Telemetry and Custom Providers
7. Telemetry Gaps, Performance Costs, and Baseline Behavior
8. Chapter Summary

Chapter 5: The Memory-Resident Threat Spectrum

1. A Working Taxonomy: From Scripted Attacks to Pure Memory Implants
2. Living-off-the-Land: Abuse of Legitimate Binaries
3. Suspicious Process Creation and Parent-Child Relationships
4. Anomalous Command Interpreters and Script Execution
5. Unexpected Module Loading and DLL Search Path Manipulation
6. In-Memory PE Artifacts and Reflective Loading Indicators
7. Process Injection Patterns: What Defenders Can Observe
8. Hollowing-Like Anomalies and Image Replacement Signals
9. Module Stomping, Token Misuse, and Persistence Telemetry
10. Chapter Summary

Chapter 6: Memory Forensics on Windows

1. Memory Acquisition Principles and Evidence Integrity
2. Virtual Versus Physical Addresses: What Matters to Defenders
3. Process Enumeration and Hidden/Anomalous Process Detection
4. Loaded Modules, Unlinked DLLs, and Image Analysis
5. VAD Trees and Memory Map Inspection
6. Executable Memory Regions and Permission Anomalies
7. Thread Stacks, Handles, and Inter-Process Artifacts
8. Strings, Configuration Data, and Embedded Artifacts
9. Network Artifacts in Memory and Connection State
10. YARA Scanning, IOC Hunting, and Behavioral Patterns
11. Timeline Reconstruction and Process Tree Recovery
12. Chapter Summary

Chapter 7: EDR Engineering and Detection Architecture

1. Endpoint Telemetry Architecture: Sensors and Data Paths
2. Kernel Versus User-Mode Visibility and Trade-offs
3. Event Normalization, Enrichment, and Entity Resolution
4. Process Ancestry Graphs and Stateful Correlation
5. Behavioral Analytics: From IOCs to Attack Patterns
6. Rule Engines, Sigma/YARA Concepts, and Detection Logic
7. Alert Scoring, Deduplication, Suppression, and Baselining
8. Telemetry Integrity, Tamper Awareness, and Sensor Health
9. Performance Overhead, Privacy, and Retention Policies
10. Chapter Summary

Chapter 8: Network Detection for Memory-Resident Threats

1. Windows Networking Fundamentals for Detection
2. Socket-to-Process Correlation and Connection Telemetry
3. DNS Patterns: DGA, Tunneling, and Beacon Indicators
4. HTTP/HTTPS Metadata: Headers, Hostnames, and Timing
5. TLS Visible Artifacts: SNI, JA3, and Certificate Signals
6. SMB, RDP, and Enterprise Protocol Anomalies
7. Beacon Detection: Periodicity, Jitter, and Statistical Methods
8. Zeek and Suricata Telemetry for Memory-Resident Threats
9. Endpoint-Network Correlation Strategies
10. Chapter Summary

Chapter 9: Building a Defensive Research Laboratory

1. Virtualization Choices and Isolation Requirements
2. Network Segmentation and Controlled Connectivity
3. Test Endpoints: Windows Versions, Configurations, and Telemetry
4. Forensic Workstations and Analysis Tools
5. Packet Capture Infrastructure and Storage
6. Synthetic Attack Telemetry and Benign Emulation Frameworks
7. Time Synchronization, Logging, and Evidence Management
8. Snapshots, Reset Procedures, and Containment Safeguards
9. Chapter Summary

Chapter 10: Detection Engineering Methodology

1. Threat-Informed Detection and Hypothesis Formulation
2. Observable Selection and Telemetry Requirements
3. Behavioral Abstractions and ATT&CK Mapping
4. Detection-as-Code: Versioning, Review, and Reproducibility
5. Unit Testing, Integration Testing, and Replay Validation
6. Synthetic Telemetry Generation for Rule Testing
7. Precision, Recall, False Positive Budgeting, and Alert Fatigue
8. Severity Scoring, Confidence Levels, and Triage Guidance
9. Coverage Measurement and Detection Gap Analysis
10. Chapter Summary

Chapter 11: Reference Implementation — A Mini EDR Lab

1. Repository Structure and Build Environment
2. Telemetry Collector: ETW Consumer and Log Normalizer
3. PE Parser and Section Analyzer Utility
4. Memory Map Inspector for Authorized Processes
5. Event Correlation Engine and Stateful Tracking
6. Rule Engine with Sigma-Style Pattern Matching
7. IOC Scanner and YARA Integration Layer
8. Investigation Console and Evidence Export
9. Chapter Summary

Chapter 12: Forensic Case Studies and Investigative Workflows

1. Case Study 1: Suspicious PowerShell Activity and Outbound Beaconing
2. Case Study 2: Anomalous DLL Loading and In-Memory PE Artifacts
3. Case Study 3: Unexpected Executable Memory and Process Injection Signals
4. Case Study 4: Correlated Endpoint and Network Indicators of Persistence
5. From Alert to Conclusion: A Repeatable Investigation Workflow
6. Chapter Summary

Chapter 13: Operations, Production Deployment, and Defense Architecture Integration

1. Deploying Detection Infrastructure into Production
2. Security Hardening Recommendations
3. Incident Response Procedures for Memory-Resident Threats
4. Coverage Validation Methodology
5. Architectural Limitations and Adversarial Trade-offs
6. Future Directions for Memory-Resident Threat Defense
7. Chapter Summary

Conclusion

References