A Practical Guide to Understanding, Reverse Engineering, and Automating Network Data Analysis
Introduction
- What Protocol Analysis Is Not
- What Protocol Analysis Is
- Why Claude Code
- Who This Book Is For
- How to Read This Book
- A Note on Ethics and Safety
- The Central Thesis
Chapter 1: How Networks Talk
- What a protocol really is
- The layers that matter (and which ones do not)
- Connection-oriented and connectionless communication
- Addressing, routing and the path a packet takes
- Ports, sockets and multiplexing
- State and conversations
Chapter 2: The Evidence in Packets
- What PCAP is and is not
- Link-layer visibility and blind spots
- IP, TCP, UDP: what each header tells you
- Payloads and encryption boundaries
- Timestamps, sequence numbers and reassembly signals
- What you can never see in a capture
Chapter 3: Claude Code as Analysis Partner
- What Claude Code is and how it differs from chat interfaces
- Setting up your analysis environment
- Context management: what to share, when and how
- Effective prompts for protocol analysis tasks
- When Claude Code excels and when it fails
- Trust calibration: validating AI-generated analysis
Chapter 4: The Investigation Method
- Starting from a question, not a tool
- Forming and documenting hypotheses
- Gathering initial evidence: the first thirty minutes
- Generating competing hypotheses
- Testing and falsifying hypotheses
- Documenting confidence levels and assumptions
- When to stop and what “solved” actually means
Chapter 5: Project Structure and Context Engineering
- Directory layout for analysis projects
- Context files: what Claude Code needs to know
- Note-taking patterns for investigations
- Code organisation: modular, testable analysis scripts
- Version control and collaboration considerations
- Reproducibility: capturing analysis decisions
Chapter 6: Capture and Ingestion
- tcpdump: the foundation, filters, options, pitfalls
- tshark: powerful CLI analysis
- Wireshark: when the GUI matters and when it does not
- Capture placement and scope decisions
- Capture loss: recognizing and mitigating it
- File formats and interoperability
Chapter 7: Stream and Session Reconstruction
- TCP streams: sequence numbers and reassembly
- UDP and pseudo-sessions
- Handling retransmissions and out-of-order delivery
- Fragmentation and its complications
- NAT: recognizing and working through it
- Implementation: stream extraction with tshark and Python
Chapter 8: Protocol Detection and Classification
- Port-based identification and why it lies
- TLS handshake fingerprints and version detection
- ASCII signatures and protocol banners
- Statistical protocol detection
- Unknown protocols: when everything fails
- Automated detection pipelines
Chapter 9: Text and Binary Decoding
- Reading text-based protocols: HTTP, SMTP, etc.
- Binary representation: hex, endian and structure
- Base64, URL encoding and other transformations
- Finding structure in binary blobs
- Mixed-content protocols
- Claude Code as a decoder: patterns and pitfalls
Chapter 10: Message Boundaries and Framing
- Length-prefixed framing
- Delimiter-based framing
- Fixed-length messages
- Session-based and timing-based boundaries
- Protocol multiplexing and sub-streams
- Practical techniques for boundary detection
Chapter 11: Field Discovery and Schema Inference
- Constant vs variable fields
- Numeric fields: integers, floats, bitfields
- String fields and character encodings
- Repeated fields and arrays
- Nested structures
- Using Claude Code for schema hypothesis generation
Chapter 12: State Machines and Protocol Logic
- Observable state transitions
- Request-response patterns and session flows
- Timeout and retransmission behaviour
- Error handling and recovery paths
- Inferring state from message sequences
Chapter 13: Fingerprinting and Identification
- TLS fingerprinting (JA3, JA4, etc.)
- HTTP client and server fingerprints
- TCP stack fingerprints
- Timing and behavioural fingerprints
- Application-level fingerprints
- Privacy implications and evasion
Chapter 14: Statistical Analysis and Pattern Discovery
- Basic statistics: distributions, outliers, correlations
- Traffic volume and timing analysis
- Protocol-level metrics
- Using Python for statistical analysis
Chapter 15: Anomaly Detection and Suspicious Behaviour
- Defining normal and abnormal
- Signature-based detection
- Statistical anomaly detection
- Protocol violations and edge cases
- Malformed and crafted packets
- False positives and validation
Chapter 16: Correlation and Cross-Flow Analysis
- Host-centric correlation
- Temporal correlation
- Payload correlation across streams
- Session linkage
- Attack-reconnaissance correlation
- Tools for correlation analysis
Chapter 17: Case Study - Reverse Engineering a Custom Protocol
- The scenario: new internal service with unknown protocol
- Initial reconnaissance and tool selection
- Stream extraction and first observations
- Hypothesis generation with Claude Code
- Iterative decoding and validation
- Final protocol documentation
Chapter 18: Case Study - Encrypted and Obscured Traffic
- The scenario: unexpected encrypted service
- TLS analysis and certificate inspection
- SNI and JA3 analysis
- Traffic pattern analysis
- Metadata and timing analysis
- Limits and what remains unknown
Chapter 19: Case Study - Large-Scale Analysis Pipeline
- The scenario: weekly analysis of gigabytes of traffic
- Architecture: ingestion, processing, storage
- Flow extraction and indexing
- Automated protocol classification
- Anomaly detection integration
- Report generation and validation
Chapter 20: Automated Analysis Pipelines
- Pipeline architecture patterns
- Zeek for scalable protocol analysis
- Scapy for custom processing
- Database and columnar storage options
- Orchestration and scheduling
- Monitoring and reliability
Chapter 21: Reproducibility, Validation and Trust
- Reproducible analysis environments
- Independent verification of Claude Code output
- Validation techniques: cross-checking, manual review
- Documenting analysis methodology
- Peer review and collaboration
- Operational decision-making from analysis