Practical Foundations of Linux Debugging, Disassembling, Reversing
Practical Foundations of Linux Debugging, Disassembling, Reversing
Training Course
About the Book
This training course is a Linux version of the previous Practical Foundations of Windows Debugging, Disassembly, Reversing book. It also complements Accelerated Linux Core Dump Analysis training course.
Although the book skeleton is the same as its Windows predecessor, the content was revised entirely because of a different operating system, debugger (GDB), toolchain (GCC, assembler, linker), application binary interface, and even an assembly language flavor, AT&T.
The course is useful for:
- Software support and escalation engineers, cloud security engineers, SRE, and DevSecOps
- Software engineers coming from JVM background
- Software testers
- Engineers coming from non-Linux environments, for example, Windows or Mac OS X
- Linux C/C++ software engineers without assembly language background
- Security researchers without assembly language background
- Beginners learning Linux software reverse engineering techniques
This book can also be used as an x64 assembly language and Linux debugging supplement for relevant undergraduate-level courses.
About the Author
Table of Contents
Contents 4
Preface 9
About the Author 10
Chapter x64.1: Memory, Registers, and Simple Arithmetic 11
Memory and Registers inside an Idealized Computer 11
Memory and Registers inside Intel 64-bit PC 12
“Arithmetic” Project: Memory Layout and Registers 13
“Arithmetic” Project: A Computer Program 14
“Arithmetic” Project: Assigning Numbers to Memory Locations 15
Assigning Numbers to Registers 17
“Arithmetic” Project: Adding Numbers to Memory Cells 18
Incrementing/Decrementing Numbers in Memory and Registers 21
Multiplying Numbers 24
Chapter x64.2: Code Optimization 27
“Arithmetic” Project: C/C++ Program 27
Downloading GDB 28
GDB Disassembly Output – No Optimization 29
GDB Disassembly Output – Optimization 32
Chapter x64.3: Number Representations 33
Numbers and Their Representations 33
Decimal Representation (Base Ten) 34
Ternary Representation (Base Three) 35
Binary Representation (Base Two) 36
Hexadecimal Representation (Base Sixteen) 37
Why are Hexadecimals used? 38
Chapter x64.4: Pointers 41
A Definition 41
“Pointers” Project: Memory Layout and Registers 42
“Pointers” Project: Calculations 43
Using Pointers to Assign Numbers to Memory Cells 44
Adding Numbers Using Pointers 50
Incrementing Numbers Using Pointers 53
Multiplying Numbers Using Pointers 56
Chapter x64.5: Bytes, Words, Double, and Quad Words 61
Using Hexadecimal Numbers 61
Byte Granularity 62
Bit Granularity 63
Memory Layout 64
Chapter x64.6: Pointers to Memory 67
Pointers Revisited 67
Addressing Types 68
Registers Revisited 73
NULL Pointers 74
Invalid Pointers 75
Variables as Pointers 76
Pointer Initialization 77
Initialized and Uninitialized Data 78
More Pseudo Notation 79
“MemoryPointers” Project: Memory Layout 80
Chapter x64.7: Logical Instructions and RIP 89
Instruction Format 89
Logical Shift Instructions 90
Logical Operations 91
Zeroing Memory or Registers 92
Instruction Pointer 93
Code Section 95
Chapter x64.8: Reconstructing a Program with Pointers 97
Example of Disassembly Output: No Optimization 97
Reconstructing C/C++ Code: Part 1 99
Reconstructing C/C++ Code: Part 2 101
Reconstructing C/C++ Code: Part 3 103
Reconstructing C/C++ Code: C/C++ program 104
Example of Disassembly Output: Optimized Program 105
Chapter x64.9: Memory and Stacks 107
Stack: A Definition 107
Stack Implementation in Memory 108
Things to Remember 110
PUSH Instruction 111
POP instruction 112
Register Review 113
Application Memory Simplified 115
Stack Overflow 116
Jumps 117
Calls 119
Call Stack 121
Exploring Stack in GDB 123
Chapter x64.10: Frame Pointer and Local Variables 127
Stack Usage 127
Register Review 128
Addressing Array Elements 129
Stack Structure (No Function Parameters) 130
Function Prolog 131
Raw Stack (No Local Variables and Function Parameters) 132
Function Epilog 134
“Local Variables” Project 135
Disassembly of Optimized Executable 138
Chapter x64.11: Function Parameters 139
“FunctionParameters” Project 139
Stack Structure 140
Function Prolog and Epilog 142
Project Disassembled Code with Comments 144
Parameter Mismatch Problem 147
Chapter x64.12: More Instructions 149
CPU Flags Register 149
The Fast Way to Fill Memory 150
Testing for 0 152
TEST - Logical Compare 153
CMP – Compare Two Operands 154
TEST or CMP? 155
Conditional Jumps 156
The Structure of Registers 157
Function Return Value 158
Using Byte Registers 159
Chapter x64.13: Function Pointer Parameters 161
“FunctionPointerParameters” Project 161
Commented Disassembly 162
Chapter x64.14: Summary of Code Disassembly Patterns 169
Function Prolog / Epilog 169
LEA (Load Effective Address) 171
Passing Parameters 172
Accessing Parameters and Local Variables 173
Other books by this author
The Leanpub 60-day 100% Happiness Guarantee
Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
See full terms
80% Royalties. Earn $16 on a $20 book.
We pay 80% royalties. That's not a typo: you earn $16 on a $20 sale. If we sell 5000 non-refunded copies of your book or course for $20, you'll earn $80,000.
(Yes, some authors have already earned much more than that on Leanpub.)
In fact, authors have earnedover $12 million USDwriting, publishing and selling on Leanpub.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
