The Mach-O File Format

Architecture, Internals, Parsing, Linking, Loading and Runtime Execution

Introduction

Chapter 1: Why Binary Formats Matter

1. The Role of Executable Formats in System Design
2. From A.OUT to ELF to Mach-O
3. Object Files vs Executables vs Shared Libraries
4. The Linker and Loader as Format Interpreters
5. Why Mach-O on Apple Platforms

Chapter 2: Computer Architecture Fundamentals for Binary Formats

1. Instruction Set Architectures and Endianness
2. Virtual Memory and Address Spaces
3. Alignment, Padding and Data Layout
4. Position-Dependent vs Position-Independent Code
5. Calling Conventions and ABI Requirements

Chapter 3: The Mach-O Header

1. Magic Numbers and Architecture Identification
2. The fat_header and FAT Architecture Descriptors
3. thin_struct (32-bit) and mach_header_64 (64-bit)
4. Platform-Specific Flags: MH_NOUNDEFS, MH_PIE, MH_DYLIB and Beyond
5. File Types
6. Version History and Compatibility Guarantees

Chapter 4: FAT and Universal Binaries

1. The FAT File Format Specification
2. Fat Architectures: cputype, cpusubtype and Alignment
3. Construction and Extraction with lipo
4. Historical Context: PowerPC to Intel Transition
5. Modern Universal Binaries and Rosetta 2

Chapter 5: Load Commands: The Nervous System of Mach-O

1. Load Command Structure and Parsing
2. LC_SEGMENT vs LC_SEGMENT_64
3. LC_SYMTAB, LC_STRTAB, LC_DYSYMTAB
4. LC_LOAD_DYLIB Family and Dependencies
5. LC_RPATH, LC_ID_DYLIB, LC_MAIN, LC_UNIXTHREAD
6. Platform-Specific Load Commands (iOS, watchOS)

Chapter 6: Segments and Memory Mapping

1. Segment Design and Protection Flags
2. Standard Segments: __TEXT, __DATA, __DATA_CONST, __LINKEDIT
3. Non-Standard and Platform-Specific Segments
4. Segment Alignment, File Offsets and Virtual Addresses
5. Virtual Address to File Offset Translation

Chapter 7: Sections, Section Types and Section Flags

1. Section Header Structure and Field Semantics
2. Code Sections: __text, __stubs, __stub_helper, __got, __lazy_symbol_pointers
3. Data Sections: __data, __const, __rodata, __cstring, __const
4. BSS and Zero-Fill Sections
5. Special Sections: __mod_init_func, __mod_term_func, _objc*

Chapter 8: Symbol Tables and String Tables

1. nlist and nlist_64 Structures
2. The String Table: Layout, Offsets and Null Termination
3. Symbol Table Layout and Symbol Sorting
4. Global vs Local Symbols: Binding and Visibility
5. Special Symbols: _mh_header, __mh_execute_header, _dyld_func_lookup

Chapter 9: Relocation Records and Position Independence

1. Relocation Record Structures (32-bit and 64-bit)
2. External vs Internal Relocations
3. Absolute, PC-Relative and Scatter Relocations
4. Relocations in Object Files vs Executables
5. Runtime Relocations and the LINKEDIT Segment

Chapter 10: The Dynamic Symbol Table

1. dysymtab_command Structure
2. Symbol Table Indices: ilocalsym, iextdefsym, iundefsym
3. Indirect Symbol Table: Layout, Encoding and Null Entries
4. The Local Symbol Table and Debugging
5. Export and Import Symbol Resolution

Chapter 11: Dynamic Library Loading

1. Library Identification: LC_ID_DYLIB, LC_LOAD_DYLIB, LC_LOAD_WEAK_DYLIB
2. Library Paths: ID, Install Name, Run Path, @rpath, @executable_path
3. rpath Resolution and Search Order
4. Weak Linking and Conditional Dependencies
5. Library Reexports and Symbol Reexports

Chapter 12: dyld Architecture and Internals

1. dyld Initialization and Bootstrap
2. Image Loading and Memory Mapping
3. Shared Cache Architecture
4. Symbol Binding: Non-Lazy vs Lazy Binding
5. Two-Level Namespace and Flat Namespace
6. dyld Interposition and Function Hooking

Chapter 13: Initialization, Finalization and Thread-Local Storage

1. Initializer Mechanisms: LC_ROUTINES, __mod_init_func, __objc_init
2. Initialization Ordering and Dependency Graphs
3. Terminators and Shutdown
4. Thread-Local Storage in Mach-O: __thread, LC_SEGMENT_64 TLS segments
5. TLS Models: Local-Exec, Initial-Exec, Local-Dynamic, Global-Dynamic

Chapter 14: Chained Fixups

1. Motivation for Chained Fixups
2. The LC_DYLD_CHAINED_FIXUPS Payload
3. Chain Pointer Structures: chained_fixup_pointer_64
4. Fixup Kinds: Target Address, Addend, Rebase, Bind
5. Parsing and Traversing Fixup Chains
6. Platform-Specific Chained Fixup Variants

Chapter 15: Export Tries and Compressed Symbol Information

1. Export Info Load Command
2. The Export Trie Data Structure
3. Trie Encoding: Terminal Nodes, Edges, Re-export Encoding
4. Parsing Algorithms for Export Tries
5. Size Savings and Performance Implications
6. Export Version Info and Platform Compatibility

Chapter 16: Rebase and Bind Operations

1. LC_DYLD_INFO: Rebase and Bind Streams
2. Rebase Operands and Encodings
3. LC_BIND Operations for Symbol Binding
4. Weak Bind, Lazy Bind and Re-export Streams
5. Optimization Through Compressed Operations
6. Relationship to Chained Fixups

Chapter 17: ARM64 and ARM64e Architecture Specifics

1. ARM64 in Mach-O: cputype, cpusubtype, Instruction Encoding
2. ARM64 Relocations and Fixup Kinds
3. ARM64e and Pointer Authentication
4. PAC Signing: aarch64-specific Instructions and Metadata
5. Pointer Mask Bits and Branch Protection
6. Platform Requirements and Compatibility

Chapter 18: Platform Differences: macOS vs iOS vs watchOS vs tvOS

1. Platform-Specific Load Commands
2. Security Requirements by Platform
3. Signing and Entitlements Differences
4. System Library Layout and Paths
5. Debug Symbols and Distribution Binaries

Chapter 19: Code Signing Architecture

1. The LC_CODE_SIGNATURE Load Command
2. __LINKEDIT and Code Signature Data Layout
3. SuperBlob Structure and Subblob Types
4. The CodeDirectory: Hashes, Teams, Identifiers
5. Requirements and the Requirements Blob
6. Entitlements and Provisioning Profiles

Chapter 20: Hardened Runtime, Notarization and Runtime Security

1. Hardened Runtime and Library Validation
2. Notarization and Gatekeeper
3. Entitlements and Capabilities
4. Library Load Restrictions and dyld Shared Cache
5. Sandbox and Mach-O Interaction
6. Secure Boot and Code Signing Chain

Chapter 21: Command-Line Analysis Tools

1. file, otool, nm and lipo Fundamentals
2. dyld_info: Dependencies, Exports and Bindings
3. codesign: Verification, Entitlements and Requirements
4. llvm-readobj and llvm-objdump (Cross-Platform)
5. LLDB: Examining Memory Layout and Symbols

Chapter 22: Building a Mach-O Parser in C

1. Design Decisions and Module Architecture
2. Reading and Validating FAT Headers
3. Parsing Mach-O Headers and Architectures
4. Safe Load Command Iteration with Bounds Checking
5. Segment and Section Enumeration
6. Symbol Table and String Table Extraction

Chapter 23: Advanced Parsing: Exports, Fixups and Signatures

1. Export Trie Parsing Implementation
2. Chained Fixup Decoding
3. Code Signature Inspection and Validation
4. Virtual Address to File Offset Mapping
5. Structural Validation and Error Reporting
6. Test Cases and Verification Against Apple Tools

Chapter 24: The Mach-O Toolchain Lifecycle

1. clang Compilation and Assembly Generation
2. Object File Creation and Format
3. Static Linking with libtool
4. Dynamic Linking with ld
5. Code Signing and Distribution
6. Runtime Loading and Execution Trace

Chapter 25: Reverse Engineering and Forensic Analysis

1. Identifying Binary Type and Architecture
2. Reconstructing Symbol Information
3. Analyzing Dependencies and Strings
4. Locating Entry Points and Initializers
5. Understanding Control Flow from Sections
6. Common Anti-Analysis Techniques

Conclusion

1. What Mach-O Design Reveals About Apple’s Philosophy
2. Ongoing Evolution and Open Questions
3. The Future of Binary Formats on Apple Platforms

References