BitTorrent from Scratch

Building a Production-Grade BitTorrent Client in Python

Introduction: Why Build a BitTorrent Client?

Chapter 1: The BitTorrent Ecosystem

1. A Brief History of File Sharing and the Birth of BitTorrent
2. The Core Problem: Efficient Large-File Distribution
3. Centralized vs. Decentralized Architectures
4. Anatomy of a BitTorrent System: Torrents, Trackers and Peers
5. Why Python? Trade-offs and Design Choices
6. Exercises
7. References

Chapter 2: Bencoding, Metadata and the Torrent File

1. Understanding Bencoding: The BitTorrent Data Format
2. Building a Bencoder and Bdecoder in Python
3. Anatomy of a .torrent File
4. Computing Info Hashes and SHA-1 Fingerprints
5. Magnet URIs: Torrent Metadata on the Fly
6. Exercises
7. References

Chapter 3: Tracker Protocols – HTTP and UDP

1. The Role of Trackers in Peer Discovery
2. HTTP Tracker Protocol Deep Dive
3. Implementing HTTP Tracker Communication
4. UDP Tracker Protocol (BEP 15)
5. Scrape Requests and Peer Statistics
6. Exercises
7. References

Chapter 4: DHT – Decentralized Peer Discovery

1. Trackerless Peer Discovery and the Kademlia Algorithm
2. BitTorrent DHT Protocol (BEP 5)
3. Routing Tables and Node Management
4. Implementing Find-Node, Get-Peers and Store Operations
5. Bootstrapping from Known Nodes
6. Exercises
7. References

Chapter 5: The Peer Wire Protocol – Connecting to Peers

1. The BitTorrent Peer Wire Protocol Overview
2. Handshake and Connection Establishment
3. Message Types and the Bitfield
4. Choke, Unchoke and Interest Exchange
5. Request, Piece and Cancel Messages
6. Exercises
7. References

Chapter 6: Choking, Unchoking and Piece Selection

1. The Choking and Unchoking Algorithm
2. Optimistic Unchoking for Peer Discovery
3. Piece Selection Strategies: Rarest-First vs. Sequential
4. Endgame Mode for Stalled Downloads
5. Implementing the Peer Manager
6. Exercises
7. References

Chapter 7: Data Integrity, File I/O and Concurrency

1. Piece Verification and SHA-1 Integrity Checking
2. Efficient Sequential File Writing
3. Async I/O Patterns with asyncio and aiofiles
4. Managing Concurrency: Streams, Buffers and Backpressure
5. Error Recovery and Partial Download Handling
6. Exercises
7. References

Chapter 8: Advanced Peer Discovery – PEX, LPD and Magnet Links

1. Peer Exchange (PEX): Sharing Peer Lists
2. Local Peer Discovery via UDP Broadcast
3. Magnet Links and Metadata Download (BEP 9/14)
4. Integrating Supplementary Discovery Mechanisms
5. Exercises
6. References

Chapter 9: Security, Encryption and IPv6

1. Protocol Encryption (PE2): Preventing ISP Throttling
2. Securing the DHT Against Attacks
3. IPv6 and Compact Address Support (BEP 32/52)
4. NAT Traversal and Network Topology Challenges
5. Exercises
6. References

Chapter 10: Performance Optimization and Production Quality

1. Connection Management and Keepalive Strategies
2. Piece Prioritization for Streaming Downloads
3. Bandwidth Shaping and Rate Limiting
4. Memory Optimization and Buffer Management
5. Structured Logging and Configuration
6. Performance Analysis: Measuring What Matters
7. Exercises
8. References

Chapter 11: Testing, Debugging and Interoperability

1. Unit Testing Protocol Components
2. Integration Testing with a Test Swarm
3. Interoperability with Existing BitTorrent Clients
4. Debugging Tools and Traffic Analysis
5. Exercises
6. References

Chapter 12: Packaging, Deployment and the Future of P2P

1. Packaging and Distribution with Python
2. Legal and Ethical Considerations in P2P File Sharing
3. Performance Benchmarks and Comparison
4. Future Directions for BitTorrent and P2P
5. Final Architecture Review and Lessons Learned
6. Exercises
7. References

Conclusion

References