Lift the Elephant
Scaling PostgreSQL Beyond Query Optimization
When your database outgrows simple optimizations, it's time to think bigger. Lift the Elephant goes beyond query tuning to reveal actionable strategies for scaling PostgreSQL, from partitioning to high-availability architectures. Built on lessons from scaling Hubstaff, this is your playbook for navigating the challenges of database growth.
Minimum price
Free!
$14.99
You pay
$14.99Cause gets
$11.99About
About the Book
Scaling a database is often a journey that starts simply and grows complicated fast. PostgreSQL is one of the most trusted and capable databases out there, but as your application evolves, so do the demands placed on it. What begins as a straightforward setup for data storage can quickly grow into a sprawling ecosystem that requires strategic tuning, scaling, and diligent maintenance.
This book is about navigating that journey. It’s not a beginner’s guide or a list of theoretical best practices—it’s a playbook for tackling the real-world challenges of scaling Postgres in high-demand environments. From parameter tuning and connection pooling to partitioning, and active-active setups, I’ll walk you through the techniques and strategies that enable Postgres to thrive under pressure.
These lessons come from scaling Hubstaff, a platform that serves thousands of businesses worldwide. As the CTO, I’ve been in the trenches of database scaling, learning through late-night incident mitigations, unexpected bottlenecks, and breakthrough optimizations. Scaling Postgres has often felt like lifting an elephant: a task that seems impossible until you figure out how to approach it.
Feedback
Author
About the Author
CTO at Hubstaff, where he has spent the last 7 years scaling a high-throughput Rails and Rust apps backed by PostgreSQL.
Contents
Table of Contents
- Introduction
- Beyond Simple Query Optimization
- Scaling Postgres, Pragmatically
- Who This Book Is For
- Ready to Lift?
- What is synchronous_commit all about?
- The Concept: When Performance Meets Assurance
- Modes of Synchronous Commit
- Practical Scenarios and Recommendations
- The Value of Contextual Tuning
- Monitor, Adapt, Perfect
- Fine-Tuning random_page_cost
- Understanding random_page_cost
- Why It Matters: The Query Planner's Decisions
- Real-World Scenarios
- How to Tune random_page_cost
- Best Practices
- The Most Important Events to Monitor in Your Logs
- Preventing Downtime: Critical Events to Monitor
- Reducing I/O Spikes: Understanding Checkpoints
- Improving Query Performance: Key Events
- Deep Diving into Performance Issues
- Handling Sensitive Data in Logs
- Best Practices for Log Monitoring
- How to Deal with Long-Running Transactions
- The Problems with Long-Running Transactions
- Strategies for Mitigating Long-Running Transactions
- Best Practices for OLTP Systems
- HOT Updates and Fill Factor
- Understanding HOT Updates
- The Fill Factor: Setting the Stage for HOT Updates
- Measuring and Monitoring HOT Updates
- Use Cases: When to Focus on HOT Updates and Fill Factor
- Best Practices for Implementation
- Guide to pg_stat_statements
- What is pg_stat_statements?
- Setting Up pg_stat_statements
- Key Metrics and Their Significance
- Using pg_stat_statements for Query Optimization
- Advanced Use Cases
- Limitations and Workarounds
- Mastering Checkpoints
- What Are Checkpoints?
- Key Parameters for Tuning Checkpoints
- Balancing Checkpoints for Performance
- Advanced Strategies
- Monitoring and Maintenance
- Closing Thoughts
- Mastering Memory: Tuning work_mem for Performance
- What is work_mem?
- When to Tune work_mem
- Best Practices for Tuning work_mem
- Example: Evaluating work_mem Impact
- Shared Buffers: The Core of Memory Allocation
- What is shared_buffers?
- Best Practices for Tuning shared_buffers
- Effective Cache Size: Guiding the Query Planner
- What is effective_cache_size?
- Best Practices for Setting effective_cache_size
- Maintenance Work Memory: Optimizing Maintenance Tasks
- What is maintenance_work_mem?
- When to Tune maintenance_work_mem
- Best Practices for Tuning maintenance_work_mem
- Effective I/O Concurrency: Optimizing SSD Usage
- What is effective_io_concurrency?
- Recommended Settings for SSDs
- Avoid Overcommitment
- Parallel Query Settings: Leveraging Multi-Core Systems
- Understanding Parallel Query Execution
- Key Parallel Query Parameters
- Monitoring and Optimizing Parallel Query Performance
- Huge Pages: Reducing Memory Overhead
- What Are Huge Pages?
- Best Practices for Configuring Huge Pages
- Strategies for Index Maintenance
- Analyzing Index Health
- Maintaining Index Integrity
- Automating Index Maintenance
- Advanced Tips for Long-Term Index Health
- Monitoring Transaction ID Wraparound Risks
- Understanding Transaction ID Wraparound
- Monitoring Transaction ID Health
- Mitigation Strategies
- Lessons from the Field
- Mastering Autovacuum
- Why Autovacuum Matters
- Tuning Autovacuum for Performance
- Leveraging Parallel Vacuuming for Large Tables
- Investigating Common VACUUM-Related Issues
- Best Practices for Effective Autovacuum
- Understanding and Managing PostgreSQL Bloat
- What is Bloat?
- Detecting Bloat
- Mitigation Strategies
- Proactive Bloat Prevention
- The Power of Connection Pooling
- Introduction to PostgreSQL Connection Poolers
- PgBouncer
- Pgpool-II
- PgCat
- Choosing the Right PostgreSQL Connection Pooler
- Horizontal Scaling Demystified: Guide to Distributed Databases
- Leveraging Hot Standby Servers
- Expanding Capacity with Multiple Read Replicas
- Introducing Time-Lagged Standby Replicas
- Customizing Data Flow with Logical Replication
- Ensuring Data Safety with Synchronous Replication
- Understanding Active-active Architecture
- Setting Up Active-active Architecture
- The Case for Partitioning
- Managing Gigantic Tables
- Key Benefits of Partitioning
- Types of Partitioning in PostgreSQL
- When Partitioning Falls Short
- Guiding Principles for Partitioning
- Enhance Query Performance with Partition-Wise Operations
- Indexing Strategies for Partitioned Tables
- Closing Thoughts
- The Scaling Mindset
- What Separates Good From Great
- The Elephant Doesn't Stay Still
- Your Turn
Get the free sample chapters
Click the buttons to get the free sample in PDF or EPUB, or read the sample online here
Causes
Causes Supported

Code Club
https://www.codeclub.org.ukThe Leanpub 60 Day 100% Happiness Guarantee
Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
Now, this is technically risky for us, since you'll have the book or course files either way. But we're so confident in our products and services, and in our authors and readers, that we're happy to offer a full money back guarantee for everything we sell.
You can only find out how good something is by trying it, and because of our 100% money back guarantee there's literally no risk to do so!
So, there's no reason not to click the Add to Cart button, is there?
See full terms...
Earn $8 on a $10 Purchase, and $16 on a $20 Purchase
We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.
(Yes, some authors have already earned much more than that on Leanpub.)
In fact, authors have earned over $14 million writing, publishing and selling on Leanpub.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
Write and Publish on Leanpub
You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!
Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.
Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.