Solving The N+1 Problem In PHP
Solving The N+1 Problem In PHP
About the Book
The web page was taking three hours to load.
Let that sink in for a moment: three hours.
The code backing the page was a reporting script written in PHP, attempting to build 40,000 entity objects from 2 million database rows. Indeed, sometimes the page never loaded at all. At least the client could tell when it failed, because a big ugly error popped up in the browser window. Unfortunately that was also around the three hour point. The client was understandably frustrated.
It turned out the cause was not that the database was unoptimized. It was not the number of rows being returned. It was not the number of objects being built.No, the cause was the high volume of queries being issued from the reporting script.
When I say "high volume", I mean two hundred thousand and one queries. These 200,000+ queries were used to build each root entity along with each of its five associated collections and entities, all as a result of how the script was assembling the entity objects. No wonder the script was was taking three hours to finish!
After discovering this, I was able to apply the techniques presented in this book to refactor how the entity objects were assembled. Over the course of a few days of refactoring I was able to reduce the load time from 3+ hours to just under 6 minutes. That's a 30x improvement, and while still slow by typical standards, it was much more bearable for the client. (Later refactorings improved the load time even further.)
The causes leading to the terrible performance of that reporting script have a name: The N+1 Problem. The problem goes by various aliases, such as the "1+N problem", "N+1 queries", "SELECT N+1", and so on, but they all refer to the same thing.
While I mention the N+1 problem more than once in Modernizing Legacy Applications in PHP, I did not address a solution directly. This book remedies that. Herein, we will discuss the specifics of the N+1 problem:
- what the N+1 problem is,
- how to diagnose and discover the problem in your application,
- why the problem occurs, and
- how to solve the problem in your codebase using plain old PHP (no ORMs needed!).
If your application code has database performance issues, or if you want to learn how to prevent creating an N+1 problem in your codebase, buy this book today!
About the Author
Table of Contents
-
1. Introduction
- The Slow Reporting Script
- Example Application
-
2. Defining The N+1 Problem
- The Problem By Example
- The Problem Defined
- Performance Implications
-
3. Diagnosing N+1 Problems
- Codebase Examination
- Log/Profile Examination
- Easier Query Examination
-
4. Why Does The N+1 Problem Occur?
- CRUD Is The Problem
- BREAD Is The Answer
-
5. Solving N+1 With A Single Query
- Handling “Many-To-One” Relationships
- Handling “One-To-One” Relationships
- Combining “Many-To-One” and “One-To-One”
- Handling “To-Many” Relationships
- Single-Query Fails With “To-Many” Relationships
-
6. Solving N+1 With Query-And-Stitch
- Tradeoffs In Query-And-Stitch
- Handling “One-To-Many” Relationships
- Handling “Many-To-Many” Relationships
- Handling “Many-To-One” Relationships
- Handling “One-To-One” Relationships
- Performance Implications
-
7. Refactoring The Codebase To Remove N+1 Problems
- Prerequisites
- The Process
- N+1 In A Single Location
- N+1 Across Separated Locations
- Test, Commit, Push, Notify QA
- Do … While
-
8. Common Questions
- Can I Combine Single-Query With Query-And-Stitch?
-
Can I Use Sub-Selects Instead Of The
IN()Operator? - Should I Use A Transaction With Query-And-Stitch?
- Can I Just Use An ORM?
-
Appendix
- SQL Schema
- PHP Classes
- Colophon
- About the Author
The Leanpub 45-day 100% Happiness Guarantee
Within 45 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
See full terms
Do Well. Do Good.
Authors have earned$10,894,929writing, publishing and selling on Leanpub, earning 80% royalties while saving up to 25 million pounds of CO2 and up to 46,000 trees.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers), EPUB (for phones and tablets) and MOBI (for Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
Top Books
- #1
Functional event-driven architecture: Powered by Scala 3
Gabriel VolpeExplore the event-driven architecture (EDA) in a purely functional way, mainly powered by Fs2 streams in Scala 3!
Leverage your functional programming skills by designing and writing stateless microservices that scale, powered by stateful message brokers.
- #2
Practical FP in Scala: A hands-on approach
Gabriel VolpeA practical book aimed for those familiar with functional programming in Scala who are yet not confident about architecting an application from scratch.
Together, we will develop a purely functional application using the best libraries in the Cats ecosystem, while learning about design patterns and best practices.
- #3
入門 日本語自然言語処理
Masato Hagiwara and Paul O'Leary McCann日本語テキストを処理したい全てのプログラマ・エンジニアの方へ。分かち書きなどの基本から、自然言語生成などの最新の話題までをカバー。動かして学べるコードや、参照文献も付いています。言語学や機械学習の知識が無くても問題ありません。
- #4
Book of Monads
Alejandro Serrano MenaIn functional programming, at the heart of input/output, failure, state, logic, and much more, lies a powerful abstraction called monad. This book provides a journey from the very first concepts, to the myriad of monads available to programmers, down to the categorical foundations.
- #5
Ansible for DevOps
Jeff GeerlingAnsible is a simple, but powerful, server and configuration management tool. Learn to use Ansible effectively, whether you manage one server—or thousands.
- #6
Functional Programming Made Easier
Charles ScalfaniA Functional Programming book from beginner to advanced without skipping a single step along the way.
In my 40 years of programming, I've felt that programming books always let me down, especially Functional Programming books. So, I wrote the book I wish I had 5 years ago.
Functional Programming will never be easy, but it can be easier.
- #7
Visualise, document and explore your software architecture
Simon BrownA short guide to visualising, documenting and exploring your software architecture.
- #8
Introducing EventStorming
Alberto BrandoliniThe deepest tutorial and explanation about EventStorming, straight from the inventor.
- #9
Introduction to Japanese Natural Language Processing
Masato Hagiwara and Paul O'Leary McCannA thorough guide for programmers working with Japanese text, covering fundamental issues like tokenization and recent research topics like generating natural language texts. Working examples are accompanied by extensive reference to allow problem solving even without a background in Japanese or Machine Learning.
- #10
Cloud Strategy
Gregor Hohpe“Strategy is the difference between making a wish and making it come true.” A successful migration to the cloud can transform your organization, but it shouldn’t be driven by wishes. This book tells you how to develop a sound strategy guided by frameworks and decision models without being overly abstract nor getting lost in product details.
Top Bundles
- #1
Software Architecture for Developers: Volumes 1 & 2 - Technical leadership and communication
2 Books
"Software Architecture for Developers" is a practical and pragmatic guide to modern, lightweight software architecture, specifically aimed at developers. You'll learn:The essence of software architecture.Why the software architecture role should include coding, coaching and collaboration.The things that you really need to think about before... - #2
Modern C++ Collection
3 Books
Get All about Modern C++C++ Standard Library, including C++20Concurrency with Modern C++, including C++20C++20Each book has about 200 complete code examples. Updates are included. When I update one of the books, you immediately get the updated bundle. You can expect significant updates to each new C++ standard (C++23, C++26, .. ) and also... - #3
CCIE Service Provider Ultimate Study Bundle
2 Books
Piotr Jablonski, Lukasz Bromirski, and Nick Russo have joined forces to deliver the only CCIE Service Provider training resource you'll ever need. This bundle contains a detailed and challenging collection of workbook labs, plus an extensively detailed technical reference guide. All of us have earned the CCIE Service Provider certification... - #4
All the Books of The Medical Futurist
6 Books
We put together the most popular books from The Medical Futurist to provide a clear picture about the major trends shaping the future of medicine and healthcare. Digital health technologies, artificial intelligence, the future of 20 medical specialties, big pharma, data privacy, digital health investments and how technology giants such as Amazon... - #6
Abandon Your Job - Leaving the Rat Race with Python
8 Books
Now, get all the great Python and Freelancing books from finxter.com in one single bundle!Coffee Break Python - All Python BasicsCoffee Break Python Workbook - Practice the basics of PythonCoffee Break Python Mastery Workouts - Practice advanced Python conceptsCoffee Break Python Slicing - All about SlicingCoffee Break Python NumPy - Get started... - #8
10 Books Bundle - A New Career in Tech 🚀
10 Books
Do you want to create yourself a new and profitable skillset - in one year or less? Do you want to position yourself on the right side of change finally? Do you want to work from home and set your own work schedule while choosing fun and interesting coding projects? Software developers and freelance developersearn six figuresin the US, according... - #9
The Python Craftsman
3 Books
The Python Craftsman series comprises The Python Apprentice, The Python Journeyman, and The Python Master. The first book is primarily suitable for for programmers with some experience of programming in another language. If you don't have any experience with programming this book may be a bit daunting. You'll be learning not just a programming...
