Chapter 1: What is Computing Power?
- 1.0 Introduction
- 1.1 Defining Computing Power
- 1.2 Not All FLOPS Are Created Equal
- 1.3 Peak vs. Real-World Performance
- 1.4 General-Purpose vs. Specialized Computing
- 1.5 A Journey Through Scales: From ENIAC to Modern AI
- 1.6 The Five Dimensions of Computing Power
- 1.7 Why This Book Matters
- 1.8 The Economics of Compute: What AI Training Actually Costs
- 1.9 The Benchmarking War: MLPerf and the Limits of Performance Metrics
- Key Takeaways
- What We Learned
- References and Further Reading
Chapter 2: The Evolution of Computing (1940s-1990s)
- 2.0 Introduction
- 2.1 ENIAC: The Beginning of Everything
- 2.2 The Transistor Revolution
- 2.3 The Supercomputer Era
- 2.4 The Personal Computer Revolution
- 2.5 The 1990s: The Internet and the Client-Server Era
- 2.6 The Hidden Thread: Path Dependencies That Shaped Today
- Key Takeaways
- What We Learned
- References and Further Reading
Chapter 3: The Internet Age and Early Data Centers
- 3.0 Introduction
- 3.1 The Dot-Com Boom and the Data Center Explosion
- 3.2 The Architecture of Early Data Centers
- 3.3 The Birth of Colocation and Web Hosting
- 3.4 VMware and the Virtualization Revolution
- 3.5 Amazon: From Online Bookstore to Cloud Pioneer
- 3.6 Google and the Cluster Computing Model
- 3.7 Microsoft’s Late Arrival
- 3.8 What We Learned
- 3.9 The OpenStack and Cloud-Native Revolution
- 3.10 The Colocation Industry’s Evolution
- 3.11 Data Center Power Density Evolution
- Key Takeaways
- References and Further Reading
Chapter 4: NVIDIA’s Rise
- 4.0 Introduction
- 4.1 The Early Years: Surviving the Graphics Wars
- 4.2 The CUDA Bet
- 4.3 The Deep Learning Trigger
- 4.4 The Financial Transformation
- 4.5 The Road to Blackwell
- 4.6 Beyond Hardware: The Full-Stack Vision
- 4.7 What We Learned
- 4.8 The Mellanox Acquisition: Why Networking Became a Strategic Necessity
- 4.9 DGX and the Turnkey AI Solution
- 4.10 The Geopolitical Twist: Export Controls and the China Dilemma
- Key Takeaways
- References and Further Reading
Chapter 5: The CUDA Ecosystem
- 5.0 Introduction
- 5.1 The CUDA Software Stack: Architecture of Dominance
- 5.2 cuDNN: The Deep Learning Standard
- 5.3 TensorRT: The Inference Engine
- 5.4 The Developer Ecosystem: Network Effects in Practice
- 5.5 The Competitive Moat: Why Rivals Can’t Catch Up
- 5.6 The Counterpoint: OpenCL, SYCL, and the Failed Attempts at Portability
- 5.7 ROCm: AMD’s Open-Source Alternative—and Why It’s Struggling
- 5.8 The Cost of Vendor Lock-In
- 5.9 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 6: AMD’s Comeback
- 6.0 Introduction
- 6.1 AMD’s Long History: From #2 to Nearly Dead
- 6.2 The Lisa Su Turnaround: Corporate Survival Class
- 6.3 The GPU Journey: From GCN to CDNA
- 6.4 The Chiplet Revolution: AMD’s Secret Weapon
- 6.5 Why AMD Still Lags: The Three Limitations
- 6.6 The MI400 Strategy and the Path Forward
- 6.7 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 7: Intel’s Struggle
- 7.0 Introduction
- 7.1 Intel’s Missed Opportunities: A History of Near-Misses
- 7.2 The Silent Success: Xeon for AI Inference
- 7.3 The Xe Architecture: Another Almost
- 7.4 Gaudi: The AI Accelerator That Might Actually Work
- 7.5 Why Intel Struggles: The Four Root Causes
- 7.6 The Pat Gelsinger Pivot: IDM 2.0
- 7.7 Intel’s AI Future: Three Scenarios
- 7.8 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 8: Cloud Giants’ Custom Chips
- 8.0 Introduction
- 8.1 Why Cloud Providers Built Their Own Chips
- 8.2 Google: The Pioneer—How TPUs Shaped AI
- 8.3 Amazon: The Pragmatist—Building Chips to Sell
- 8.4 Microsoft: The Late Arriver—and Why It Matters
- 8.5 Other Players: Meta, OpenAI, and the Custom Chip Tipping Point
- 8.6 The Competitive Dynamics: “Coopetition” with NVIDIA
- 8.7 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 9: AWS — Amazon’s Computing Empire
- 9.0 Introduction: The Core Argument
- 9.1 The Birth of Cloud Computing: Why Amazon?
- 9.2 AWS Growth Story: The Flywheel in Action
- 9.3 AWS Architecture: The Nitro Revolution
- 9.4 The Custom Silicon Strategy: Building the Stack
- 9.5 Why AWS Dominates: The Moat and Its Weaknesses
- 9.6 AWS’s AI Strategy: Bedrock, SageMaker, and the GPU Gamble
- 9.7 The Future: Challenges and Opportunities
- 9.8 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 10: Microsoft Azure & Google Cloud
- 10.0 Introduction: The Core Argument
- 10.1 Microsoft Azure: The Enterprise Giant Awakens
- 10.2 Google Cloud: The AI Specialist
- 10.3 The Cloud AI Arms Race
- 10.4 The Fourth Player: Alibaba Cloud
- 10.6 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 11: The AI Explosion
- 11.0 Introduction: The Core Argument
- 11.1 The Deep Learning Renaissance (2012-2017)
- 11.2 The Transformer Revolution (2017)
- 11.3 The Scaling Laws (2020)
- 11.4 The Foundation Model Era (2020-2022)
- 11.5 The ChatGPT Moment (2022)
- 11.6 Training vs. Inference: The Economics Divide
- 11.7 The Open-Source AI Movement
- 11.8 The Regulatory Landscape
- 11.9 The Future: Where Is AI Compute Going?
- 11.10 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 12: Huawei Ascend — The Sanctions and the Comeback
- 12.0 Introduction
- 12.1 The Rise of HiSilicon (2004–2019)
- 12.2 The Ascend Chips: China’s AI Compute Ambition
- 12.3 The Entity List: A Devastating Blow
- 12.4 The Software Breakthrough: HarmonyOS and the Kirin 9000S
- 12.5 Ascend Reborn: The 910B, 920, and Beyond
- 12.6 The Road Ahead: Constraints and Uncertainties
- 12.7 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 13: Chinese Chip Makers — The Challengers Beneath the Export Controls
- 13.0 Introduction
- 13.1 Cambricon: The AI Chip Pioneer That Struggled to Scale
- 13.2 Hygon: China’s x86 Alternative
- 13.3 JingJia Micro: The GPU Dream
- 13.4 BIRENTECH: The $1 Billion Bet
- 13.5 MetaX: The AMD Connection
- 13.6 Enflame: The Anchor Customer Model
- 13.7 Moore Threads and Others
- 13.8 The Supply Chain Reality
- 13.9 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 14: Chinese Supercomputers — The Rise, Dominance, and Strategic Silence
- 14.0 Introduction
- 14.1 The Foundations: China’s Supercomputing Origins
- 14.2 Tianhe Series: The Rise and the Sanctions Wake-Up
- 14.3 Sunway TaihuLight: The Pinnacle of Domestic Design
- 14.4 The Strategic Silence (2018–Present)
- 14.5 The Exascale Race (Without the Publicity)
- 14.6 The Military Connection
- 14.7 Supercomputing vs. AI: Two Converging Worlds
- 14.8 The Quantum Alternative
- 14.9 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 15: Moore’s Law Is Dead? — The End of Silicon Scaling and What Comes Next
- 15.0 Introduction
- 15.1 The Physics of Transistor Scaling — Why Smaller Is Harder
- 15.2 EUV Lithography — ASML’s Monopoly and the Gatekeeper of Advanced Chips
- 15.3 The Economics of Scaling — Why Smaller No Longer Means Cheaper
- 15.4 Chiplets — The Modular Revolution
- 15.5 Advanced Packaging Economics — The New Critical Path
- 15.6 Beyond Silicon — What’s Next After Moore’s Law?
- Key Takeaways
- What We Learned
- References and Further Reading
Chapter 16: Quantum Computing
- 16.0 Introduction
- 16.1 Quantum Computing Basics
- 16.2 The Qubit Technology Landscape
- 16.3 Quantum Error Correction: The Heart of the Challenge
- 16.4 The NISQ Era and Beyond
- 16.5 The Cryptography Threat
- 16.6 China’s Quantum Program: A Detailed Assessment
- 16.7 The Investment Landscape
- 16.8 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 17: The Energy Crisis
- 17.0 Introduction
- 17.1 The AI Energy Problem
- 17.2 Data Center Power Consumption
- 17.3 Cooling: The Hidden Challenge
- 17.4 Data Center Location Strategy
- 17.5 Green Computing: Renewable and Low-Carbon Solutions
- 17.6 Nuclear Power for AI: The New Frontier
- 17.7 The Policy Response
- 17.8 The AI Energy Debate: Estimates and Controversies
- 17.9 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 18: Edge Computing
- 18.0 Introduction
- 18.1 What Is Edge Computing?
- 18.2 The Rise of the NPU
- 18.3 AI PCs: The New Hardware Category
- 18.4 Automotive Computing: The Demanding Edge
- 18.5 TinyML and the Internet of Intelligent Things
- 18.6 Hybrid Architectures: The Real Future
- 18.7 Challenges and Counterpoints
- 18.8 What We Learned
- Key Takeaways
- References and Further Reading
Chapter 19: The Computing Power Economy
- 19.0 Introduction
- 19.1 Computing Power as Economic Infrastructure
- 19.2 Supply Chain Concentration: The Hidden Risk
- 19.3 The US-CHIPS Act and Global Responses
- 19.4 The US-China Chip War: Geopolitical Dynamics
- 19.5 Cloud Market Dynamics
- 19.6 Computing as a Utility: The Next Frontier
- 19.7 The Talent and Knowledge Economy
- 19.8 The Macroeconomic Impact
- 19.9 What We Learned
- Key Takeaways
- References and Further Reading