Leanpub Header

Skip to main content

AI GPU INFRASTRUCTURE AND DATA CENTER ARCHITECTURE

From GPU Servers and High-Speed Networking to AI Clusters, Cloud Infrastructure, and Large-Scale Model Deployment

AI GPU INFRASTRUCTURE AND DATA CENTER ARCHITECTURE
This book is 100% completeLast updated on 2026-11-21

Discover the infrastructure behind modern AI systems. Learn GPU server architecture, high-speed networking, distributed training, large-scale inference, cloud GPU platforms, data center power and cooling, Kubernetes scheduling, and infrastructure operations through practical explanations, coding exercises, case studies, and architecture projects.

Minimum price

$12.99

$19.99

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
About

About

About the Book

AI GPU Infrastructure and Data Center Architecture

From GPU Servers and High-Speed Networking to AI Clusters, Cloud Infrastructure, and Large-Scale Model Deployment

AI GPU Infrastructure and Data Center Architecture is a practical guide to the computing systems that support modern artificial intelligence. It explains how GPU servers, high-speed interconnects, data center networks, distributed computing frameworks, storage systems, power and cooling infrastructure, and cloud platforms work together to train and deploy AI models.

The book begins with the foundations of AI computing infrastructure and GPU parallel processing. It then explores AI server architecture, single-GPU and multi-GPU configurations, GPU-to-GPU communication, networking fabrics, and distributed training. Readers develop a system-level understanding of how computing resources are connected, coordinated, and scaled to support demanding AI workloads.

The later chapters focus on AI inference infrastructure, including batching, latency, throughput, concurrency, model serving, and large language model deployment. Dedicated chapters explain data center power distribution, rack-level power planning, air and liquid cooling, thermal monitoring, energy efficiency, and Power Usage Effectiveness (PUE).

The book also introduces public, private, and hybrid GPU cloud environments, GPU virtualization, containers, Kubernetes-based scheduling, cloud provisioning, and multi-tenant resource management. Reliability, security, monitoring, observability, capacity planning, and incident management are discussed as essential parts of operating production AI infrastructure.

Practical coding examples, exercises, planning activities, case studies, and architecture projects help connect the concepts with real infrastructure decisions. Topics include GPU performance benchmarking, cluster capacity planning, network analysis, shared GPU cloud planning, and the design of AI infrastructure architectures.

Who Should Read This Book?

This book is intended for BCA, MCA, Computer Science, Information Technology, and Data Science students; researchers and educators; AI/ML engineers; software engineers; system administrators; cloud practitioners; and technology professionals interested in GPU computing and AI data center systems.

How to Use This Book

Read the chapters in sequence to build a complete system-level understanding, or consult individual chapters when planning a GPU server, network fabric, distributed training cluster, inference service, or cloud deployment. Use the practical examples and exercises to reinforce the concepts. Always validate hardware specifications, software versions, cloud pricing, and deployment requirements against current vendor documentation.

Scope and Technical Note

The book concentrates on system-level infrastructure for AI training and deployment, including servers, interconnects, networking, distributed computing, power, cooling, cloud platforms, and operations. Semiconductor fabrication, package engineering, and chip-design algorithms are outside its main scope.

Author

About the Author

Anshuman Mishra

Anshuman Kumar Mishra, M.Tech (Computer Science) Assistant Professor, Doranda College, Ranchi University

Prolific Author of 50+ Books on AI, Machine Learning & Computer Science | 20+ Years Experience

Anshuman Kumar Mishra is a dedicated educator, researcher, and highly prolific author with over 20 years of experience in Computer Science and Information Technology. Holding an M.Tech in Computer Science from BIT Mesra, he brings a rare combination of academic depth and practical teaching expertise.

Currently serving as Assistant Professor at Doranda College under Ranchi University, he has mentored thousands of students, helping them build strong foundations in programming, data science, and artificial intelligence. His student-centric teaching style emphasizes conceptual clarity, hands-on practice, and real-world application.

Anshuman is a prolific author with more than 50 books published across a wide spectrum of computer science and emerging technology domains. From foundational programming languages to advanced topics in Artificial Intelligence, Machine Learning, Reinforcement Learning, Decision Theory, and Computer Vision — his books are widely appreciated by students, educators, and professionals for their clear explanations, strong theoretical foundation, and practical approach.

His extensive body of work reflects his deep commitment to making complex subjects accessible and meaningful for learners at all levels. He is particularly recognized for creating well-structured learning paths that help readers progress from beginner to advanced levels with confidence.

Driven by the mission to democratize quality technical education, Anshuman continues to write and update books that bridge the gap between academic theory and industry practice.

When not teaching or writing, he actively follows and explores new developments in AI, Quantum Machine Learning, and Ethical Intelligence systems.

Contents

Table of Contents

TABLE OF CONTENTS About the Book 2 Preface 3 Learning Outcomes 3 Copyright 3 Disclaimer 4 About the Author 4 Chapter 1: Foundations of AI Computing Infrastructure 7 1.1 Evolution of AI Computing Systems 7 1.2 Training Versus Inference Infrastructure 10 1.3 Workloads of Modern AI Systems 13 1.4 Infrastructure Requirements of Large AI Models 15 1.5 AI Computing Ecosystem 19 Chapter 2: GPU Computing and Parallel Workloads 27 2.1 CPU and GPU Workload Characteristics 27 2.2 Parallel Processing Fundamentals 29 2.3 SIMT Execution Model 32 2.4 GPU Execution and Scheduling Concepts 34 2.5 GPU Computing Software Ecosystems 36 2.6 Measuring GPU Workload Performance 39 Chapter 3: AI GPU Server Architecture 45 3.1 Components of an AI Server 45 3.2 Single-GPU and Multi-GPU Servers 48 3.3 Host Processors and GPU Coordination 51 3.4 PCI Express and Device Connectivity 53 3.5 Server Configuration and Compatibility 56 3.6 GPU Server Deployment Models 58 Chapter 4: GPU Interconnects and Distributed Computing 65 4.1 GPU-to-GPU Communication 65 4.2 NVLink and NVSwitch Concepts 67 4.3 PCIe-Based Communication 69 4.4 Collective Communication Operations 70 4.5 Distributed Training Communication Patterns 73 4.6 Communication Bottlenecks 75 Chapter 5: Networking for AI Data Centers 82 5.1 Data Center Network Architecture 82 5.2 Ethernet and InfiniBand 84 5.3 Network Topologies and Fabric Design 85 5.4 RDMA and Remote Memory Access 87 5.5 Congestion Management and Network Latency 89 5.6 Network Performance Evaluation 91 Chapter 6: Scaling AI Training Across GPU Clusters 97 6.1 Single-Node and Multi-Node Training 97 6.2 Data, Tensor, and Pipeline Parallelism 100 6.3 Distributed Training Frameworks 104 6.4 Cluster Scheduling and Resource Allocation 107 6.5 Checkpointing and Fault Recovery 109 6.6 Scaling Efficiency Analysis 113 Chapter 7: AI Inference Infrastructure 118 7.1 Inference Workload Characteristics 118 7.2 Batch Processing and Continuous Batching 120 7.3 Latency, Throughput, and Concurrency 122 7.4 Model Serving Architectures 124 7.5 Inference Optimization and Quantization 127 7.6 Serving Large Language Models at Scale 129 Chapter 8: AI Data Center Power and Cooling 139 8.1 Power Distribution Fundamentals 139 8.2 Rack-Level Power Budgets 141 8.3 Air Cooling and Liquid Cooling 142 8.4 Thermal Monitoring and Management 144 8.5 Energy Efficiency and PUE 146 8.6 Sustainable AI Infrastructure 147 8.7 Practical Coding Examples for Power and Cooling 151 8.8 Coding Exercises 156 Chapter 9: Cloud GPU Infrastructure 157 9.1 Public, Private, and Hybrid AI Clouds 158 9.2 GPU Virtualization and Isolation 161 9.3 Containers and GPU Workloads 163 9.4 Kubernetes-Based GPU Scheduling 165 9.5 Cloud GPU Provisioning and Cost Models 168 9.6 Multi-Tenant Infrastructure Management 171 9.7 Practical Python Programs and Cloud GPU Calculations 173 9.8 Practical Lab: Plan a Small Shared GPU Cloud 179 Chapter 10: Reliability, Security, and Operations 184 10.1 Infrastructure Monitoring and Telemetry 185 10.2 Hardware and Network Failure Detection 188 10.3 Observability and Performance Diagnostics 191 10.4 AI Infrastructure Security 194 10.5 Capacity Planning and Availability 198 10.6 Operational Incident Management 202 Chapter 11: AI Infrastructure Case Studies 211 11.1 Building a Small GPU Research Cluster 212 11.2 Designing Infrastructure for LLM Training 215 11.3 Planning an AI Inference Service 218 11.4 Comparing Cloud and On-Premises Deployments 220 11.5 Infrastructure Cost and Performance Analysis 223 Chapter 12: Practical Projects and Future Directions 233 12.1 GPU Performance Benchmarking 234 12.2 Cluster Capacity Planning Worksheet 236 12.3 Network Performance Analysis 238 12.4 AI Infrastructure Architecture Project 241 12.5 Emerging Trends in AI Computing Infrastructure 244

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub