LiteRT.js: High-Performance On-Device AI Inference in the Browser

The Definitive Guide to Running Machine Learning Models Locally with WebAssembly, WebGPU, and WebNN

Introduction

1. What This Book Covers
2. What This Book Is Not
3. How to Use This Book
4. Prerequisites
5. A Note on Versions

Chapter 1: The Web AI Revolution – Why LiteRT.js Matters

1. The Browser as an AI Platform
2. From TensorFlow.js to LiteRT.js: A Generational Shift
3. What Is LiteRT? The Unified On-Device Runtime
4. The Value Proposition: Privacy, Performance, and Cost
5. When to Use (and Not Use) LiteRT.js
6. The Landscape of Browser AI

Chapter 2: Architecture Deep Dive

1. The JS-to-WebAssembly Bridge Pattern
2. Model Loading and Compilation Pipeline
3. Tensor Buffer Management and Memory Layout
4. Accelerator Delegation System
5. SignatureRunner and Named Entry Points

Chapter 3: Getting Started – Installation, Initialization, and First Inference

1. Installing @litertjs/core
2. Serving WebAssembly Binaries
3. Runtime Initialization with loadLiteRt
4. Loading and Compiling Your First Model
5. Running Inference and Reading Results

Chapter 4: Hardware Acceleration – WebGPU, WebNN, and XNNPACK

1. WebGPU: GPU-Accelerated Inference
2. WebNN: Neural Processing Unit Access
3. XNNPack on WebAssembly: Optimized CPU Execution
4. Accelerator Selection Strategy and Fallback Patterns
5. JSPI (JavaScript Promise Integration) for Asynchronous Execution

Chapter 5: Model Conversion – From PyTorch, TensorFlow, and JAX to .tflite

1. The .tflite FlatBuffer Format
2. Converting PyTorch Models with litert_torch
3. Converting TensorFlow and JAX Models
4. Ultralytics YOLO Export Integration
5. Conversion Troubleshooting and Common Pitfalls

Chapter 6: Model Optimization – Quantization, Size Reduction, and Performance Tuning

1. Understanding Quantization: INT8, INT4, FP16
2. Dynamic, Static, and Weight-Only Quantization
3. Using the AI Edge Quantizer
4. Selective Quantization and Mixed-Precision Strategies
5. Model Size vs. Accuracy Trade-offs

Chapter 7: The Complete API Reference

1. Core Runtime Functions (loadLiteRt, loadAndCompile)
2. Tensor Class: Creation, Manipulation, and Transfer
3. CompiledModel and SignatureRunner APIs
4. Configuration Options and CompileOptions
5. Error Handling and Debugging Utilities

Chapter 8: Memory Management and Performance Engineering

1. Manual Memory Management: Why delete() Matters
2. GPU-to-CPU Data Transfer Patterns
3. Benchmarking Inference Performance
4. Profiling and Diagnosing Bottlenecks
5. Production Optimization Checklist

Chapter 9: Integration with Modern Web Frameworks

1. React Integration (react-litert and Custom Hooks)
2. Vue.js Composition API Integration
3. Angular Services and Components
4. Svelte Stores and Reactive Inference
5. Next.js and Vite Build Configuration for WASM

Chapter 10: Computer Vision Applications

1. Image Classification with MobileNet and ResNet
2. Real-Time Object Detection with YOLO
3. Semantic Segmentation Pipelines
4. Depth Estimation and 3D Point Clouds
5. Image Upscaling with Real-ESRGAN

Chapter 11: Audio, Speech, Text, and Multimodal AI

1. Audio Processing and Noise Suppression
2. Speech Recognition Patterns in the Browser
3. Text Embeddings and On-Device RAG
4. OCR and Document Intelligence
5. LLM Inference with LiteRT-LM.js

Chapter 12: Production Deployment – Security, Compatibility, and Best Practices

1. Security Model and Threat Considerations
2. Browser Compatibility and Feature Detection
3. Progressive Enhancement and Graceful Degradation
4. Migrating from TensorFlow.js
5. Production Deployment Checklist

Conclusion

References