LiteRT.js: High-Performance On-Device AI Inference in the Browser

The Definitive Guide to Running Machine Learning Models Locally with WebAssembly, WebGPU and WebNN

Introduction

1. What This Book Covers
2. What This Book Is Not
3. How to Use This Book
4. Prerequisites
5. A Note on Versions

Chapter 1: The Web AI Revolution – Why LiteRT.js Matters

1. The Browser as an AI Platform
2. From TensorFlow.js to LiteRT.js: A Generational Shift
3. What Is LiteRT? The Unified On-Device Runtime
4. The Value Proposition: Privacy, Performance and Cost
5. When to Use (and Not Use) LiteRT.js
6. The Landscape of Browser AI

Chapter 2: Architecture Deep Dive

1. The JS-to-WebAssembly Bridge Pattern
2. Model Loading and Compilation Pipeline
3. Tensor Buffer Management and Memory Layout
4. Accelerator Delegation System
5. SignatureRunner and Named Entry Points

Chapter 3: Getting Started – Installation, Initialization and First Inference

1. Installing @litertjs/core
2. Serving WebAssembly Binaries
3. Runtime Initialization with loadLiteRt
4. Loading and Compiling Your First Model
5. Running Inference and Reading Results

Chapter 4: Hardware Acceleration – WebGPU, WebNN and XNNPACK

1. WebGPU: GPU-Accelerated Inference
2. WebNN: Neural Processing Unit Access
3. XNNPack on WebAssembly: Optimized CPU Execution
4. Accelerator Selection Strategy and Fallback Patterns
5. JSPI (JavaScript Promise Integration) for Asynchronous Execution

Chapter 5: Model Conversion – From PyTorch, TensorFlow and JAX to .tflite

1. The .tflite FlatBuffer Format
2. Converting PyTorch Models with litert_torch
3. Converting TensorFlow and JAX Models
4. Ultralytics YOLO Export Integration
5. Conversion Troubleshooting and Common Pitfalls

Chapter 6: Model Optimization – Quantization, Size Reduction and Performance Tuning

1. Understanding Quantization: INT8, INT4, FP16
2. Dynamic, Static and Weight-Only Quantization
3. Using the AI Edge Quantizer
4. Selective Quantization and Mixed-Precision Strategies
5. Model Size vs. Accuracy Trade-offs

Chapter 7: The Complete API Reference

1. Core Runtime Functions (loadLiteRt, loadAndCompile)
2. Tensor Class: Creation, Manipulation and Transfer
3. CompiledModel and SignatureRunner APIs
4. Configuration Options and CompileOptions
5. Error Handling and Debugging Utilities

Chapter 8: Memory Management and Performance Engineering

1. Manual Memory Management: Why delete() Matters
2. GPU-to-CPU Data Transfer Patterns
3. Benchmarking Inference Performance
4. Profiling and Diagnosing Bottlenecks
5. Production Optimization Checklist

Chapter 9: Integration with Modern Web Frameworks

1. React Integration (react-litert and Custom Hooks)
2. Vue.js Composition API Integration
3. Angular Services and Components
4. Svelte Stores and Reactive Inference
5. Next.js and Vite Build Configuration for WASM

Chapter 10: Computer Vision Applications

1. Image Classification with MobileNet and ResNet
2. Real-Time Object Detection with YOLO
3. Semantic Segmentation Pipelines
4. Depth Estimation and 3D Point Clouds
5. Image Upscaling with Real-ESRGAN

Chapter 11: Audio, Speech, Text and Multimodal AI

1. Audio Processing and Noise Suppression
2. Speech Recognition Patterns in the Browser
3. Text Embeddings and On-Device RAG
4. OCR and Document Intelligence
5. LLM Inference with LiteRT-LM.js

Chapter 12: Production Deployment – Security, Compatibility and Best Practices

1. Security Model and Threat Considerations
2. Browser Compatibility and Feature Detection
3. Progressive Enhancement and Graceful Degradation
4. Migrating from TensorFlow.js
5. Production Deployment Checklist

Conclusion

References