Leanpub Header

Skip to main content

The Local AI Stack: Building a Sovereign Machine Learning Workstation with Hyper-V, WSL2, and GPU Virtualization

The Local AI Stack: Building a Sovereign Machine Learning Workstation with Hyper-V, WSL2, and GPU Virtualization
This book is 100% completeLast updated on 2026-08-27

Cloud GPU rental costs more than compute time. It can also introduce recurring expenses, external dependencies, and additional questions about where your prompts, datasets, and model artifacts are processed. A local machine learning workstation gives you direct control over the hardware, software stack, and data workflows. The Local AI Stack is a technical engineering reference for building a Windows-based machine learning workstation with Linux environments, NVIDIA GPU acceleration, and virtualization. Across 140 pages, it follows the stack from firmware configuration and Windows virtualization features to CUDA-enabled containers, model inference, storage optimization, and system recovery.

Minimum price

$18.00

$18.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
140
Pages
Discussion Forum
About

About

About the Book

The Local AI Stack: Building a Sovereign Machine Learning Workstation with Hyper-V, WSL2, and GPU Virtualization

Cloud GPU rental costs more than compute time. It can also introduce recurring expenses, external dependencies, and additional questions about where your prompts, datasets, and model artifacts are processed. A local machine learning workstation gives you direct control over the hardware, software stack, and data workflows. The Local AI Stack is a technical engineering reference for building a Windows-based machine learning workstation with Linux environments, NVIDIA GPU acceleration, and virtualization. Across 140 pages, it follows the stack from firmware configuration and Windows virtualization features to CUDA-enabled containers, model inference, storage optimization, and system recovery.

From Hardware to CUDA Workloads

The book covers the core components of a local AI environment: CPU virtualization capabilities, BIOS/UEFI settings, Hyper-V, Windows Subsystem for Linux 2, Linux distributions, NVIDIA drivers, CUDA, and deep-learning libraries. You will explore WSL2 resource management through .wslconfig, CPU and memory limits, swap placement, virtual disk storage, and access to datasets across Windows and Linux. The storage chapters examine when to use DrvFs mounts, dedicated VHDX files, and native Linux filesystems for demanding workloads. GPU acceleration receives particular attention. The guide covers NVIDIA driver configuration, GPU access from WSL2, CUDA Toolkit and cuDNN compatibility, and GPU-enabled containers using Docker Desktop and the NVIDIA Container Toolkit. It also examines GPU paravirtualization, GPU partitioning, and Discrete Device Assignment for Hyper-V environments, with the understanding that availability and configuration depend on the hardware, drivers, Windows edition, and virtualization setup.

Building a Usable Local AI Environment

Beyond the virtualization layer, the book moves into the tools needed to develop, train, and run machine learning workloads. Topics include Python environment management with Conda or Mamba, PyTorch and TensorFlow installation, VS Code Remote – WSL, JupyterLab, WSLg, Hugging Face model management, Git LFS, and local model runtimes such as Ollama and LM Studio. The infrastructure chapters address virtual networking, SSH access, PowerShell automation, performance monitoring, backup and recovery, reproducible version tracking, and Windows Firewall hardening. You will also find guidance on benchmarking a local language model and documenting the configuration needed to reproduce your results. The goal is to help you understand how the layers interact, identify configuration constraints, and build a workstation suited to your own hardware and workload.

Who This Book Is For

This reference is intended for machine learning engineers building local training and inference environments, AI researchers who want to use their own hardware, infrastructure specialists designing locally controlled AI systems, and Windows practitioners who need Linux and CUDA without abandoning their existing operating system. You should be comfortable using the command line, understand basic Linux concepts, and be able to navigate BIOS/UEFI settings. This is a technical reference for readers who want to configure and understand the stack—not a beginner's click-by-click introduction.

PDF and TXT Editions

The book is delivered in PDF and plain-text formats. Use the PDF as a structured technical reference, or process the TXT edition with local search tools, scripts, indexing pipelines, or your own locally running language model. The text edition is particularly useful for building a private technical corpus: you can grep it, index it, or incorporate it into a local retrieval workflow without needing to send the book's contents to a cloud-based AI service.

Part of the Sovereign Product Line

The Local AI Stack complements Sovereign Intelligence, The Quantization Black Book, and The Dark Mesh. Together, these titles explore different layers of a locally controlled technology environment: AI architecture, model quantization, workstation infrastructure, and off-grid networking. The common principle is practical control over the systems you build, the models you run, and the infrastructure on which your work depends.

Engineering Responsibility

This material is an engineering reference and architectural roadmap informed by Microsoft, NVIDIA, and Linux documentation. The exact procedures and supported configurations can change with Windows releases, drivers, GPU models, and virtualization features. Verify compatibility against your own environment before applying system-level changes. Readers are responsible for adapting configurations to their hardware, following applicable software licenses, and complying with the laws of their jurisdiction. Build locally. Understand every layer. Keep control of your stack.

Table of Contents

1. Research hardware requirements including SLAT-capable CPUs and NVIDIA Tensor Core GPUs for AI workloads.

2. Enable hardware virtualization (VT-x/AMD-V) and IOMMU support within the BIOS/UEFI settings.

3. Activate the Hyper-V feature set via the Windows Features dialog or PowerShell Enable-WindowsOptionalFeature.

4. Install the Virtual Machine Platform and Windows Subsystem for Linux (WSL) optional components.

5. Configure the system to use WSL 2 as the default version for all future Linux distribution installs.

6. Download and install the latest Linux kernel update package for Windows from Microsoft.

7. Install a high-performance Linux distribution like Ubuntu 22.04 LTS from the Microsoft Store or via CLI.

8. Set up the .wslconfig file in the user profile to manage global resource limits for all WSL 2 instances.

9. Allocate specific CPU core counts and memory limits within .wslconfig to prevent host OS starvation.

10. Implement custom swap file locations on high-speed NVMe storage to improve virtualization paging.

11. Install the NVIDIA Windows GPU Driver (Game Ready or Studio) to support GPU Paravirtualization.

12. Download and install the NVIDIA Container Toolkit to allow Docker to interface with the GPU in WSL 2.

13. Install Docker Desktop and configure it to use the WSL 2 backend for improved performance.

14. Alternatively, install Podman or Rancher Desktop for an open-source containerization approach.

15. Configure the NVIDIA runtime in the Docker daemon.json file to ensure container access to CUDA.

16. Verify GPU acceleration inside WSL 2 using the nvidia-smi command to check for driver connectivity.

17. Install the CUDA Toolkit inside the Linux distribution, ensuring version compatibility with host drivers.

18. Configure cuDNN libraries within the virtualized Linux environment for optimized deep learning.

19. Optimize VHDX storage by moving the WSL 2 virtual disk to a dedicated non-system partition.

20. Use the wsl --mount command to attach physical disks directly to the Linux environment for raw I/O.

21. Configure Hyper-V Virtual Switches (Internal vs. External) to manage network isolation for AI labs.

22. Implement Discrete Device Assignment (DDA) for passing through an entire GPU to a Hyper-V VM.

23. Explore GPU Partitioning (GPU-PV) scripts to share a single GPU across multiple Hyper-V VMs.

24. Setup Miniconda or Mamba within the WSL 2 environment for isolated Python environment management.

25. Install PyTorch and TensorFlow with specific CUDA support flags to leverage hardware acceleration.

26. Configure VS Code with the Remote - WSL extension for a seamless Windows-to-Linux development flow.

27. Set up Jupyter Lab servers within WSL 2 and map ports to access them via the Windows browser.

28. Enable WSLg (WSL GUI) to run Linux-native visualization tools like TensorBoard or Matplotlib windows.

29. Implement Nested Virtualization to allow running Hyper-V or Docker containers within a Hyper-V VM.

30. Adjust Windows Defender exclusions for WSL 2 project folders to improve file system performance.

31. Configure SSH keys for secure remote access to the Windows-centric AI virtualization stack.

32. Use PowerShell to automate the creation and teardown of Hyper-V VMs for ephemeral AI training tasks.

33. Setup a local model repository using Ollama or LM Studio within the virtualized layer.

34. Integrate Hugging Face CLI tools for efficient model weight management across environments.

35. Configure Git LFS in the Linux layer to handle large AI model files and datasets.

36. Optimize network throughput using the Mirrored network mode in WSL 2 (experimental features).

37. Setup a bridge network for Hyper-V VMs to allow them to act as independent nodes on the local network.

38. Monitor system performance using Windows Performance Monitor (PerfMon) and Linux htop/nvtop.

39. Implement automated backups of WSL 2 distributions using the wsl --export command.

40. Create a recovery plan for VHDX corruption or host system failures.

41. Fine-tune the Windows kernel power management settings to 'High Performance' for consistent AI throughput.

42. Test the stack by running a sample LLM (Large Language Model) inference benchmark.

43. Document the specific versions of CUDA, cuDNN, and drivers used to ensure stack reproducibility.

44. Evaluate the use of Windows Sandbox for isolated testing of untrusted AI libraries or scripts.

45. Finalize the production-ready AI stack by hardening network ports and user permissions within the Windows Firewall.

Author

About the Author

Krzysztof Rybiński

I am an independent technology developer and systems engineer who built my technical path largely through self-directed engineering, experimentation, and continuous learning outside a traditional academic or corporate technology career.

My professional background began far from the technology industry. I spent years working in manufacturing, while independently developing my knowledge of software engineering, computer systems, and advanced computing. Over time, that self-directed work evolved into a broad technical practice spanning autonomous AI, cybersecurity, systems programming, GPU computing, automation, and advanced computational architectures.

Today, I design, build, and publish projects involving agentic AI, autonomous defense systems, SIEM/EDR integration, secure software architecture, C/C++, Go, Python, CUDA, quantum computing, cryptography, and privacy-oriented local AI infrastructure.

I approach technology from a systems perspective — from low-level software, memory architecture, and GPU performance to distributed systems, intelligent agents, and high-assurance security architectures.

I also explore aerospace and high-assurance software concepts, including safety-critical architectures, multi-level security, cross-domain solutions, and advanced computational systems.

Alongside active development, I publish long-form engineering projects covering AI, cybersecurity, cloud engineering, quantum computing, GPU programming, cryptography, automation, blockchain, and aerospace engineering.

My current focus is on autonomous software agents, privacy-first local infrastructure, advanced computing, and reliable systems designed to operate with a high degree of independence.

I am open to opportunities involving AI engineering, cybersecurity, software engineering, autonomous systems, HPC/GPU computing, and advanced technology development.

https://businessofmachines.blogspot.com/

https://learn.microsoft.com/en-us/users/machinadeusex/

https://g.dev/machinadeusex

https://github.com/porucznikswext-source

https://www.linkedin.com/in/krzysztof-r-93a37b287/

Get the free sample chapters

Click the buttons to get the free sample in PDF or EPUB, or read the sample online here

Also by the Author

Also by the Author

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub