[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6,"_7":-5,"_8":9,"_10":11},"actionData","errors","root",{"_455":-5,"_462":-5,"_463":464,"_465":466,"_467":468,"_469":470,"_471":472,"_473":474,"_475":301,"_476":477},"routes/_store+/_readers+/read_/route","routes/_store+/_readers+/read_.$slug/route",{"_16":295,"_296":297,"_298":299,"_300":301,"_302":-7},"routes/_store+/_readers+/read_.$slug.$chapterId/route",{"_12":13,"_14":15,"_16":17,"_18":19,"_20":21},"title","Practical Python Artificial Intelligence Programming","content","

Part IV - Overviews of Image Generation, Reinforcement Learning, and Recommendation Systems

This final part of this book consists of overviews of three important topics that I cover briefly, with perhaps more material added in the next edition of this book.

","book",{"_33":34,"_12":13,"_35":36,"_37":38,"_39":40,"_41":42,"_43":15},"nextChapter",{"_22":23,"_12":24,"_25":-5,"_26":27,"_28":29,"_30":-5,"_31":-5,"_32":-5},"sections",[],"level",2,"Overview of Image Generation","description","path","#overview-of-image-generation","fileCount",1,"quizType","sectionNum","youtubeId","previewContent","

\nCover Material, Copyright, and License\n
\nPreface\n
- \nAbout the Author\n
- \nUsing the Example Code\n
- \nBook Cover\n
- \nAcknowledgements\n
\n
\nPython Development Environment\n
- \nManaging Python with uv\n
\n
\nPart 1 - Machine Learning\n
\n“Classic” Machine Learning\n
- \nExample Material\n
- \nClassification Models using Scikit-learn\n
- \nClassic Machine Learning Wrap-up\n
\n
\nRegression and Clustering\n
- \nRegression: Predicting Housing Prices\n
- \nClustering: Discovering Groups in Data\n
- \nRegression and Clustering Wrap-up\n
\n
\nExploratory Data Analysis and Feature Engineering\n
- \nExploratory Data Analysis\n
- \nFeature Engineering\n
- \nEDA and Feature Engineering Wrap-up\n
\n
\nPart 2 - Deep Learning\n
\nThe Basics of Deep Learning\n
- \nUsing PyTorch for Building a Cancer Prediction Model\n
\n
\nNatural Language Processing Using Deep Learning\n
- \nHugging Face and the Transformers Library\n
- \nComparing Sentences for Similarity Using Transformer Models\n
- \nDeep Learning Natural Language Processing Wrap-up\n
\n
\nPart IV - Overviews of Image Generation, Reinforcement Learning, and Recommendation Systems\n
\nOverview of Image Generation\n
- \nImage Generation Using Stable Diffusion and PyTorch\n
- \nMini-DALL·E: A Lightweight Alternative\n
- \nRecommended Reading for Image Generation\n
\n
\nOverview of Reinforcement Learning (Optional Material)\n
- \nOverview\n
- \nAvailable RL Tools\n
- \nAn Introduction to Markov Decision Process\n
- \nA Concrete Example: Q-Learning with Gymnasium\n
- \nReinforcement Learning Wrap-up\n
\n
\nOverview of Recommendation Systems (Optional Material)\n
- \nTensorFlow Recommenders\n
- \nRecommendation Systems Wrap-up\n
\n
\nPart 3 - Large Language Models\n
\nIntroduction to Transformers and Large Language Models\n
- \nThe Transformer Architecture\n
- \nTokenization\n
- \nFrom Transformers to Large Language Models\n
- \nKey Capabilities of Modern LLMs\n
- \nPractical Considerations\n
\n
\nLLMs with Public APIs\n
- \nSetup and Authentication\n
- \nText Generation\n
- \nThinking Models\n
- \nMulti-Turn Conversations\n
- \nMultimodal Input: Analyzing Images\n
- \nWeb Search with LLMs\n
- \nStructured Output\n
- \nPractical Considerations\n
- \nSummary\n
\n
\nLLMs with Local Models\n
- \nInstalling Ollama\n
- \nDownloading and Running Models\n
- \nUsing Ollama from Python\n
- \nReasoning with Local Models\n
- \nConversation Memory with Ollama\n
- \nPrompt Caching for Performance\n
- \nOpenAI-Compatible API\n
- \nAlternative Tools for Running Local Models\n
- \nHardware Considerations\n
- \nSummary\n
\n
\nPart 4 - Symbolic AI and Knowledge Representation\n
\nSymbolic AI\n
- \nComparison of Symbolic AI and Deep Learning\n
- \nImplementing Frame Data Structures in Python\n
- \nUse Predicate Logic by Calling Swi-Prolog\n
- \nSwi-Prolog and Python Deep Learning Interop\n
- \nSoar Cognitive Architecture\n
- \nConstraint Programming with MiniZinc and Python\n
- \nGood Old Fashioned Symbolic AI Wrap-up\n
\n
\nPart 5 - Knowledge Representation\n
\nGetting Setup To Use Graph and Relational Databases\n
- \nQuerying Wikidata with SPARQL and Python\n
- \nThe SQLite Relational Database for Knowledge Representation\n
\n
\nOptional Material: A Deeper Dive Into Semantic Web and Linked Data\n
- \nOverview and Theory\n
\n

Cover Material, Copyright, and License

This eBook will be updated occasionally so please periodically check the leanpub.com web page for this book for updates.

Please visit my website and follow me on social media.

Preface

This book is intended, dear reader, to show you a wide variety of practical AI techniques and examples, and to be a jumping off point when you discover things that interest you or may be useful in your work. A common theme here is covering AI programming tasks that used to be difficult or impossible but are now much simpler using modern technology. I also cover a wide variety of non-deep learning material including a chapter on Symbolic AI that has historic interest and some current practical value.

This book is meant to be a quick four to five hour introduction to AI for Python programmers. If you have experience with Large Language Models, Deep Learning, general Machine Leaning and Symbolic AI then you can spend a couple of hours experimenting with the examples.

This book is not intended as a textbook that is to be read start to finish. The parts or even chapters of this book can be read in any order. I have tried making each chapter independent and able to stand on its own even when I am building on earlier themes in the book.

I try to update my books a few times a year so when purchasing on Leanpub please indicate that you want to be notified when new editions are available. Updates to new editions are free for my Leanpub books.

My career developing AI applications and tools began in 1982. Until the advent of breakthroughs in deep learning around 2010 most of my development work was in Common Lisp, Java, and C++. My language preference changed when I started spending most of my time creating deep learning models. Python has the most tooling, libraries, and frameworks for deep learning so as a practical matter I have adopted Python as a primary programming language. That said I still also heavily use Common Lisp, Haskell, Swift, and Scheme. I recommend not having an “always use one programming language” mindset.

Why this book? Some of what I cover here has already been covered in the Common Lisp, Java, Clojure and Haskell artificial intelligence books I have previously written. My goal here is to prioritize more material material for Python developers.

Why Python? Python is a very high level language that is easily readable by other programmers. Since Python is one of the most popular programming languages there are many available libraries and frameworks. The best code is code that we don’t have to write ourselves as long as third party code is open source so we can read and modify it if needed. Another reason to use Python, that we lean heavily on in this book, is using pre-trained deep learning models that are wrapped into Python packages and libraries.

About the Author

I have written over 20 books, I have over 50 US patents, and I have worked at interesting companies like Google, Capital One, SAIC, Mind AI, and others. You can read all of my recent books (including this book) for free on my web site https://markwatson.com. If I had to summarize my career the short take would be that I have had a lot of fun and enjoyed my work. I hope that what you learn here will be both enjoyable and help you in your work.

If you would like to support my work please consider purchasing my books on Leanpub and star my git repositories that you find useful on GitHub. You can also interact with me on social media on Mastodon and Twitter.

Using the Example Code

The example code that I have written for this book is Apache 2 licensed so feel free to reuse it. I also use several existing open source packages and libraries in the examples that use liberal-use licenses (I link GitHub repositories, so check the licenses for applicability in your projects).

The examples for this book (and the entire book manuscript!) are in the GitHub repository https://github.com/mark-watson/PythonPracticalAIBook. The code examples are in the directory source-code.

A few of the examples use APIs from Google and OpenAI’s GPT-4. I assume that you have signed up an have access keys that should be available in the environment variables GOOGLE_API_KEY and OPENAI_API_KEY. If you don’t want to sign up for these services I still hope that you enjoy reading the sample code and example output.

Book Cover

I used to live in Sedona Arizona. I have been fortunate to have visited close to one hundred ancient Native American Indian sites in the Verde Valley. I took the cover picture at one of these sites.

This picture shows me and my wife Carol who helps me with book production.

Acknowledgements

I would like to thank my wife Carol Watson who edits all of my books.

I would like to thank the following readers who reported errors or typos in this book: Ryan O’Connor (typo).

Python Development Environment

All of the examples in this book use the tool uv to manage Python library installation and running Python programs.

Here are a few things to consider when setting up your Python development environment:

\nUse uv as your primary tool for managing Python versions, virtual environments, and package dependencies. It is very fast and replaces older tools like pip, venv, and conda.\n
\nUse Git or another version control system to manage your codebase. I will not cover Git here so read a good tutorial if you have not used it before.\n
\nUse an IDE like PyCharm or VSCode, or a text editor like Emacs or Vim — whatever you are comfortable with.\n
\nKeep your environment and dependencies up-to-date.\n
\nAdd comments and documentation to your code. Your “future you” will thank you.\n

Managing Python with uv

I use uv exclusively for managing Python versions and installed libraries. Developed by Astral, uv is an extremely fast Python package and project manager written in Rust. It serves as a drop-in replacement for pip, pip-tools, pipx, poetry, pyenv, virtualenv, and more.

Install uv by following the instructions on the official uv website. On macOS and Linux:

1 curl -LsSf https://astral.sh/uv/install.sh | sh\n

\n\n

Setting Up a Project

Every source code directory in this book follows the same pattern: initialize a project with uv init --bare, add the required libraries with uv add, and run scripts with uv run. Here is a typical setup:

1 # Initialize a new project (creates pyproject.toml)\n2 $ uv init --bare\n3 \n4 # Add libraries the project needs\n5 $ uv add scikit-learn pandas\n6 \n7 # Run a script — uv creates a virtual environment automatically\n8 $ uv run my_script.py\n

\n\n

The uv run command handles everything: it creates a virtual environment if one does not exist, installs the dependencies listed in pyproject.toml, and runs your script. You never need to manually activate a virtual environment.

Managing Python Versions

uv also acts as a Python version manager. If you specify a Python version that is not installed on your system, uv will automatically download and install it:

1 $ uv python install 3.12\n

\n\n

Code Formatting (Optional)

I recommend black for automatic code formatting and isort for import sorting:

1 uv pip install black isort\n2 black *.py\n3 isort *.py\n

\n\n

Part 1 - Machine Learning

This part introduces the foundations of machine learning with Python. We cover the core concepts and algorithms that every practitioner should understand: supervised and unsupervised learning, classification, regression, and clustering.

We start with a survey of machine learning approaches and work through practical examples using scikit-learn. We then cover regression and clustering in depth, followed by a chapter on exploratory data analysis and feature engineering — the often-overlooked skills that determine whether a model succeeds or fails in practice.

“Classic” Machine Learning

“Classic” Machine Learning (ML) is a broad field that encompasses a variety of algorithms and techniques for learning from data. These techniques are used to make predictions, classify data, and uncover patterns and insights. Some of the most common types of classic ML algorithms include:

\nLinear regression: a method for modeling the relationship between a dependent variable and one or more independent variables by fitting a linear equation to the observed data.\n
\nLogistic regression: a method for modeling the relationship between a binary dependent variable and one or more independent variables by fitting a logistic function to the observed data.\n
\nDecision Trees: a method for modeling decision rules based on the values of the input variables, which are organized in a tree-like structure.\n
\nRandom Forest: a method that creates multiple decision trees and averages the results to improve the overall performance of the model\n
\nK-Nearest Neighbors (K-NN): a method for classifying data by finding the K-nearest examples in the training data and assigning the most similar common class among them.\n
\nNaive Bayes: a method for classifying data based on Bayes’ theorem and the assumption of independence between the input variables.\n

We will be covering a very small subset of “classic” ML, and then dive deeper into Deep Learning in later chapters. Deep Learning differs from classic ML in several ways:

\nScale: Classic ML algorithms are typically designed to work with small to medium-sized datasets, while deep learning algorithms are designed to work with large-scale datasets, such as millions or billions of examples.\n
\nArchitecture: Classic ML algorithms have a relatively shallow architecture, with a small number of layers and parameters, while deep learning algorithms have a deep architecture, with many layers and millions or billions of parameters.\n
\nNon-linearity: Classic ML algorithms are sometimes linear, (i.e., the relationship between the input and output is modeled by a linear equation), while deep learning algorithms are non-linear, (i.e., the relationship is modeled by a non-linear function).\n
\nFeature extraction: “Classic” ML requires feature extraction, which is the process of transforming the raw input data into a set of features that can be used by the algorithm. Deep learning can automatically learn features from raw data, so it does not usually require too much separate effort for feature extraction.\n

So, Deep Learning is a subfield of machine learning that is focused on the design and implementation of artificial neural networks with many layers which are capable of learning from large-scale and complex data. It is characterized by its deep architecture, non-linearity, and ability to learn features from raw data, which sets it apart from “classic” machine learning algorithms.

Example Material

Here we cover just a single example of what I think of as “classic machine learning” using the scikit-learn Python library. Later we cover deep learning in three separate chapters. Deep learning models are more general and powerful but it is important to recognize the types of problems that can be solved using the simpler techniques.

The only requirements for this chapter are:

1 uv pip install scikit-learn pandas\n

\n\n

Please note that the content in this book is heavily influenced by what I use in my own work. I mostly use deep learning so its coverage comprises half this book. For this classic machine learning chapter I only use a classification model. I will not be covering regression or clustering models.

We will use the same Wisconsin cancer dataset for both the following classification example and a deep learning classification example in a later chapter. Here are the first few rows of the file labeled_cancer_data.csv:

The last column class indicates the class of the sample, 0 for non-malignant and 1 for malignant. The scikit-learn library has high level and simple to use utilities for reading CSV (spreadsheet) data and for preparing the data for training and testing. I don’t use these utilities here because I am reusing the data loading code from the later deep learning example.

We will use the Pandas library and if you have not used Pandas before you might want to reference the Pandas documentation. Here is a summary of why Pandas is generally useful: it is a popular Python library for data manipulation and analysis that provides data structures and data analysis tools for working with structured data (often spreadsheet data). One of the key data structures in Pandas is the DataFrame, which is a two-dimensional table of data with rows and columns.

A DataFrame is essentially a labeled, two-dimensional array, where each column has a name and each row has an index. DataFrames are similar to tables in a relational database or data in a spreadsheet. They can be created from a variety of data sources such as CSV, Excel, SQL databases, or even Python lists and dictionaries. They can also be transformed, cleaned, and manipulated using a wide range of built-in methods and functionalities.

Pandas DataFrames provide a lot of functionalities to handle and process data. The most common ones are:

\nIndexing: data can be selected by its label or index.\n
\nFiltering: data can be filtered by any condition.\n
\nGrouping: data can be grouped by any column.\n
\nMerging and joining: data can be joined or merged with other data.\n
\nData type conversion: data can be converted to any data type.\n
\nHandling missing data: data can be filled in or removed based on any condition.\n

DataFrames are widely used in data science and machine learning projects for loading, cleaning, processing, and analyzing data. They are also used for data visualization, data preprocessing, and feature engineering tasks.

Listing of load_data.py:

 1 import pandas as pd\n 2 \n 3 \n 4 def load_data():\n 5     train_df = pd.read_csv(\"labeled_cancer_data.csv\")\n 6     test_df = pd.read_csv(\"labeled_test_data.csv\")\n 7 \n 8     train = train_df.to_numpy()\n 9     X_train = train[:, 0:9].astype(float)  # 9 input features\n10     print(\"Number training examples:\", len(X_train))\n11     # Training target: one output (0 for non-malignant, 1 for malignant)\n12     Y_train = train[:, -1].astype(float)\n13 \n14     test = test_df.to_numpy()\n15     X_test = test[:, 0:9].astype(float)\n16     Y_test = test[:, -1].astype(float)\n17     print(\"Number testing examples:\", len(X_test))\n18     return (X_train, Y_train, X_test, Y_test)\n

\n\n

In line 6 we read the CSV training data into a Pandas DataFrame. In line 9 we convert the DataFrame to a NumPy array using the to_numpy() method (preferred over the older .values property). In line 10 we copy all rows of data, skipping the last column (target classification we want to be able to predict) and converting all data to floating point numbers. In line 13 we copy just the last column of the training data array for use as the target classification.

Classification Models using Scikit-learn

Classification is a type of supervised machine learning problem where the goal is to predict the class or category of an input sample based on a set of features. The goal of a classification model is to learn a mapping from the input features to the output class labels.

Scikit-learn (sklearn) is a popular Python library for machine learning that is effective for several reasons (partially derived from the Skikit-learn documentation):

\nScikit-learn provides a consistent and intuitive API for a wide range of machine learning models, which makes it easy to switch between different algorithms and experiment with different approaches.\n
\nScikit-learn includes a large collection of models for supervised, unsupervised, and semi-supervised learning (e.g., linear and non-linear models, clustering, and dimensionality reduction).\n
\nScikit-learn includes a wide range of preprocessing and data transformation tools, such as feature scaling, one-hot encoding, feature extraction, and more, that can be used to prepare and transform the data before training the model.\n
\nScikit-learn provides a variety of evaluation metrics, such as accuracy, precision, recall, F1-score, and more, that can be used to evaluate the performance of a model on a given dataset.\n
\nScikit-learn is designed to be scalable, which means it can handle large datasets and high-dimensional feature spaces. It also provides tools for parallel and distributed computing, which can be used to speed up the training process on large datasets.\n
\nScikit-learn is designed to be easy to use, with clear and concise code, a well-documented API, and plenty of examples and tutorials that help users get started quickly.\n
\nScikit-learn is actively developed and maintained, with regular updates and bug fixes, and a large and active community of users and developers who contribute to the library and provide support through mailing lists, forums, and other channels.\n

All these features make Scikit-learn a powerful and widely used machine learning library for various types of machine learning tasks.

 1 from sklearn.preprocessing import StandardScaler\n 2 from sklearn.neighbors import KNeighborsClassifier\n 3 from sklearn.metrics import classification_report, confusion_matrix\n 4 \n 5 from load_data import load_data\n 6 \n 7 (X_train, Y_train, X_test, Y_test) = load_data()\n 8 \n 9 # Remove mean and scale to unit variance:\n10 scaler = StandardScaler()\n11 X_train = scaler.fit_transform(X_train)\n12 X_test = scaler.transform(X_test)\n13 \n14 # Use the KNN classifier to fit data:\n15 classifier = KNeighborsClassifier(n_neighbors=5)\n16 classifier.fit(X_train, Y_train)\n17 \n18 # Predict y data with classifier:\n19 y_predict = classifier.predict(X_test)\n20 \n21 # Print results:\n22 print(confusion_matrix(Y_test, y_predict))\n23 print(classification_report(Y_test, y_predict))\n

\n\n

Note that we fit the StandardScaler on the training data and then use the same fitted scaler to transform the test data. This is important: scaling the test data with parameters learned from the training set prevents data leakage and ensures a fair evaluation.

We can now train and test the model and evaluate how accurate the model is. In reading the following output, you should understand a few definitions. In machine learning, precision, recall, F1-score, and support are all metrics used to evaluate the performance of a classification model, specifically in regards to binary classification:

\nPrecision: the proportion of true positive predictions out of the total of all positive predictions made by the model. It is a measure of how many of the positive predictions were actually correct.\n
\nRecall: the proportion of true positive predictions out of all actual positive observations in the data. It is a measure of how well the model is able to find all the positive observations.\n
\nF1-score: the harmonic mean of precision and recall. It is a measure of the balance between precision and recall and is generally used when you want to seek a balance between precision and recall.\n
\nSupport: number of observations in each class.\n

These metrics provide an overall view of a model’s performance in terms of both correctly identifying positive observations and avoiding false positive predictions.

 1 $ python classification.py\n 2 Number training examples: 554\n 3 Number testing examples: 15\n 4 [[8 1]\n 5  [0 6]]\n 6               precision    recall  f1-score   support\n 7 \n 8          0.0       1.00      0.89      0.94         9\n 9          1.0       0.86      1.00      0.92         6\n10 \n11     accuracy                           0.93        15\n12    macro avg       0.93      0.94      0.93        15\n13 weighted avg       0.94      0.93      0.93        15\n

\n\n

Classic Machine Learning Wrap-up

I have already admitted my personal biases in favor of deep learning over simpler machine learning and I proved that by using perhaps only 1% of the functionality of Scikit-learn in this chapter.

Regression and Clustering

In the previous chapter we used a K-Nearest Neighbors classifier with scikit-learn. Here we expand our coverage of “classic” machine learning with two more fundamental techniques: regression for predicting continuous values, and clustering for discovering groups in data without labeled targets.

The requirements for this chapter are:

1 uv pip install scikit-learn pandas numpy\n

\n\n

The examples for this chapter are in the directory source-code/regression_and_clustering.

Regression: Predicting Housing Prices

Regression is a type of supervised machine learning where the goal is to predict a continuous numerical value rather than a class label. While classification asks “which category?”, regression asks “how much?” or “how many?”

Common regression algorithms include:

\nLinear Regression: models the relationship between features and target as a straight line (or hyperplane in multiple dimensions).\n
\nPolynomial Regression: extends linear regression by adding polynomial terms, allowing the model to capture non-linear relationships.\n
\nRidge and Lasso Regression: variants of linear regression that add regularization to prevent overfitting.\n

We will use the California Housing dataset, which is built into scikit-learn. It contains 20,640 samples with 8 features describing California census block groups (median income, average number of rooms, location, etc.), with the target being the median house value in units of $100,000s.

Linear Regression

Our first example fits a linear regression model to all 8 features:

 1 import numpy as np\n 2 import pandas as pd\n 3 from sklearn.datasets import fetch_california_housing\n 4 from sklearn.linear_model import LinearRegression\n 5 from sklearn.preprocessing import PolynomialFeatures, StandardScaler\n 6 from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score\n 7 from sklearn.model_selection import train_test_split\n 8 \n 9 \n10 def load_housing_data():\n11     \"\"\"Load the California Housing dataset into a DataFrame.\"\"\"\n12     housing = fetch_california_housing()\n13     df = pd.DataFrame(housing.data, columns=housing.feature_names)\n14     df[\"MedHouseVal\"] = housing.target\n15     return df\n

\n\n

We use train_test_split to hold out 20% of the data for evaluation, scale the features with StandardScaler, and train the model:

 1 df = load_housing_data()\n 2 X = df.drop(\"MedHouseVal\", axis=1)\n 3 y = df[\"MedHouseVal\"]\n 4 \n 5 X_train, X_test, y_train, y_test = train_test_split(\n 6     X, y, test_size=0.2, random_state=42\n 7 )\n 8 \n 9 scaler = StandardScaler()\n10 X_train_scaled = scaler.fit_transform(X_train)\n11 X_test_scaled = scaler.transform(X_test)\n12 \n13 model = LinearRegression()\n14 model.fit(X_train_scaled, y_train)\n15 y_pred = model.predict(X_test_scaled)\n

\n\n

To evaluate a regression model, we use different metrics than classification:

\nMAE (Mean Absolute Error): average magnitude of prediction errors in the same units as the target.\n
\nRMSE (Root Mean Squared Error): like MAE but penalizes larger errors more heavily.\n
\nR² (R-squared): the proportion of variance in the target explained by the model (1.0 = perfect, 0.0 = no better than predicting the mean).\n

Here is the output of running the full regression.py script:

 1 $ python regression.py\n 2 Dataset shape: (20640, 9)\n 3 Target range: 0.15 - 5.00 (units: $100,000s)\n 4 \n 5 === Linear Regression Results ===\n 6   MAE:  0.5332\n 7   RMSE: 0.7456\n 8   R²:   0.5758\n 9 \n10 Feature coefficients (scaled):\n11   MedInc               +0.8544\n12   HouseAge             +0.1225\n13   AveRooms             -0.2944\n14   AveBedrms            +0.3393\n15   Population           -0.0023\n16   AveOccup             -0.0408\n17   Latitude             -0.8969\n18   Longitude            -0.8698\n

\n\n

Because we scaled the features, the coefficients tell us the relative importance of each feature. MedInc (median income) has the largest positive coefficient, confirming that income is the strongest predictor of housing prices. The geographic features (Latitude and Longitude) are also very influential — location matters!

An R² of 0.58 means our simple linear model explains about 58% of the variance in housing prices. Not bad for a single line of model code, but there is room for improvement.

Polynomial Regression

We can capture non-linear relationships by creating polynomial features. For interpretability, we use just two features (MedInc and AveRooms) and generate all degree-2 combinations:

1 poly = PolynomialFeatures(degree=2, include_bias=False)\n2 X_train_poly = poly.fit_transform(X_train_scaled)\n3 X_test_poly = poly.transform(X_test_scaled)\n4 \n5 model = LinearRegression()\n6 model.fit(X_train_poly, y_train)\n

\n\n

The PolynomialFeatures transformer with degree=2 creates five features from our original two: x0, x1, x0², x0·x1, and x1². The interaction term x0·x1 lets the model learn that the relationship between income and price depends on the number of rooms.

1 === Polynomial Regression (degree=2, 2 features) ===\n2   MAE:  0.6163\n3   RMSE: 0.8402\n4   R²:   0.4613\n5 \n6   Polynomial feature names: ['x0', 'x1', 'x0^2', 'x0 x1', 'x1^2']\n

\n\n

With only 2 features, the polynomial model achieves R²=0.46 compared to 0.58 with all 8 features linearly. This illustrates an important tradeoff: polynomial features add expressive power but using too few input features limits the information available to the model.

Clustering: Discovering Groups in Data

Clustering is an unsupervised learning technique — unlike classification and regression, we do not provide labeled targets. Instead, the algorithm discovers natural groupings in the data.

K-Means is the most widely used clustering algorithm. It works by:

\nChoosing K initial cluster centers (centroids).\n
\nAssigning each data point to the nearest centroid.\n
\nRecomputing each centroid as the mean of its assigned points.\n
\nRepeating steps 2-3 until convergence.\n

We will use the classic Iris dataset (150 samples, 4 features measuring sepal and petal dimensions, with 3 known species). This lets us evaluate how well K-Means recovers the true species groupings.

Choosing K with the Silhouette Score

A common question is: how many clusters should we use? The silhouette score measures how similar each point is to its own cluster compared to other clusters (ranging from -1 to +1, higher is better):

 1 from sklearn.cluster import KMeans\n 2 from sklearn.preprocessing import StandardScaler\n 3 from sklearn.metrics import silhouette_score\n 4 \n 5 scaler = StandardScaler()\n 6 X_scaled = scaler.fit_transform(X)\n 7 \n 8 for k in range(2, 7):\n 9     km = KMeans(n_clusters=k, random_state=42, n_init=10)\n10     labels = km.fit_predict(X_scaled)\n11     score = silhouette_score(X_scaled, labels)\n12     print(f\"  k={k}: silhouette score = {score:.4f}\")\n

\n\n

Output from running clustering.py:

 1 $ python clustering.py\n 2 Dataset: 150 samples, 4 features\n 3 Features: ['sepal length (cm)', 'sepal width (cm)',\n 4            'petal length (cm)', 'petal width (cm)']\n 5 Known species: ['setosa', 'versicolor', 'virginica']\n 6 \n 7 === Silhouette Scores for Different k Values ===\n 8   k=2: silhouette score = 0.5818\n 9   k=3: silhouette score = 0.4599\n10   k=4: silhouette score = 0.3869\n11   k=5: silhouette score = 0.3459\n12   k=6: silhouette score = 0.3171\n

\n\n

The silhouette score is highest at k=2, which suggests that the data most naturally splits into two groups. This makes biological sense: setosa is very distinct from the other two species, while versicolor and virginica overlap considerably.

Comparing Clusters to True Labels

Since we happen to know the true species labels, we can use a cross-tabulation to see how well K-Means (with k=3) recovers them:

1 === K-Means (k=3) Cluster vs. True Species ===\n2 Cluster      0   1   2\n3 Species\n4 setosa       0  50   0\n5 versicolor  39   0  11\n6 virginica   14   0  36\n

\n\n

K-Means perfectly separates all 50 setosa samples into cluster 1. However, it struggles to distinguish versicolor from virginica — 14 virginica samples are assigned to the versicolor-dominant cluster 0, and 11 versicolor samples end up in the virginica-dominant cluster 2. This confirms what the silhouette scores suggested.

The cluster centers (transformed back to the original scale) reveal the “typical” flower in each group:

1 === Cluster Centers (original scale) ===\n2          sepal length  sepal width  petal length  petal width\n3 Cluster\n4 0                5.80         2.67          4.37         1.41\n5 1                5.01         3.43          1.46         0.25\n6 2                6.78         3.10          5.51         1.97\n

\n\n

Cluster 1 (setosa) has much smaller petals, while cluster 2 (mostly virginica) has the largest sepals and petals.

Regression and Clustering Wrap-up

In this chapter we covered two fundamental machine learning techniques beyond classification:

\nRegression lets us predict continuous values. We saw that even simple linear regression can be informative, and that feature coefficients reveal which inputs drive predictions.\n
\nClustering discovers structure in unlabeled data. We used silhouette scores to evaluate cluster quality and cross-tabulations to compare clusters against known labels.\n

Together with the classification example in the previous chapter, you now have hands-on experience with the three core tasks of classic machine learning. In the next chapter we will explore the data preparation work that comes before training any model.

Exploratory Data Analysis and Feature Engineering

Before training any machine learning model, you need to understand your data. Exploratory Data Analysis (EDA) is the process of examining a dataset to summarize its main characteristics, find patterns, detect anomalies, and check assumptions. Feature engineering is the art of creating new input variables — or transforming existing ones — to improve model performance.

These steps often make the difference between a mediocre model and a good one. As the saying goes: “garbage in, garbage out.”

The requirements for this chapter are:

1 uv pip install scikit-learn pandas numpy\n

\n\n

The examples for this chapter are in the directory source-code/data_analysis_and_feature_engineering.

We continue using the California Housing dataset from the previous chapter.

Exploratory Data Analysis

Loading and Inspecting the Data

The first thing to do with any dataset is to understand its shape, types, and basic statistics:

 1 import numpy as np\n 2 import pandas as pd\n 3 from sklearn.datasets import fetch_california_housing\n 4 \n 5 housing = fetch_california_housing()\n 6 df = pd.DataFrame(housing.data, columns=housing.feature_names)\n 7 df[\"MedHouseVal\"] = housing.target\n 8 \n 9 print(f\"Shape: {df.shape[0]} rows × {df.shape[1]} columns\")\n10 print(df.dtypes)\n

\n\n

Running our eda.py script gives us:

 1 $ python eda.py\n 2 === Dataset Overview ===\n 3 Shape: 20640 rows × 9 columns\n 4 \n 5 Column types:\n 6 MedInc         float64\n 7 HouseAge       float64\n 8 AveRooms       float64\n 9 AveBedrms      float64\n10 Population     float64\n11 AveOccup       float64\n12 Latitude       float64\n13 Longitude      float64\n14 MedHouseVal    float64\n

\n\n

All columns are floating point. Next, summary statistics:

1 === Summary Statistics ===\n2          MedInc  HouseAge  AveRooms  AveBedrms  Population  AveOccup\n3 count  20640.00  20640.00  20640.00   20640.00    20640.00  20640.00\n4 mean       3.87     28.64      5.43       1.10     1425.48      3.07\n5 std        1.90     12.59      2.47       0.47     1132.46     10.39\n6 min        0.50      1.00      0.85       0.33        3.00      0.69\n7 max       15.00     52.00    141.91      34.07    35682.00   1243.33\n

\n\n

Notice the wide range differences: Population ranges from 3 to 35,682 while AveBedrms ranges from 0.33 to 34.07. This tells us we will need feature scaling before training most models.

Also notice the extreme maximum values for AveRooms (141.91) and AveOccup (1,243.33) — these are likely outliers or data quality issues.

Checking for Missing Values

Missing data can silently break your models or introduce bias. Always check:

1 === Missing Values ===\n2 No missing values found.\n

\n\n

This dataset is clean, but real-world data rarely is. We will practice handling missing values in the feature engineering section below.

Correlation Analysis

Understanding which features correlate with the target helps guide feature selection and engineering:

1 === Correlation with MedHouseVal ===\n2   MedInc                +0.6881\n3   AveRooms              +0.1519\n4   Latitude              -0.1442\n5   HouseAge              +0.1056\n6   AveBedrms             -0.0467\n7   Longitude             -0.0460\n8   Population            -0.0246\n9   AveOccup              -0.0237\n

\n\n

MedInc (median income) stands out with a correlation of +0.69 — by far the strongest predictor. This aligns with what we saw from the regression coefficients in the previous chapter. The other features have relatively weak linear correlations, suggesting that non-linear relationships or feature combinations might be more informative.

Outlier Detection

The IQR (Interquartile Range) method flags values that fall more than 1.5 × IQR below Q1 or above Q3:

1 === Outlier Counts (IQR method) ===\n2   MedInc                  681 outliers (3.3%)\n3   AveRooms                511 outliers (2.5%)\n4   AveBedrms              1424 outliers (6.9%)\n5   Population             1196 outliers (5.8%)\n6   AveOccup                711 outliers (3.4%)\n7   MedHouseVal            1071 outliers (5.2%)\n

\n\n

Nearly 7% of AveBedrms values are outliers. In practice, you would investigate whether these are genuine extreme values or data errors, and decide whether to clip, transform, or remove them depending on your use case.

Feature Engineering

Feature engineering is where domain knowledge meets data science. By creating new features that better represent the underlying patterns, we can significantly improve model performance — sometimes more than choosing a fancier algorithm.

Creating New Features

We can derive meaningful features by combining existing ones:

1 df[\"RoomsPerHousehold\"] = df[\"AveRooms\"] / df[\"AveOccup\"]\n2 df[\"BedroomRatio\"] = df[\"AveBedrms\"] / df[\"AveRooms\"]\n3 df[\"PopPerHousehold\"] = df[\"Population\"] / df[\"HouseAge\"]\n

\n\n

\nRoomsPerHousehold: a proxy for house size relative to occupancy.\n
\nBedroomRatio: what fraction of rooms are bedrooms (a measure of house layout).\n
\nPopPerHousehold: population growth rate proxy (newer areas with high population).\n

Encoding Categorical Features

Many real-world datasets contain categorical variables (e.g., “color”, “region”, “type”). Most ML algorithms require numerical inputs, so we need to encode these.

In our example, we create a categorical feature by binning latitude into California regions, then one-hot encode it:

1 df[\"Region\"] = pd.cut(\n2     df[\"Latitude\"],\n3     bins=[32, 35, 38, 42],\n4     labels=[\"South\", \"Central\", \"North\"]\n5 )\n6 \n7 region_dummies = pd.get_dummies(df[\"Region\"], prefix=\"Region\", dtype=int)\n8 df = pd.concat([df, region_dummies], axis=1)\n

\n\n

1 === Region Distribution ===\n2 Region\n3 South      11294\n4 Central     6331\n5 North       3015\n

\n\n

One-hot encoding creates a separate binary column for each category (Region_South, Region_Central, Region_North). This avoids imposing a false numerical ordering on the categories.

Handling Missing Data

Real datasets almost always have missing values. Common strategies include:

\nDrop rows: simple but loses data.\n
\nFill with mean/median: preserves dataset size; median is more robust to outliers.\n
\nFill with a model prediction: more sophisticated but adds complexity.\n

We demonstrate median imputation:

1 # Simulate 5% missing values\n2 rng = np.random.default_rng(42)\n3 mask = rng.random(len(df)) < 0.05\n4 df.loc[mask, \"PopPerHousehold\"] = np.nan\n5 \n6 # Fill with median\n7 median_val = df[\"PopPerHousehold\"].median()\n8 df[\"PopPerHousehold\"] = df[\"PopPerHousehold\"].fillna(median_val)\n

\n\n

1 === Handling Missing Data ===\n2 Introduced 1028 missing values in PopPerHousehold\n3 Filled with median: 41.833\n4 Remaining missing: 0\n

\n\n

Feature Scaling

Features on vastly different scales cause problems for distance-based algorithms (K-NN, K-Means) and gradient-based optimizers. StandardScaler transforms each feature to have zero mean and unit variance:

1 === Feature Scaling Impact ===\n2 Before scaling:\n3   MedInc           mean=      3.87  std=      1.90\n4   AveRooms         mean=      5.43  std=      2.47\n5   Population       mean=   1425.48  std=   1132.46\n6 After StandardScaler:\n7   MedInc           mean=    0.0000  std=    1.0000\n8   AveRooms         mean=    0.0000  std=    1.0000\n9   Population       mean=   -0.0000  std=    1.0000\n

\n\n

After scaling, all features are on the same footing. Remember to always fit the scaler on training data only and apply it to both train and test sets to prevent data leakage.

Measuring the Impact

The ultimate test of feature engineering is whether it improves model performance. We compare a Linear Regression model with the original 8 features against one with our 14 engineered features:

1 === Model Comparison: Original vs. Engineered Features ===\n2   Original (8 features)           R² = 0.5758\n3   Engineered (14 features)        R² = 0.6622\n

\n\n

Our engineered features improved R² from 0.58 to 0.66 — a 15% improvement in explained variance, using the exact same algorithm. This demonstrates why feature engineering is often more valuable than model selection for improving results.

EDA and Feature Engineering Wrap-up

In this chapter we covered the essential data preparation skills that precede model training:

\nEDA helps you understand your data through summary statistics, correlation analysis, and outlier detection. Never skip this step.\n
\nFeature engineering transforms raw data into more informative inputs: creating derived features, encoding categories, handling missing values, and scaling.\n
\nThe payoff is real: our engineered features produced a 15% improvement in model performance with zero algorithm changes.\n

These techniques apply to every machine learning project, whether you are using classic algorithms from scikit-learn or deep learning frameworks. In the next part of this book, we move into deep learning.

Part 2 - Deep Learning

Deep learning extends classical machine learning with neural networks that can learn directly from raw data — images, text, audio — without manual feature engineering. In this part we cover the fundamentals of neural networks using PyTorch and then apply them to practical problems.

We start with the basics of deep learning: network architectures, training, and evaluation. We then explore natural language processing with deep learning, including text classification and working with pre-trained transformer models. The remaining chapters provide overviews of image generation, reinforcement learning, and recommendation systems.

When you have finished reading this section and want to learn more about specific deep learning architectures, I recommend this curated list of explained papers: ML-Papers-Explained. This ML Papers directory lists papers in chronological order and also includes LLM papers you might find useful after finishing this book.

The Basics of Deep Learning

Deep learning is a subfield of machine learning that is concerned with the design and implementation of artificial neural networks (ANNs) with multiple layers, also known as deep neural networks (DNNs). These networks are inspired by the structure and function of the human brain, and are designed to learn from large amounts of data such as images, text, and audio.

A neural network consists of layers of interconnected nodes, or neurons, which are organized into an input layer, one or more hidden layers, and an output layer. Each neuron receives input from the neurons in the previous layer, performs a computation, and passes the result to the neurons in the next layer. The computation typically involves a dot product of the input with a set of weights and an activation function, which is a non-linear function applied to the result. The weights are the parameters of the network that are learned during training.

The basic building block of a deep neural network is an artificial neuron, also known as perceptron, which is a simple mathematical model for a biological neuron. A perceptron receives input from other neurons and it applies a linear transformation to the input, followed by a non-linear activation function.

Deep learning networks can be feedforward networks where the data flows in one direction from input to output, or recurrent networks where the data can flow in a cyclic fashion.

There are different types of deep learning architectures such as feedforward neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs) and Generative Adversarial Networks (GANs). These architectures are designed to learn specific types of features and patterns from different types of data.

Deep learning models are trained using large amounts of labeled data, and typically use supervised or semi-supervised learning techniques. The training process involves adjusting the weights of the network to minimize a loss function, which measures the difference between the predicted output and the true output. This process is known as back-propagation, which is an algorithm for training the weights in a neural network by propagating the error back through the network. In the first AI book I wrote in the 1980s I covered the implementation of back-propagation in detail. As I write the material here on deep learning I think that it is more important for you to have the skills to choose appropriate tools for different applications and be less concerned about low-level implementation details. I think this characterizes the change in trajectory of AI from being about tool building to the skills of using available tools and sometimes previously trained models while spending more of your effort analyzing business functions and in general application domains.

Deep Learning has been applied to various fields such as Computer Vision, Natural Language Processing, Speech Recognition, etc.

Using PyTorch for Building a Cancer Prediction Model

We will use PyTorch to build a neural network that classifies the same University of Wisconsin cancer dataset we used in the scikit-learn chapter. This lets us directly compare the deep learning approach with the classic K-Nearest Neighbors classifier.

The examples for this chapter are in the directory source-code/deep_learning_basics.

The requirements for this chapter are:

1 uv pip install torch scikit-learn pandas numpy\n

\n\n

Why PyTorch?

PyTorch is the most widely used deep learning framework in both research and industry. Developed originally by Meta AI, it provides:

\nDynamic computation graphs: build and modify your network on-the-fly, making debugging natural.\n
\nPythonic API: feels like writing regular Python code, not configuring a separate computation engine.\n
\nExtensive ecosystem: integrates with Hugging Face Transformers, torchvision, torchaudio, and many more libraries.\n
\nStrong GPU support: seamlessly move computations between CPU, CUDA GPUs, and Apple Silicon (MPS).\n

Loading and Preparing the Data

We reuse the same CSV data files from our machine learning chapter. The data loading code converts the Pandas DataFrames to NumPy arrays, scales the features, then wraps everything in PyTorch tensors:

 1 import numpy as np\n 2 import pandas as pd\n 3 import torch\n 4 import torch.nn as nn\n 5 from torch.utils.data import DataLoader, TensorDataset\n 6 from sklearn.preprocessing import StandardScaler\n 7 \n 8 def load_data():\n 9     \"\"\"Load the cancer CSV files.\"\"\"\n10     train_df = pd.read_csv(\"../machine-learning/labeled_cancer_data.csv\")\n11     test_df = pd.read_csv(\"../machine-learning/labeled_test_data.csv\")\n12 \n13     train = train_df.to_numpy()\n14     X_train = train[:, 0:9].astype(np.float32)\n15     Y_train = train[:, -1].astype(np.float32).reshape(-1, 1)\n16 \n17     test = test_df.to_numpy()\n18     X_test = test[:, 0:9].astype(np.float32)\n19     Y_test = test[:, -1].astype(np.float32).reshape(-1, 1)\n20 \n21     scaler = StandardScaler()\n22     X_train = scaler.fit_transform(X_train)\n23     X_test = scaler.transform(X_test)\n24 \n25     return X_train, Y_train, X_test, Y_test\n

\n\n

Note that we use np.float32 — PyTorch expects 32-bit floats by default (unlike NumPy’s default of 64-bit).

Defining the Model

In PyTorch, we define neural network architectures by subclassing nn.Module. Our network has two hidden layers of 15 neurons with ReLU activation, a dropout layer for regularization, and a single output neuron:

 1 class CancerNet(nn.Module):\n 2     \"\"\"Simple feedforward network: 9 → 15 → 15 → 1.\"\"\"\n 3 \n 4     def __init__(self):\n 5         super().__init__()\n 6         self.network = nn.Sequential(\n 7             nn.Linear(9, 15),\n 8             nn.ReLU(),\n 9             nn.Linear(15, 15),\n10             nn.ReLU(),\n11             nn.Dropout(0.2),\n12             nn.Linear(15, 1),\n13         )\n14 \n15     def forward(self, x):\n16         return self.network(x)\n

\n\n

The forward method defines how data flows through the network. PyTorch’s autograd system automatically computes the gradients needed for backpropagation — we never need to write the backward pass manually.

The Training Loop

Unlike higher-level frameworks, PyTorch gives you explicit control over the training loop. This makes it easy to customize training behavior, add logging, or implement complex training schedules:

 1 def train_model(model, train_loader, epochs=60, lr=0.01):\n 2     criterion = nn.BCEWithLogitsLoss()\n 3     optimizer = torch.optim.SGD(model.parameters(), lr=lr)\n 4 \n 5     for epoch in range(epochs):\n 6         total_loss = 0.0\n 7         for X_batch, y_batch in train_loader:\n 8             optimizer.zero_grad()\n 9             outputs = model(X_batch)\n10             loss = criterion(outputs, y_batch)\n11             loss.backward()\n12             optimizer.step()\n13             total_loss += loss.item()\n14 \n15         if (epoch + 1) % 10 == 0:\n16             avg_loss = total_loss / len(train_loader)\n17             print(f\"  Epoch {epoch+1:3d}/{epochs}  loss: {avg_loss:.4f}\")\n

\n\n

We use BCEWithLogitsLoss which combines a sigmoid activation with binary cross-entropy loss — this is numerically more stable than applying sigmoid separately. The training loop follows the standard PyTorch pattern: zero gradients, forward pass, compute loss, backward pass, update weights.

Running the Example

Here is the complete output from running cancer_model.py:

 1 $ python cancer_model.py\n 2 Training examples: 554\n 3 Test examples:     15\n 4 \n 5 CancerNet(\n 6   (network): Sequential(\n 7     (0): Linear(in_features=9, out_features=15, bias=True)\n 8     (1): ReLU()\n 9     (2): Linear(in_features=15, out_features=15, bias=True)\n10     (3): ReLU()\n11     (4): Dropout(p=0.2, inplace=False)\n12     (5): Linear(in_features=15, out_features=1, bias=True)\n13   )\n14 )\n15 \n16 Training:\n17   Epoch  10/60  loss: 0.6913\n18   Epoch  20/60  loss: 0.6559\n19   Epoch  30/60  loss: 0.6250\n20   Epoch  40/60  loss: 0.5874\n21   Epoch  50/60  loss: 0.5421\n22   Epoch  60/60  loss: 0.4969\n23 \n24 Confusion Matrix:\n25 [[9 0]\n26  [1 5]]\n27 \n28 Classification Report:\n29               precision    recall  f1-score   support\n30 \n31          0.0       0.90      1.00      0.95         9\n32          1.0       1.00      0.83      0.91         6\n33 \n34     accuracy                           0.93        15\n35    macro avg       0.95      0.92      0.93        15\n36 weighted avg       0.94      0.93      0.93        15\n37 \n38 Sample predictions (should be close to [[0], [1]]):\n39 [[0.8588059]\n40  [0.9907742]]\n

\n\n

The model achieves 93% accuracy on the test set — matching the performance of our scikit-learn KNN classifier from the earlier chapter. The loss decreases steadily during training, and the sample predictions show that the model learns to distinguish between non-malignant and malignant cases.

You can compare this PyTorch example to our similar classification example using the same data where we used the Scikit-learn library. The deep learning approach requires more code but gives us full control over the model architecture, training process, and the ability to scale to much larger and more complex problems.

Natural Language Processing Using Deep Learning

I spent several years in the 1980s using symbolic AI approaches to Natural Language Processing (NLP) like augmented transition networks and conceptual dependency theory with mixed results. For small vocabularies and small domains of discourse these techniques yielded modestly successful results. I now only use Deep Learning approaches to NLP in my work.

Deep Learning in NLP is a branch of machine learning that utilizes deep neural networks to understand, interpret and generate human language. It has revolutionized the field of NLP by improving the accuracy of various NLP tasks such as text classification, language translation, sentiment analysis, and natural language generation (e.g., ChatGPT).

Deep learning models such as Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and Transformer models have been used to achieve state-of-the-art performance on various NLP tasks. These models have been trained on large amounts of text data, which has allowed them to learn complex patterns in human language and improve their understanding of the context and meaning of words.

The use of pre-trained models, such as BERT and GPT-4, has also become popular in NLP and I use both frequently for my work. These models have been pre-trained on a large corpus of text data, and can be fine-tuned for a specific task, which significantly reduces the amount of data and computing resources required to train a derived model.

Deep learning in NLP has been applied in various industries such as chatbots, automated customer service, and language translation services. It has also been used in research areas such as natural language understanding, question answering, and text summarization.

In the last decade deep learning techniques have solved most NLP problems, at least in a “good enough” engineering sense. In this chapter we will experiment with a few useful pre-trained models that you can run locally on your laptop using PyTorch.

Hugging Face and the Transformers Library

Hugging Face provides an extensive library of pre-trained models and the transformers Python library that allows developers to quickly integrate NLP capabilities into their applications. The pre-trained models are based on state-of-the-art transformer architectures, trained on large corpora of data. Hugging Face maintains a task page listing all kinds of machine learning that they support at https://huggingface.co/tasks for task domains:

\nComputer Vision\n
\nNatural Language Processing\n
\nAudio\n
\nTabular Data\n
\nMultimodal\n
\nReinforcement Learning\n

As an open source and open model company, Hugging Face is a leading provider of NLP technology. They have developed a library of pre-trained models, including models based on transformer architectures such as BERT, DeBERTa, and GPT variants, which can be fine-tuned for various tasks such as language understanding, language translation, and text generation.

The requirements for the examples in this chapter are:

1 uv pip install torch transformers sentence-transformers\n

\n\n

The examples for this chapter are in the directory source-code/deep_learning_nlp.

All models will be downloaded automatically to ~/.cache/huggingface the first time you run each script. Subsequent runs will use the cached models without re-downloading.

Summarizing Text Using a Pre-trained Model on Your Laptop

We use the facebook/bart-large-cnn model for text summarization. This model runs locally on your laptop using PyTorch — no API keys or cloud services needed. The downloaded model and associated files are a little less than two gigabytes of data.

In the current version of the transformers library (v5+), we load the model and tokenizer explicitly using AutoModelForSeq2SeqLM and AutoTokenizer:

 1 from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n 2 \n 3 model_name = \"facebook/bart-large-cnn\"\n 4 tokenizer = AutoTokenizer.from_pretrained(model_name)\n 5 model = AutoModelForSeq2SeqLM.from_pretrained(model_name)\n 6 \n 7 text = (\n 8     \"The President sent a request for changing the debt ceiling to \"\n 9     \"Congress. The president might call a press conference. The Congress \"\n10     \"was not oblivious of what the Supreme Court's majority had ruled on \"\n11     \"budget matters. Even four Justices had found nothing to criticize in \"\n12     \"the President's requirement that the Federal Government's four-year \"\n13     \"spending plan. It is unclear whether or not the President and \"\n14     \"Congress can come to an agreement before Congress recesses for a \"\n15     \"holiday. There is major disagreement between the Democratic and \"\n16     \"Republican parties on spending.\"\n17 )\n18 \n19 inputs = tokenizer(text, return_tensors=\"pt\", max_length=1024, truncation=True)\n20 summary_ids = model.generate(\n21     **inputs, max_length=60, num_beams=4, early_stopping=True\n22 )\n23 summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)\n24 print(summary)\n

\n\n

Here is the output from running summarization.py:

 1 $ python summarization.py\n 2 Loading summarization model (facebook/bart-large-cnn)...\n 3 \n 4 Original text (87 words):\n 5 The President sent a request for changing the debt ceiling to Congress...\n 6 \n 7 Summary:\n 8 The President sent a request for changing the debt ceiling to\n 9 Congress. The Congress was not oblivious of what the Supreme\n10 Court's majority had ruled on budget matters. Even four Justices\n11 had found nothing to criticize in the President's requirement\n12 that the Federal Government's four-year spending plan be changed\n

\n\n

Note: for production summarization work with longer documents, consider using larger instruction-tuned models (e.g., Llama 3 or Phi-4) that support context windows of 128K+ tokens. The BART model used here is limited to 1,024 tokens of input but is excellent for learning and prototyping.

Zero Shot Classification Using a Local Model

Zero shot classification models work by specifying which classification labels you want to assign to input texts — no pre-training on labeled examples is required. We use the MoritzLaurer/deberta-v3-base-zeroshot-v2.0 model, which is a modern and efficient replacement for the older bart-large-mnli model:

 1 from pprint import pprint\n 2 from transformers import pipeline\n 3 \n 4 classifier = pipeline(\n 5     \"zero-shot-classification\",\n 6     model=\"MoritzLaurer/deberta-v3-base-zeroshot-v2.0\",\n 7 )\n 8 \n 9 text = (\n10     \"Hi, I recently bought a device from your company but it is not \"\n11     \"working as advertised and I would like to get reimbursed!\"\n12 )\n13 candidate_labels = [\"refund\", \"faq\", \"legal\"]\n14 \n15 result = classifier(text, candidate_labels)\n16 pprint(result)\n

\n\n

Here is the output from running zero_shot_classification.py:

 1 $ python zero_shot_classification.py\n 2 Loading zero-shot classification model...\n 3 \n 4 Input text: Hi, I recently bought a device from your company but\n 5 it is not working as advertised and I would like to get reimbursed!\n 6 Candidate labels: ['refund', 'faq', 'legal']\n 7 \n 8 {'labels': ['refund', 'legal', 'faq'],\n 9  'scores': [0.9839024543762207, 0.01550459023565054, 0.000593014934565872],\n10  'sequence': 'Hi, I recently bought a device from your company but it is not '\n11              'working as advertised and I would like to get reimbursed!'}\n

\n\n

The model correctly classifies the customer message as a “refund” request with 98.4% confidence. This approach is powerful because you can define any set of labels at runtime without training a custom model.

Comparing Sentences for Similarity Using Transformer Models

The sentence-transformers library provides an easy-to-use interface for computing sentence embeddings using PyTorch. We use the all-MiniLM-L6-v2 model which is small (80 MB) and fast while producing high-quality embeddings:

 1 from sentence_transformers import SentenceTransformer, util\n 2 \n 3 model = SentenceTransformer(\"all-MiniLM-L6-v2\")\n 4 \n 5 sentences = [\n 6     \"The IRS has new tax laws.\",\n 7     \"Congress debating the economy.\",\n 8     \"The politician fled to South America.\",\n 9     \"Canada and the US will be in the playoffs.\",\n10     \"The cat ran up the tree.\",\n11     \"The meal tasted good but was expensive and perhaps not worth the price.\",\n12 ]\n13 \n14 # Encode all sentences\n15 embeddings = model.encode(sentences)\n16 \n17 # Compute cosine similarities between all pairs\n18 cos_sim = util.cos_sim(embeddings, embeddings)\n19 \n20 # Collect and sort all unique pairs\n21 pairs = []\n22 for i in range(len(cos_sim) - 1):\n23     for j in range(i + 1, len(cos_sim)):\n24         pairs.append((cos_sim[i][j].item(), i, j))\n25 \n26 pairs.sort(key=lambda x: x[0], reverse=True)\n27 \n28 print(\"Top-8 most similar pairs:\")\n29 for score, i, j in pairs[:8]:\n30     print(f\"  {score:.4f}  {sentences[i]}\")\n31     print(f\"          {sentences[j]}\")\n

\n\n

Output from running sentence_similarity.py:

 1 $ python sentence_similarity.py\n 2 Loading sentence-transformers model (all-MiniLM-L6-v2)...\n 3 \n 4 Top-8 most similar pairs:\n 5   0.1793  The IRS has new tax laws.\n 6           Congress debating the economy.\n 7 \n 8   0.1210  Congress debating the economy.\n 9           Canada and the US will be in the playoffs.\n10 \n11   0.1131  Congress debating the economy.\n12           The meal tasted good but was expensive and perhaps not worth the price.\n13 \n14   0.1061  Congress debating the economy.\n15           The politician fled to South America.\n16 \n17   0.0901  The politician fled to South America.\n18           The meal tasted good but was expensive and perhaps not worth the price.\n19 \n20   0.0677  The politician fled to South America.\n21           The cat ran up the tree.\n22 \n23   0.0432  The cat ran up the tree.\n24           The meal tasted good but was expensive and perhaps not worth the price.\n25 \n26   0.0388  Congress debating the economy.\n27           The cat ran up the tree.\n

\n\n

The results make intuitive sense: sentences about government and economics cluster together, while unrelated sentences about cats and meals have very low similarity scores.

A common use case is a customer service chatbot where we match the user’s question with all recorded questions that have accepted “canned answers.” The runtime to get the best match is O(N) where N is the number of previously recorded user questions. The cosine similarity calculation, given two embedding vectors, is very fast.

For a practical application you can use the cos_sim function from the sentence-transformers utility library:

1 >>> from sentence_transformers import util\n2 >>> util.cos_sim(embeddings[0], embeddings[1])\n3 tensor([[0.1793]])\n

\n\n

Deep Learning Natural Language Processing Wrap-up

In this chapter we have seen examples of how effective deep learning is for NLP using PyTorch and the Hugging Face transformers ecosystem. All three examples — summarization, zero-shot classification, and sentence similarity — run locally on your laptop without requiring API keys or cloud services.

I worked on other methods of NLP over a 25-year period and I ask you, dear reader, to take my word on this: deep learning has revolutionized NLP and for almost all practical NLP applications, deep learning libraries and pre-trained models from organizations like Hugging Face should be the first thing that you consider using.

Part IV - Overviews of Image Generation, Reinforcement Learning, and Recommendation Systems

This final part of this book consists of overviews of three important topics that I cover briefly, with perhaps more material added in the next edition of this book.

Overview of Image Generation

I have never used deep learning image generation at work but I have fun experimenting with both code and model examples, as well as turn-key web apps like DALL·E. In this chapter we look at two approaches to generating images from text prompts using PyTorch.

The requirements for this chapter are:

1 uv pip install torch diffusers transformers accelerate\n

\n\n

The examples for this chapter are in the directory source-code/deep_learning_image_generation.

Image Generation Using Stable Diffusion and PyTorch

Stable Diffusion is an open-source deep learning model for text-to-image generation. The Hugging Face diffusers library makes it straightforward to load and run Stable Diffusion models using PyTorch. Here is a complete example that generates an image from a text prompt:

 1 import torch\n 2 from diffusers import DiffusionPipeline\n 3 \n 4 # A smaller model (~1GB) for faster downloading\n 5 model_id = \"segmind/tiny-sd\"\n 6 print(f\"Loading model: {model_id}\")\n 7 \n 8 # Use float16 for GPU/MPS, float32 for CPU\n 9 if torch.cuda.is_available():\n10     pipe = DiffusionPipeline.from_pretrained(\n11         model_id, torch_dtype=torch.float16\n12     )\n13     pipe = pipe.to(\"cuda\")\n14 elif torch.backends.mps.is_available():\n15     pipe = DiffusionPipeline.from_pretrained(\n16         model_id, torch_dtype=torch.float16\n17     )\n18     pipe = pipe.to(\"mps\")\n19 else:\n20     pipe = DiffusionPipeline.from_pretrained(model_id)\n21 \n22 prompt = \"a serene mountain landscape at sunset, oil painting style\"\n23 image = pipe(prompt, num_inference_steps=25).images[0]\n24 image.save(\"generated_landscape.png\")\n

\n\n

The first time you run this code, the model weights (about 1.1 GB) will be downloaded to ~/.cache/huggingface in your home directory. Subsequent runs use the cached model.

The code automatically detects available hardware: NVIDIA GPU (CUDA), Apple Silicon (MPS), or CPU. GPU acceleration dramatically speeds up image generation — from minutes on CPU to seconds on a modern GPU.

You can experiment with different prompts, and the num_inference_steps parameter controls the quality/speed tradeoff (higher = better quality, slower). Here is sample output running on Apple Silicon:

1 $ python image_generation.py\n2 Loading model: segmind/tiny-sd\n3 (First run will download about 1.1 GB of model weights)\n4 \n5 Generating image for prompt: 'a serene mountain landscape at sunset, oil painting style'\n6 Image saved to: generated_landscape.png\n

\n\n

Understanding the Diffusion Process

Stable Diffusion works by a process called denoising diffusion:

\nStart with pure random noise (a tensor of random values).\n
\nGradually remove noise over many steps, guided by the text prompt.\n
\nThe result is an image that matches the prompt description.\n

The text prompt is converted to an embedding vector using a text encoder (CLIP), which guides the denoising process at each step. This is why the same prompt can generate different images with different random seeds.

Mini-DALL·E: A Lightweight Alternative

For a lighter-weight alternative, Brett Kuprel’s Mini-Dalle model is a reduced size port of DALL·E Mini to PyTorch. It requires less GPU memory and can run on more modest hardware:

1 uv pip install min-dalle\n

\n\n

 1 import torch\n 2 from min_dalle import MinDalle\n 3 \n 4 model = MinDalle(\n 5     dtype=torch.float32,\n 6     device='cuda',    # use 'cpu' if no GPU\n 7     is_mega=True,\n 8     is_reusable=True\n 9 )\n10 \n11 text = \"parrot sitting on old man's shoulder\"\n12 \n13 image = model.generate_image(\n14     text=text,\n15     seed=-123,\n16     grid_size=2,\n17     temperature=1.5,\n18     top_k=256,\n19     supercondition_factor=12\n20 )\n21 \n22 image.save(text.replace(\" \", \"_\") + \".png\")\n

\n\n

If is_mega is true then a larger model is constructed. If is_reusable is true then the same model is reused to create additional images.

You can try changing the temperature (increase for more randomness and differences from training examples), random seed, and text prompt.

Here is a sample generated image:

The three Python model files in the GitHub repository comprise about 600 lines of code making this a fairly short complete Attention Network/Transformer example. If you are interested in the implementation please read the original paper from Open AI Zero-Shot Text-to-Image Generation before reading the code for the models.

Overview of Reinforcement Learning (Optional Material)

Reinforcement Learning has been used in various applications such as robotics, game playing, recommendation systems, and more. Reinforcement Learning (RL) is a broad topic and we will only cover basic aspects of RL.

The requirements for this chapter are:

1 uv pip install gymnasium numpy pymdptoolbox\n

\n\n

The examples for this chapter are in the directory source-code/reinforcement_learning.

Overview

Reinforcement Learning is a type of machine learning that is concerned with decision-making in dynamic and uncertain environments. RL uses the concept of an agent which interacts with its environment by taking actions and receiving feedback in the form of rewards or penalties. The goal of the agent is to learn a policy which is a mapping from states of the environment to actions with the goal of maximizing the expected cumulative reward over time.

There are several key components to RL:

\nEnvironment: the system or “world” that the agent interacts with.\n
\nAgent: the decision-maker that chooses actions based on its current state, the current environment, and its policy.\n
\nState: a representation of the current environment, the parameters and trained policy of the agent, and possibly the visible actions of other agents in the environment.\n
\nAction: a decision taken by the agent.\n
\nReward: a scalar value that the agent receives as feedback for its actions.\n

Reinforcement learning algorithms can be divided into two main categories: value-based and policy-based. In value-based RL the agent learns an estimate of the value of different states or state-action pairs which are then used to determine the optimal policy. In contrast, in policy-based RL the agent directly learns a policy without estimating the value of states or state-action pairs.

Reinforcement Learning can be implemented using different techniques such as Q-learning, SARSA, DDPG, A2C, PPO, etc. Some of these techniques are model-based, which means that the agent uses a model of the environment to simulate the effects of different actions and plan ahead. Others are model-free, which means that the agent learns directly from the rewards and transitions experienced in the environment.

If you enjoy the overview material in this chapter I recommend that you consider investing the time in the Coursera RL specialization https://www.coursera.org/learn/fundamentals-of-reinforcement-learning taught by Martha and Adam White. There are 50+ RL courses on Coursera. I took the courses taught by Martha and Adam White before starting my RL project at Capital One.

My favorite RL book is “Reinforcement Learning: An Introduction, second edition” by Richard Sutton and Andrew Barto, that can be read online for free at http://www.incompleteideas.net/book/the-book-2nd.html. They originally wrote their book examples in Common Lisp but most of the code is available rewritten in Python. The Common Lisp code for the examples is here. Shangtong Zhang translated the book examples to Python, available here. Martha and Adam White’s Coursera class uses this book as a reference.

The core idea of RL is that we train a software agent to interact with and change its environment based on its expectations of the utility of current actions improving metrics for success in the future. There is some tension between writing agents that simply reuse past actions that proved to be useful, rather than aggressively exploring new actions in the environment. There are interesting formalisms for this that we will cover.

There are two general approaches to providing training environments to Reinforcement Learning trained agents: physically devices in the real world or simulated environments. This is not a book on robotics so we use the second option.

The end goal for modeling a RL problem is calculating a policy that can be used to control an agent in environments that are similar to the training environment. In a model at time t we have a given State_t. RL policies can be continually be updated during training and in production environments. A policy given a State_t, calculates an Action_t to execute and changes the state to State_t+1.

Available RL Tools

For initial experiments with RL, I would recommend taking the same path that I took before using RL at work:

\nUsing a maintained fork of OpenAI’s Gym library Gymnasium.\n
\nTaking the Coursera classes by Martha and Adam White.\n
\nThe Sutton/Barto RL Book and accompanying Common Lisp or Python examples.\n

The original OpenAI RL Gym was a good environment for getting started with simple environments and examples but I didn’t get very far with self-study. The RL Coursera classes were a great overview of theory and practice, and I then spend as much time as I could spare working through Sutton/Barto before my project started.

An Introduction to Markov Decision Process

Before we can write a reinforcement learning agent, we need to understand the mathematical framework that RL is built upon: the Markov Decision Process (MDP). An MDP provides a formal way to model sequential decision-making problems where outcomes are partly random and partly under the control of a decision-maker.

Let’s start by defining the key terms:

\n
Sequential decision problem: a problem where decisions are made in sequence over time, and each decision affects future states and rewards. Unlike one-shot classification or regression, you must think ahead.
\n
\n
Fully observable: the agent can see the complete state of the environment at each step. No hidden information or partial views.
\n
\n
Stochastic environment: transitions between states are not deterministic. An action taken in a given state may lead to different outcomes with certain probabilities. The real world is uncertain, and MDPs model this uncertainty.
\n
\n
Markov property: the future depends only on the current state and action, not on the history of how you got there. Formally, P(State_t+1 | State_t, Action_t) = P(State_t+1 | State_t, Action_t, State_t-1, Action_t-1, …).
\n
\n
Bellman equation: the recursive relationship that expresses the value of a state as the expected immediate reward plus the discounted value of the next state. This is the foundation of dynamic programming in RL: V(s) = max_a [ R(s,a) + γ Σ P(s’|s,a) V(s’) ] where γ (gamma) is the discount factor.
\n

The discount factor γ (between 0 and 1) controls how much the agent values future rewards versus immediate rewards:

\nγ close to 0: agent is shortsighted, cares mostly about immediate reward\n
\nγ close to 1: agent is farsighted, cares about long-term cumulative reward\n

Solving MDPs with pymdptoolbox

The pymdptoolbox library provides classic MDP solution algorithms. Let’s work through two examples: a custom grid world and the built-in forest management problem.

Listing of mdp_demo.py:

 1 import mdptoolbox.example\n 2 import numpy as np\n 3 \n 4 print(\"=\" * 55)\n 5 print(\"Markov Decision Process Demo\")\n 6 print(\"=\" * 55)\n 7 \n 8 print(\"\\n--- Example 1: Custom 3x3 Grid World ---\")\n 9 n_states = 9\n10 n_actions = 4\n11 \n12 P = np.zeros((n_actions, n_states, n_states))\n13 grid = [(r, c) for r in range(3) for c in range(3)]\n14 for s, (r, c) in enumerate(grid):\n15     for a, (dr, dc) in enumerate([(-1, 0), (0, 1), (1, 0), (0, -1)]):\n16         nr, nc = r + dr, c + dc\n17         if 0 <= nr < 3 and 0 <= nc < 3:\n18             ns = nr * 3 + nc\n19         else:\n20             ns = s\n21         P[a, s, ns] = 1.0\n22 \n23 R = np.zeros((n_states, n_actions))\n24 R[8, :] = 10.0\n25 R[5, :] = -5.0\n26 \n27 vi = mdptoolbox.mdp.ValueIteration(P, R, 0.9)\n28 vi.run()\n29 actions = [\"↑\", \"→\", \"↓\", \"←\"]\n30 print(\"Optimal policy:\")\n31 for r in range(3):\n32     row = \"\"\n33     for c in range(3):\n34         s = r * 3 + c\n35         row += f\"  {actions[vi.policy[s]]}  \"\n36     print(row)\n37 print(f\"\\nValue function: {np.round(vi.V, 2)}\")\n38 print(f\"Iterations to converge: {vi.iter}\")\n39 \n40 print(\"\\n--- Policy Iteration on same grid ---\")\n41 pi = mdptoolbox.mdp.PolicyIteration(P, R, 0.9)\n42 pi.run()\n43 print(f\"Policy: {tuple(pi.policy)}\")\n44 print(f\"Iterations: {pi.iter}\")\n45 \n46 print(\"\\n--- Example 2: Forest Management (built-in) ---\")\n47 P2, R2 = mdptoolbox.example.forest(S=5, r1=4, r2=2, p=0.1)\n48 print(\"States: 5 (forest age classes 0-4)\")\n49 print(\"Action 0 = Wait, Action 1 = Cut\")\n50 print(\"p(fire) = 0.1 each year\")\n51 print(f\"Reward shape: {R2.shape} (states × actions)\")\n52 \n53 vi2 = mdptoolbox.mdp.ValueIteration(P2, R2, 0.9)\n54 vi2.run()\n55 print(f\"Optimal policy: {tuple(vi2.policy)}\")\n56 for s, a in enumerate(vi2.policy):\n57     print(f\"  Forest age {s}: {'Wait' if a == 0 else 'Cut'}\")\n58 print(f\"Value function: {np.round(vi2.V, 2)}\")\n59 print(f\"Iterations: {vi2.iter}\")\n

\n\n

Here is the output of running mdp_demo.py (the unicode characters for directionsin the Optimal policy output don’t print correctly):

 1 $ python mdp_demo.py\n 2 =======================================================\n 3 Markov Decision Process Demo\n 4 =======================================================\n 5 \n 6 --- Example 1: Custom 3x3 Grid World ---\n 7 Optimal policy:\n 8   →    ↓    ↓  \n 9   →    ↓    ↓  \n10   →    →    →  \n11 \n12 Value function: [ 6.56 13.85 17.45 13.85 21.95 25.95 21.95 30.95 40.95]\n13 Iterations to converge: 5\n14 \n15 --- Policy Iteration on same grid ---\n16 Policy: (1, 2, 2, 1, 2, 2, 1, 1, 1)\n17 Iterations: 5\n18 \n19 --- Example 2: Forest Management (built-in) ---\n20 States: 5 (forest age classes 0-4)\n21 Action 0 = Wait, Action 1 = Cut\n22 p(fire) = 0.1 each year\n23 Reward shape: (5, 2) (states × actions)\n24 Optimal policy: (0, 0, 0, 0, 0)\n25   Forest age 0: Wait\n26   Forest age 1: Wait\n27   Forest age 2: Wait\n28   Forest age 3: Wait\n29   Forest age 4: Wait\n30 Value function: [ 3.49  5.62  8.24 11.48 15.48]\n31 Iterations: 6\n

\n\n

Walking through Example 1, we create a 3×3 grid where each cell is a state (0 through 8). The agent can move up, right, down, or left. The transition matrix P has shape (actions, states, states) — for each action and current state, it specifies the probability of landing in each next state. Our grid is deterministic (probability of 1.0 for each move), and bumping into walls keeps the agent in place.

The reward matrix R gives 10 for reaching the goal (bottom-right, state 8) and -5 for the trap (state 5, which is the middle-right cell). Value iteration converges in 5 iterations and produces a sensible policy: the arrows point around the trap toward the goal. The value function shows that states closer to the goal have higher values.

I also ran Policy Iteration on the same grid. Both algorithms arrived at the same policy but through different means: value iteration improves value estimates until convergence, while policy iteration alternates between evaluating a fixed policy and improving it greedily.

In Example 2, we use pymdptoolbox’s built-in forest management problem. At each year, you choose to Wait (let the forest grow one age class) or Cut (harvest timber, resetting the forest to age 0). There is a 10% chance of fire each year that also resets the forest to age 0. With these parameters the optimal policy is to always Wait — the expected value of older forest outweighs the immediate reward of cutting.

The key takeaway: MDPs give you a formal language to describe decision problems, and algorithms like value iteration and policy iteration compute optimal policies. In the next section, we will tackle environments where we do not know the transition probabilities in advance — which is where reinforcement learning comes in.

A Concrete Example: Q-Learning with Gymnasium

When the transition and reward models are unknown, the agent must learn through trial and error. Q-learning is one of the simplest and most widely used model-free RL algorithms. It learns an action-value function Q(s,a) — the expected cumulative reward of taking action a in state s and following the optimal policy thereafter.

The Q-learning update rule is:

Q(s,a) ← Q(s,a) + α [ r + γ · max_a’ Q(s’,a’) — Q(s,a) ]

Where:

\nα (alpha) is the learning rate\n
\nγ (gamma) is the discount factor\n
\nr is the immediate reward received\n
\ns’ is the next state\n

The agent also needs to balance exploration (trying random actions to discover better strategies) against exploitation (using known good actions). We use an epsilon-greedy strategy: with probability ε, take a random action; otherwise take the action with the highest Q-value. Over time we decay ε so the agent explores less and exploits more.

We will use Gymnasium (the maintained successor to OpenAI Gym) and its FrozenLake environment. In FrozenLake, the agent navigates a 4×4 grid of frozen and cracked ice to reach a goal without falling through. By default, the ice is slippery — your intended move only succeeds with 1/3 probability, and you slide perpendicular otherwise.

Listing of frozen_lake_qlearning.py:

  1 import gymnasium as gym\n  2 import numpy as np\n  3 \n  4 np.random.seed(42)\n  5 \n  6 \n  7 def q_learning(env, episodes=10000, alpha=0.1, gamma=0.99,\n  8                epsilon=1.0, epsilon_decay=0.999, min_epsilon=0.01):\n  9     n_states = env.observation_space.n\n 10     n_actions = env.action_space.n\n 11     Q = np.zeros((n_states, n_actions))\n 12     success_history = []\n 13 \n 14     for episode in range(episodes):\n 15         state, _ = env.reset()\n 16         done = False\n 17 \n 18         while not done:\n 19             if np.random.random() < epsilon:\n 20                 action = env.action_space.sample()\n 21             else:\n 22                 action = np.argmax(Q[state])\n 23 \n 24             next_state, reward, terminated, truncated, _ = env.step(action)\n 25             done = terminated or truncated\n 26 \n 27             best_next = np.max(Q[next_state])\n 28             Q[state, action] += alpha * (\n 29                 reward + gamma * best_next - Q[state, action]\n 30             )\n 31 \n 32             state = next_state\n 33 \n 34         epsilon = max(min_epsilon, epsilon * epsilon_decay)\n 35 \n 36         if (episode + 1) % 1000 == 0:\n 37             success_rate = evaluate(env, Q, episodes=100)\n 38             success_history.append((episode + 1, success_rate))\n 39             print(f\"  Episode {episode + 1:>5}: success rate = {success_rate:.2f}\")\n 40 \n 41     return Q, success_history\n 42 \n 43 \n 44 def evaluate(env, Q, episodes=100):\n 45     successes = 0\n 46     for _ in range(episodes):\n 47         state, _ = env.reset()\n 48         done = False\n 49         while not done:\n 50             action = np.argmax(Q[state])\n 51             state, reward, terminated, truncated, _ = env.step(action)\n 52             done = terminated or truncated\n 53             if terminated and reward == 1.0:\n 54                 successes += 1\n 55     return successes / episodes\n 56 \n 57 \n 58 def print_policy(Q, env_name):\n 59     n_states = Q.shape[0]\n 60     size = int(np.sqrt(n_states))\n 61     arrows = {0: \"←\", 1: \"↓\", 2: \"→\", 3: \"↑\"}\n 62 \n 63     print(f\"\\nLearned policy for {env_name}:\")\n 64     for row in range(size):\n 65         row_str = \"\"\n 66         for col in range(size):\n 67             s = row * size + col\n 68             row_str += arrows[np.argmax(Q[s])]\n 69         print(f\"  {row_str}\")\n 70     print(\"  (←=left ↓=down →=right ↑=up)\")\n 71 \n 72 \n 73 print(\"=\" * 55)\n 74 print(\"Q-Learning on FrozenLake (slippery)\")\n 75 print(\"=\" * 55)\n 76 \n 77 env = gym.make(\"FrozenLake-v1\", is_slippery=True)\n 78 print(f\"States: {env.observation_space.n}\")\n 79 print(f\"Actions: {env.action_space.n}\")\n 80 print(f\"Map size: 4x4\")\n 81 \n 82 print(\"\\nTraining:\")\n 83 Q, history = q_learning(env)\n 84 \n 85 print_policy(Q, \"FrozenLake-v1 (slippery)\")\n 86 \n 87 print(\"\\nFinal evaluation (1000 episodes):\")\n 88 final_rate = evaluate(env, Q, episodes=1000)\n 89 print(f\"  Success rate: {final_rate:.2f}\")\n 90 \n 91 print(\"\\n\" + \"=\" * 55)\n 92 print(\"Q-Learning on FrozenLake (deterministic, no slipping)\")\n 93 print(\"=\" * 55)\n 94 \n 95 env2 = gym.make(\"FrozenLake-v1\", is_slippery=False)\n 96 print(f\"States: {env2.observation_space.n}\")\n 97 print(f\"Actions: {env2.action_space.n}\")\n 98 \n 99 print(\"\\nTraining:\")\n100 Q2, _ = q_learning(env2, episodes=2000)\n101 \n102 print_policy(Q2, \"FrozenLake-v1 (deterministic)\")\n103 \n104 print(\"\\nFinal evaluation (1000 episodes):\")\n105 final_rate2 = evaluate(env2, Q2, episodes=1000)\n106 print(f\"  Success rate: {final_rate2:.2f}\")\n107 \n108 env.close()\n109 env2.close()\n

\n\n

Here is the output:

 1 $ python frozen_lake_qlearning.py\n 2 =======================================================\n 3 Q-Learning on FrozenLake (slippery)\n 4 =======================================================\n 5 States: 16\n 6 Actions: 4\n 7 Map size: 4x4\n 8 \n 9 Training:\n10   Episode  1000: success rate = 0.68\n11   Episode  2000: success rate = 0.77\n12   Episode  3000: success rate = 0.69\n13   Episode  4000: success rate = 0.75\n14   Episode  5000: success rate = 0.77\n15   Episode  6000: success rate = 0.50\n16   Episode  7000: success rate = 0.77\n17   Episode  8000: success rate = 0.77\n18   Episode  9000: success rate = 0.70\n19   Episode 10000: success rate = 0.75\n20 \n21 Learned policy for FrozenLake-v1 (slippery):\n22   ←↑↑↑\n23   ←←→←\n24   ↑↓←←\n25   ←→↓←\n26   (←=left ↓=down →=right ↑=up)\n27 \n28 Final evaluation (1000 episodes):\n29   Success rate: 0.74\n30 \n31 =======================================================\n32 Q-Learning on FrozenLake (deterministic, no slipping)\n33 =======================================================\n34 States: 16\n35 Actions: 4\n36 \n37 Training:\n38   Episode  1000: success rate = 1.00\n39   Episode  2000: success rate = 1.00\n40 \n41 Learned policy for FrozenLake-v1 (deterministic):\n42   ↓←←←\n43   ↓←↓←\n44   →→↓←\n45   ←→→←\n46   (←=left ↓=down →=right ↑=up)\n47 \n48 Final evaluation (1000 episodes):\n49   Success rate: 1.00\n

\n\n

The q_learning function implements the core algorithm. We maintain a Q-table of shape (n_states, n_actions) initialized to zeros. Each episode runs until termination (falling in a hole or reaching the goal) or truncation (100-step limit). The epsilon-greedy strategy decays from 1.0 (pure exploration) to 0.01 over time, following the decay schedule epsilon *= 0.999 each episode.

The Q-value update is the heart of Q-learning:

1 best_next = np.max(Q[next_state])\n2 Q[state, action] += alpha * (reward + gamma * best_next - Q[state, action])\n

\n\n

This says: move Q(s,a) a small step (controlled by alpha) toward r + γ·max_a' Q(s',a'). The difference r + γ·max Q - Q is called the temporal-difference error — how much better or worse the outcome was than predicted.

Let’s discuss the results:

Slippery version: Achieves ~75% success rate after 10,000 episodes. This is about as good as tabular Q-learning gets on the slippery FrozenLake — the randomness of ice physics means some runs are doomed regardless of policy. The learned policy arrows point toward the goal, though the randomness makes the policy harder to interpret visually than the deterministic case. You can also see the success rate fluctuate (e.g., dropping to 0.50 at episode 6000) — this is normal for Q-learning on stochastic environments and reflects the exploration-exploitation trade-off.

Deterministic version (is_slippery=False): Converges to a 100% success rate within 1,000 episodes. Without slipping, the task reduces to a simple shortest-path problem and Q-learning finds it quickly.

Other Gymnasium Environments Worth Exploring

Once you are comfortable with FrozenLake, Gymnasium offers many environments to experiment with:

\nTaxi-v3: 500 discrete states, the agent must pick up and drop off a passenger at correct locations. 5 actions (move directions + pickup/dropoff). Good for testing discrete Q-learning at slightly larger scale.\n
\nCartPole-v1: balance a pole on a cart. Continuous observation space (4 values), discrete actions (2). Requires function approximation (e.g., a neural network) rather than a Q-table.\n
\nMountainCar-v0: drive an underpowered car up a hill. Continuous observations, discrete actions. Classic example where exploration strategy matters deeply.\n
\nLunarLander-v3: land a spacecraft on a landing pad. 8 continuous observations, 4 discrete actions. More complex dynamics.\n

For continuous observation spaces, you cannot use a lookup table — you need Deep Q-Networks (DQN) which use neural networks to approximate Q(s,a). That topic is beyond our scope here, but the Sutton/Barto book covers it well.

When learning RL, start with tabular methods (Q-learning on discrete environments) before moving to deep RL. The concepts transfer directly, but debugging is far easier when you can inspect every Q-value in a table.

Reinforcement Learning Wrap-up

In this chapter we covered:

\nMarkov Decision Processes: the mathematical foundation of RL, including states, actions, rewards, transition probabilities, the Bellman equation, and discount factors.\n
\nMDP solving with pymdptoolbox: value iteration and policy iteration on custom and built-in problems.\n
\nQ-learning: a model-free algorithm that learns from experience without needing transition probabilities. We implemented it from scratch and trained agents on the FrozenLake environment.\n
\nExploration vs exploitation: controlled by the epsilon-greedy strategy with decaying epsilon.\n

If this chapter sparked your interest, I encourage you to work through the Coursera specialization by Martha and Adam White and the Sutton/Barto book. For Python-focused RL, the Stable-Baselines3 library provides reliable implementations of DQN, A2C, PPO, and other algorithms ready to use with Gymnasium environments.

I tagged this chapter as optional material because I believe most readers will get more immediate value from mastering deep learning and pre-trained models. But if you find yourself working on sequential decision-making problems — robotics, game AI, resource allocation, dynamic pricing — the RL toolkit becomes indispensable.

Overview of Recommendation Systems (Optional Material)

Recommendation systems are a type of information filtering system that utilize historical data, such as past user behavior or interactions, to predict the likelihood of a user’s interest in certain items or products. As an example application: if a product web site has 100K products that is too many for customers to browse through. Based on a customer’s past purchases, finding other customers with similar purchases, etc. it is possible to filter the products shown to a customer.

Note: This is an advanced topic and you will need to reference the linked documentation resources to fully understand the material.

Writing recommendation systems is a common requirement for almost all businesses that sell products to customers. Before we get started we need to define two terms that you may not be familiar with: Collaborative filtering: uses both similarities between users and items to calculate recommendations. This linked Wikipedia article also discusses content-based filtering which uses user and item features.

The Movie Lens dataset created by the GroupLens Research organization uses the user movie preference https://movielens.org dataset. This dataset is a standard for developing and evaluating recommendation system algorithms and models.

There are at least three good approaches to take:

\nUse a turnkey recommendation system like Amazon Personalize that is a turn-key service on AWS. You can evaluate Amazon Personalize for your company’s use by spending about one hour working through the getting started tutorial.\n
\nUse one of the standard libraries or TensorFlow implementations for the classic approach using [Matrix Factorization](https://en.wikipedia.org/wiki/Matrix_factorization_(recommender_systems) for collaborative filtering. Examples are Eric Lundquist’s Python library rankfm and the first example for the TensorFlow Recommenders library. Google has a good Matrix Factorization tutorial. While I prefer using the much more complicated TensorFlow Recommenders library, using matrix factorization is probably a good way to start and I recommend taking an hour to work through this Google Colab tutorial\n
\nUse the TensorFlow Recommenders library that supports multi-tower deep learning models that use data for user interactions, user detail data, and product detail data.\n

We will not write any recommendation systems from scratch in this chapter. We will review one open source recommendation system that I have used at work, the TensorFlow Recommenders library.

Recommendation systems can use a wide variety of techniques, such as collaborative filtering, content-based filtering, and hybrid methods combining filtering algorithms, Matrix Factorization, or Deep Learning technologies, etc. to generate personalized recommendations for users. Collaborative filtering algorithms make recommendations based on the actions of similar users, while content-based filtering algorithms base recommendations on the attributes of items that a user has previously shown interest in. Hybrid methods may be further enhanced by incorporating additional data sources, such as demographic information, or by utilizing more advanced machine learning techniques, such as deep learning or reinforcement learning.

TensorFlow Recommenders

I used Google’s TensorFlow Recommenders library for a work project. I recommend it because it has very good documentation, many examples using the Movie Lens dataset, and is fairly easy to adapt to general user/product recommendation systems.

We will refer to the documentation and examples https://www.tensorflow.org/recommenders and follow the last Movie Lens example.

There are several types of data that could be used for recommending movies:

\nUser interactions (selecting movies).\n
\nUser data. User data is not used in this example, but for a product recommendation system I have created embeddings of all available data features associated with users.\n
\nMovie data based on text embedding of movie titles. Note: for product recommendation systems you might still use text embedding of product descriptions, but you would also likely create embeddings based on product features.\n

We will use the TensorFlow Recommenders using rich features example. For the following overview discussion, you may want to either open this link to read this example or open the alternative Google Colab example link to run the example on Colab. Please note that this example takes about ten minutes to run on Colab. For our discussion, I will use short code snippets and use one screenshot of the example in Colab so you can optionally just follow along without opening either link for now.

The TF recommenders example starts with reading the Movie Lens dataset using the TensorFlow Data library:

 1 import tensorflow_datasets as tfds\n 2 ratings = tfds.load(\"movielens/100k-ratings\", split=\"train\")\n 3 movies = tfds.load(\"movielens/100k-movies\", split=\"train\")\n 4 \n 5 ratings = ratings.map(lambda x: {\n 6     \"movie_title\": x[\"movie_title\"],\n 7     \"user_id\": x[\"user_id\"],\n 8     \"timestamp\": x[\"timestamp\"],\n 9 })\n10 movies = movies.map(lambda x: x[\"movie_title\"])\n

\n\n

We need to later generate embedding layers for both unique movie titles and also unique user IDs. We start with getting sequences for unique movie titles and user IDs:

1 unique_movie_titles = np.unique(np.concatenate(list(movies.batch(1000))))\n2 unique_user_ids = np.unique(np.concatenate(list(ratings.batch(1_000).map(\n3     lambda x: x[\"user_id\"]))))\n

\n\n

In this example Python user and movie models are derived from the Python class tf.keras.Model. Let’s look at the implementation of these two models:

 1 class UserModel(tf.keras.Model):\n 2   \n 3   def __init__(self):\n 4     super().__init__()\n 5 \n 6     self.user_embedding = tf.keras.Sequential([\n 7         tf.keras.layers.StringLookup(\n 8             vocabulary=unique_user_ids, mask_token=None),\n 9         tf.keras.layers.Embedding(len(unique_user_ids) + 1, 32),\n10     ])\n11     self.timestamp_embedding = tf.keras.Sequential([\n12         tf.keras.layers.Discretization(timestamp_buckets.tolist()),\n13         tf.keras.layers.Embedding(len(timestamp_buckets) + 1, 32),\n14     ])\n15     self.normalized_timestamp = tf.keras.layers.Normalization(\n16         axis=None\n17     )\n18 \n19     self.normalized_timestamp.adapt(timestamps)\n20 \n21   def call(self, inputs):\n22     # Take the input dictionary, pass it through each input layer,\n23     # and concatenate the result.\n24     return tf.concat([\n25         self.user_embedding(inputs[\"user_id\"]),\n26         self.timestamp_embedding(inputs[\"timestamp\"]),\n27         tf.reshape(self.normalized_timestamp(inputs[\"timestamp\"]), (-1, 1)),\n28     ], axis=1)\n

\n\n

The function tf.keras.layers.StringLookup is used to create an embedding layer from a sequence of unique string IDs. Timestamps for user selection events are fairly continuous so we use tf.keras.layers.Discretization to collapse a wide range of timestamp values into discrete bins.

Classed derived from class tf.keras.Model are expected to implement a call method that is passed a inputs and returns a single Tensor of concatenated inputs and timestamp embeddings.

We build a similar model for movies:

 1 class MovieModel(tf.keras.Model):\n 2   \n 3   def __init__(self):\n 4     super().__init__()\n 5 \n 6     max_tokens = 10_000\n 7 \n 8     self.title_embedding = tf.keras.Sequential([\n 9       tf.keras.layers.StringLookup(\n10           vocabulary=unique_movie_titles,mask_token=None),\n11       tf.keras.layers.Embedding(len(unique_movie_titles) + 1, 32)\n12     ])\n13 \n14     self.title_vectorizer = tf.keras.layers.TextVectorization(\n15         max_tokens=max_tokens)\n16 \n17     self.title_text_embedding = tf.keras.Sequential([\n18       self.title_vectorizer,\n19       tf.keras.layers.Embedding(max_tokens, 32, mask_zero=True),\n20       tf.keras.layers.GlobalAveragePooling1D(),\n21     ])\n22 \n23     self.title_vectorizer.adapt(movies)\n24 \n25   def call(self, titles):\n26     return tf.concat([\n27         self.title_embedding(titles),\n28         self.title_text_embedding(titles),\n29     ], axis=1)\n

\n\n

The class MovieModel is different than the class UserModel since we create embeddings for movie titles instead of IDs.

We also wrap the user model in a separate query model that combines a user model with dense fully connected layers:

 1 class QueryModel(tf.keras.Model):\n 2   \"\"\"Model for encoding user queries.\"\"\"\n 3 \n 4   def __init__(self, layer_sizes):\n 5     \"\"\"Model for encoding user queries.\n 6 \n 7     Args:\n 8       layer_sizes:\n 9         A list of integers where the i-th entry represents the number of units\n10         the i-th layer contains.\n11     \"\"\"\n12     super().__init__()\n13 \n14     # We first use the user model for generating embeddings.\n15     self.embedding_model = UserModel()\n16 \n17     # Then construct the layers.\n18     self.dense_layers = tf.keras.Sequential()\n19 \n20     # Use the ReLU activation for all but the last layer.\n21     for layer_size in layer_sizes[:-1]:\n22       self.dense_layers.add(tf.keras.layers.Dense(layer_size, activation=\"relu\"))\n23 \n24     # No activation for the last layer.\n25     for layer_size in layer_sizes[-1:]:\n26       self.dense_layers.add(tf.keras.layers.Dense(layer_size))\n27     \n28   def call(self, inputs):\n29     feature_embedding = self.embedding_model(inputs)\n30     return self.dense_layers(feature_embedding)\n

\n\n

The call method returns the values calculated from feeding the input layer into the dense fully connected layers that have a relu non-linear activation function.

We also wrap the movie model in a candidate recommendation model:

 1 class CandidateModel(tf.keras.Model):\n 2   \"\"\"Model for encoding movies.\"\"\"\n 3 \n 4   def __init__(self, layer_sizes):\n 5     \"\"\"Model for encoding movies.\n 6 \n 7     Args:\n 8       layer_sizes:\n 9         A list of integers where the i-th entry represents the number of units\n10         the i-th layer contains.\n11     \"\"\"\n12     super().__init__()\n13 \n14     self.embedding_model = MovieModel()\n15 \n16     # Then construct the layers.\n17     self.dense_layers = tf.keras.Sequential()\n18 \n19     # Use the ReLU activation for all but the last layer.\n20     for layer_size in layer_sizes[:-1]:\n21       self.dense_layers.add(tf.keras.layers.Dense(layer_size, activation=\"relu\"))\n22 \n23     # No activation for the last layer.\n24     for layer_size in layer_sizes[-1:]:\n25       self.dense_layers.add(tf.keras.layers.Dense(layer_size))\n26     \n27   def call(self, inputs):\n28     feature_embedding = self.embedding_model(inputs)\n29     return self.dense_layers(feature_embedding)\n

\n\n

The call method returns the values calculated from feeding the input layer for a movie model into the dense fully connected layers that have a relu non-linear activation function.

We finally train a deep learning model by creating an instance of class MovielensModel and calling its inherited fit method:

 1 model = MovielensModel([64, 32])\n 2 model.compile(optimizer=tf.keras.optimizers.Adagrad(0.1))\n 3 \n 4 two_layer_history = model.fit(\n 5     cached_train,\n 6     validation_data=cached_test,\n 7     validation_freq=5,\n 8     epochs=num_epochs,\n 9     verbose=0)\n10 \n11 accuracy = two_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"][-1]\n12 print(f\"Top-100 accuracy: {accuracy:.2f}.\")\n

\n\n

The output looks like:

1 Top-100 accuracy: 0.29.\n

\n\n

This top-100 accuracy means that if you make a movie recommendation, it has a 29% chance of being in the top 100 recommendations for a user.

We can plot the training accuracy vs. training epoch for both one and two layers:

The example Google Colab project has an additional training run that gets better accuracy by stacking many additional hidden layers in the user and movie wrapper Python classes.

Recommendation Systems Wrap-up

If you need to write a recommendation system for your work then I hope this short overview chapter will get you started. Here are alternative approaches and a few resources:

\nConsider using Amazon Personalize which is a turn-key service on AWS.\n
\nConsider using Google’s turn-key Recommendations AI.\n
\nEric Lundquist has written a Python library rankfm for factorization machines for recommendation and ranking problems.\n
\nIf product data includes pictures then consider using this Keras example as a guide for creating embeddings for images and implementing image search.\n
\nA research idea: a Keras example that was written by Khalid Salama that transforms input training data to a textual representation for input to a Transformer model. This example is based on the paper Behavior Sequence Transformer for E-commerce Recommendation in Alibaba by Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou.\n

Part 3 - Large Language Models

Large Language Models (LLMs) are arguably the most transformative development in AI in the last decade. Built on the transformer architecture, LLMs have moved from research curiosities to indispensable tools used daily by millions of people for writing, coding, analysis, and creative work.

We start with an introduction to the transformer architecture, self-attention, tokenization, and the training techniques that produce modern LLMs. We then move to practical chapters: using LLMs through cloud APIs from Google Gemini and OpenAI, and running open-weights models locally on your own hardware with Ollama.

Introduction to Transformers and Large Language Models

The transformer architecture, introduced in the 2017 paper “Attention Is All You Need” by Vaswani et al., fundamentally changed artificial intelligence. Before transformers, recurrent neural networks (RNNs) and their variants (LSTMs, GRUs) were the dominant approach for processing sequential data like text. These models processed tokens one at a time, which made them slow to train and poor at capturing relationships between words that were far apart in a sentence.

Transformers replaced this sequential processing with a mechanism called self-attention that allows every token in a sequence to attend to every other token simultaneously. This parallel processing made transformers dramatically faster to train on modern GPU hardware, and the attention mechanism proved remarkably effective at capturing the complex patterns in language. Within a few years, transformer-based models achieved state-of-the-art results across virtually every natural language processing benchmark, and the architecture has since been successfully applied to computer vision, protein folding, music generation, and many other domains.

Large Language Models (LLMs) are transformer models trained on vast corpora of text — often trillions of tokens scraped from the internet, books, and code repositories. Through this training, LLMs develop an impressive ability to generate coherent text, answer questions, write code, reason about problems, and follow complex instructions. Models like Google’s Gemini, Anthropic’s Claude, OpenAI’s GPT series, and Meta’s Llama family have become indispensable tools for developers and knowledge workers.

This chapter covers the core concepts behind transformers and LLMs. The practical work of using these models — calling cloud APIs and running models locally — is covered in the next two chapters.

The Transformer Architecture

The transformer architecture consists of two main components: an encoder that reads and processes an input sequence, and a decoder that generates an output sequence. Some models use both (the original transformer for translation), while modern LLMs typically use only the decoder (GPT, Claude, Gemini, Llama) or only the encoder (BERT, for understanding tasks).

Self-Attention: The Core Innovation

The key insight of the transformer is the self-attention mechanism. For each token in a sequence, self-attention computes how much that token should “attend to” every other token. This is done by projecting each token’s embedding into three vectors:

\nQuery (Q): What this token is looking for\n
\nKey (K): What this token offers to other tokens\n
\nValue (V): The actual information this token contributes\n

The attention score between two tokens is computed as the dot product of the query of one token with the key of another, scaled by the square root of the dimension. These scores are passed through a softmax function to produce weights, which are then used to compute a weighted sum of the value vectors. The result is that each token’s representation becomes a context-aware blend of information from all tokens in the sequence.

The mathematical formulation is:

1 Attention(Q, K, V) = softmax(QK^T / √d_k) V\n

\n\n

where d_k is the dimension of the key vectors. The scaling factor √d_k prevents the dot products from growing too large, which would push the softmax into regions with very small gradients.

Multi-Head Attention

Rather than computing a single attention function, transformers use multi-head attention: the Q, K, and V projections are split into multiple parallel “heads,” each learning to attend to different aspects of the input. One head might learn syntactic relationships (subject-verb agreement), another might learn semantic relationships (word meaning in context), and another might learn positional patterns. The outputs of all heads are concatenated and projected back to the model’s dimension.

This mechanism allows the model to simultaneously capture multiple types of relationships between tokens, which is far more expressive than a single attention computation.

Positional Encoding

Since the self-attention mechanism processes all tokens in parallel, it has no inherent notion of token order — the sentence “the cat sat on the mat” would produce the same attention scores as “mat the on sat cat the” without additional information. Positional encodings solve this by adding position information to each token’s embedding before it enters the transformer layers.

The original transformer used fixed sinusoidal functions of different frequencies for positional encoding. Modern models like Llama use Rotary Position Embeddings (RoPE), which encode position information directly into the attention computation and generalize better to sequence lengths not seen during training.

Layer Structure

Each transformer layer consists of:

\nMulti-head self-attention — tokens attend to each other\n
\nLayer normalization — stabilizes training by normalizing activations\n
\nFeed-forward network — a two-layer neural network applied independently to each token position, providing additional capacity for learning complex transformations\n
\nResidual connections — the input to each sub-layer is added to its output, which helps gradients flow during training and allows deeper networks\n

A typical LLM stacks dozens of these layers. For example, GPT-3 has 96 layers, and Llama 3 70B has 80 layers.

Encoder-Only, Decoder-Only, and Encoder-Decoder Models

The original transformer used both an encoder and decoder for machine translation: the encoder processes the source language and the decoder generates the target language, attending to both its own output and the encoder’s representation.

In practice, three variants have emerged:

\nEncoder-only (e.g., BERT): Processes the full input bidirectionally. Excellent for understanding tasks like classification, named entity recognition, and semantic similarity. Each token can attend to all other tokens in both directions.\n
\nDecoder-only (e.g., GPT, Claude, Gemini, Llama): Processes tokens left-to-right, where each token can only attend to tokens that came before it (causal masking). This is the architecture used by virtually all modern LLMs because it naturally supports text generation: predict the next token given all previous tokens.\n
\nEncoder-decoder (e.g., T5, the original transformer): Uses both components. Still used for some translation and summarization tasks, but decoder-only models have largely superseded this approach for general-purpose language modeling.\n

Tokenization

Before text can be processed by a transformer, it must be converted into a sequence of integers — a process called tokenization. The choice of tokenization strategy has significant practical implications for model performance, cost, and capability.

Why Not Just Use Characters or Words?

Character-level tokenization produces very long sequences (a single word might be 5-10 tokens), which is computationally expensive because the self-attention mechanism scales quadratically with sequence length. Word-level tokenization has the opposite problem: the vocabulary must be enormous to cover all possible words, and any word not in the vocabulary (misspellings, technical jargon, foreign words) cannot be represented at all.

Modern tokenizers use a middle ground: subword tokenization, which splits text into chunks that balance vocabulary size against sequence length.

Byte Pair Encoding (BPE)

BPE, used by GPT models and Llama, starts with individual characters and iteratively merges the most frequent adjacent pairs. For example, starting with the characters “l”, “o”, “w”, the frequent pair “lo” would be merged into a single token, then “low” might be merged next. After many iterations, common words become single tokens while rare words are split into meaningful subword pieces.

The result is a vocabulary of typically 32,000 to 128,000 tokens. Common English words like “the” or “and” are single tokens. Less common words are split into recognizable pieces: “tokenization” might become “token” + “ization”. This means the model can handle any text, including words it has never seen before, by decomposing them into known subword units.

WordPiece and SentencePiece

WordPiece, used by BERT and related models, is similar to BPE but uses a slightly different merging criterion based on likelihood rather than frequency.

SentencePiece is a language-independent tokenizer that treats the input as a raw byte stream rather than pre-tokenized words. This makes it effective for multilingual models because it does not assume whitespace-separated words (important for languages like Chinese and Japanese). SentencePiece is used by models like T5 and Llama.

Practical Implications of Tokenization

Understanding tokenization matters for practical use of LLMs:

\nCost: API providers charge per token, not per word. A word like “indistinguishable” might be 3-4 tokens, while “the” is one token. Roughly, one token is about 0.75 English words, or about 4 characters.\n
\nContext window: Models have a maximum number of tokens they can process at once (the context window). GPT-4 supports up to 128K tokens; Gemini 2.5 supports up to 1M tokens. Understanding token counts helps you estimate how much text fits in the context window.\n
\nNon-English text: Tokenizers trained primarily on English text tend to produce more tokens per word for other languages, which means non-English text uses more of the context window and costs more per word.\n
\nCode: Programming languages tokenize differently from natural language. Variable names, operators, and whitespace all consume tokens.\n

From Transformers to Large Language Models

The path from the transformer architecture to modern LLMs involved three key developments: scale, training methodology, and alignment.

Scale

The most striking trend in LLM development has been the dramatic scaling of model size and training data. The original transformer (2017) had 65 million parameters. GPT-2 (2019) had 1.5 billion. GPT-3 (2020) jumped to 175 billion. Current frontier models are estimated to have hundreds of billions to over a trillion parameters, trained on trillions of tokens of text.

This scaling has revealed emergent capabilities — abilities that appear only once a model reaches a certain size. Small models can complete simple sentences; large models can write essays, debug code, solve math problems, and engage in multi-step reasoning. The precise mechanisms behind these emergent capabilities are an active area of research.

Pre-training and the Next Token Prediction Objective

LLMs are pre-trained using a simple but powerful objective: predict the next token given all preceding tokens. The model reads vast amounts of text and learns to predict what comes next at each position. Despite the simplicity of this objective, it forces the model to learn grammar, facts, reasoning patterns, coding conventions, and much more — because predicting the next token accurately requires understanding the context deeply.

Pre-training is enormously expensive, typically requiring thousands of GPUs running for weeks or months, at a cost of millions to hundreds of millions of dollars. This is why only a handful of organizations train frontier models from scratch.

Instruction Tuning and RLHF

A raw pre-trained model is a powerful text predictor but not a useful assistant. It will continue any text you give it, but it does not naturally follow instructions or engage in helpful dialogue. Two additional training stages transform it into a useful tool:

\n
Instruction tuning (also called supervised fine-tuning or SFT): The model is trained on examples of instructions paired with high-quality responses. This teaches the model to follow directions rather than simply predict the next likely token.
\n
\n
Reinforcement Learning from Human Feedback (RLHF): Human evaluators rank model outputs by quality, and these rankings are used to train a reward model. The LLM is then fine-tuned using reinforcement learning to produce outputs that score highly according to the reward model. This process improves helpfulness, reduces harmful outputs, and aligns the model’s behavior with human preferences.
\n

Some newer approaches, such as Direct Preference Optimization (DPO), achieve similar results without explicitly training a separate reward model.

Key Capabilities of Modern LLMs

Modern LLMs exhibit several capabilities that emerge from their training:

In-Context Learning

LLMs can learn new tasks from examples provided in the prompt, without any changes to the model’s weights. By showing the model a few input-output examples (few-shot prompting), it can generalize and apply the pattern to new inputs. This is remarkable because the model is not being retrained — it is performing a form of learning purely through the attention mechanism at inference time.

Chain-of-Thought Reasoning

When prompted to “think step by step,” LLMs produce intermediate reasoning steps that lead to more accurate final answers, particularly for math, logic, and multi-step problems. This technique, called chain-of-thought prompting, significantly improves performance on tasks that require sequential reasoning.

Modern “reasoning models” like OpenAI’s o-series and Google’s Gemini 2.5 have been specifically trained to engage in extended internal reasoning before producing a response.

Tool Use and Agents

LLMs can be trained to use external tools — search engines, calculators, code interpreters, APIs — by generating structured function calls. This allows them to overcome their inherent limitations (such as lack of real-time information or difficulty with precise arithmetic) by delegating to specialized tools.

AI agents take this further by using LLMs as the reasoning core of autonomous systems that can plan multi-step tasks, execute actions, observe results, and adapt their approach. This is an active and rapidly evolving area of development.

Multimodal Understanding

Recent models accept not just text but also images, audio, and video as input. Google’s Gemini and OpenAI’s GPT-4o are natively multimodal, capable of analyzing photographs, reading charts, transcribing audio, and understanding video content. This expands LLMs from pure text processing to general-purpose perception and reasoning systems.

Practical Considerations

When working with LLMs there are two fundamentally different approaches, each covered in its own chapter:

Using Public APIs

Cloud providers offer access to frontier models through APIs. Google’s Gemini, OpenAI’s GPT models, and Anthropic’s Claude are available through simple HTTP calls or Python client libraries. This approach requires no GPU hardware and gives you access to the most capable models available. The tradeoffs are cost (you pay per token), latency (network round-trips), privacy (your data is sent to a third party), and dependence on the provider’s availability and pricing. We cover this approach in the next chapter, LLMs with Public APIs.

Running Local Models

Open-weights models like Meta’s Llama, Mistral, Qwen, and Google’s Gemma can be downloaded and run on your own hardware. Tools like Ollama and llama.cpp make it straightforward to run quantized models on consumer hardware, including laptops with Apple Silicon. Local models offer privacy, no per-token cost, and the ability to fine-tune for specific tasks. The tradeoffs are that local models are generally less capable than frontier API models, and running larger models requires significant GPU memory. We cover this approach in the chapter LLMs with Local Models.

Choosing Between APIs and Local Models

The choice depends on your requirements:

\nPrivacy-sensitive data: Use local models to keep data on your own hardware.\n
\nMaximum capability: Use frontier API models (Gemini, Claude, GPT) for the most challenging tasks.\n
\nCost at scale: Local models have no per-token cost once the hardware is available, making them economical for high-volume applications.\n
\nOffline or edge deployment: Local models can run without an internet connection.\n
\nRapid prototyping: APIs offer the fastest path from idea to working prototype.\n

Many production systems use a combination: a fast, inexpensive local model for routine tasks and an API call to a frontier model for complex ones.

LLMs with Public APIs

The fastest way to use large language models is through cloud APIs. Google, OpenAI, and Anthropic all offer APIs that give you access to their most capable models with just a few lines of Python code. You don’t need a GPU, you don’t need to download model weights, and you can start building applications in minutes.

In this chapter we work through practical examples using the Google Gemini API and the OpenAI API. Both provide Python client libraries that handle authentication, request formatting, and response parsing. The patterns you learn here apply to other API providers as well — the core concepts of sending prompts, receiving completions, and managing conversations are the same across providers.

The examples for this chapter are in the directory source-code/llm_public_apis.

Setup and Authentication

Google Gemini

Google’s Gemini models are accessed through the Google AI API using the google-genai Python SDK. You need a free API key from Google AI Studio.

Install the SDK:

1 uv pip install google-genai\n

\n\n

Store your API key in an environment variable:

1 export GOOGLE_API_KEY=\"your-api-key-here\"\n

\n\n

Here is the simplest possible example — send a prompt to Gemini and print the response:

 1 # gemini_text.py - Basic text generation with Google Gemini\n 2 \n 3 import os\n 4 from google import genai\n 5 \n 6 client = genai.Client(api_key=os.getenv(\"GOOGLE_API_KEY\"))\n 7 \n 8 response = client.models.generate_content(\n 9     model=\"gemini-3-flash-preview\",\n10     contents=\"Briefly explain what a transformer model is in AI.\"\n11 )\n12 \n13 print(response.text)\n

\n\n

The output will be a concise explanation of transformer models. Each call to generate_content sends a request to Google’s servers, which run the model and return the generated text.

OpenAI

OpenAI’s GPT models are accessed through the openai Python SDK. You need an API key from OpenAI’s platform.

Install the SDK:

1 uv pip install openai\n

\n\n

Store your API key:

1 export OPENAI_API_KEY=\"your-api-key-here\"\n

\n\n

Here is the equivalent example using OpenAI:

 1 # openai_text.py - Basic text generation with OpenAI\n 2 \n 3 from openai import OpenAI\n 4 \n 5 client = OpenAI()  # reads OPENAI_API_KEY from environment\n 6 \n 7 response = client.responses.create(\n 8     model=\"gpt-5.4-nano\",\n 9     input=\"Briefly explain what a transformer model is in AI.\"\n10 )\n11 \n12 output_items = list(response.output)\n13 for item in reversed(output_items):\n14     if getattr(type(item), \"__name__\", \"\") == \"ResponseOutputMessage\":\n15         for content in item.content:\n16             if type(content).__name__ == \"ResponseOutputText\":\n17                 print(content.text)\n18                 break\n

\n\n

Both APIs follow the same pattern: create a client, send a prompt, and extract the generated text from the response.

Text Generation

Text generation is the most fundamental LLM capability. You provide a prompt and the model generates a continuation or response.

Controlling Output with Temperature

The temperature parameter controls how creative or deterministic the output is. A temperature of 0 produces the most predictable output (the model always picks the highest-probability next token). Higher temperatures (up to 1.0 or 2.0) produce more varied and creative output.

 1 # gemini_temperature.py - Effect of temperature on text generation\n 2 \n 3 import os\n 4 from google import genai\n 5 from google.genai import types\n 6 \n 7 client = genai.Client(api_key=os.getenv(\"GOOGLE_API_KEY\"))\n 8 \n 9 prompt = \"Write a one-sentence tagline for a coffee shop.\"\n10 \n11 # Low temperature: deterministic, predictable\n12 response_low = client.models.generate_content(\n13     model=\"gemini-3-flash-preview\",\n14     contents=prompt,\n15     config=types.GenerateContentConfig(temperature=0.0)\n16 )\n17 print(f\"Temperature 0.0: {response_low.text}\")\n18 \n19 # High temperature: creative, varied\n20 response_high = client.models.generate_content(\n21     model=\"gemini-3-flash-preview\",\n22     contents=prompt,\n23     config=types.GenerateContentConfig(temperature=1.5)\n24 )\n25 print(f\"Temperature 1.5: {response_high.text}\")\n

\n\n

For most practical applications — code generation, data extraction, question answering — use a low temperature (0.0 to 0.3). For creative writing and brainstorming, higher temperatures (0.7 to 1.5) produce more interesting results.

Thinking Models

Some models can engage in extended internal reasoning before producing a response. Google’s Gemini 2.5 Flash supports a thinking budget that controls how much computation the model devotes to reasoning through the problem before answering.

 1 # gemini_thinking.py - Using Gemini's thinking mode for complex reasoning\n 2 \n 3 from google import genai\n 4 from google.genai import types\n 5 import os\n 6 \n 7 client = genai.Client(api_key=os.getenv(\"GOOGLE_API_KEY\"))\n 8 \n 9 prompt = \"\"\"\n10 A farmer has a fox, a chicken, and a bag of grain. He needs to cross\n11 a river in a boat that can only carry him and one item at a time.\n12 If left alone, the fox will eat the chicken, and the chicken will eat\n13 the grain. How does the farmer get everything across safely?\n14 \"\"\"\n15 \n16 response = client.models.generate_content(\n17     model=\"gemini-3-flash-preview\",\n18     contents=prompt,\n19     config=types.GenerateContentConfig(\n20         thinking_config=types.ThinkingConfig(\n21             thinking_budget=1000  # allow up to 1000 tokens of reasoning\n22         )\n23     )\n24 )\n25 \n26 print(response.text)\n

\n\n

The thinking budget is specified in tokens. A budget of 0 disables thinking entirely (useful for simple tasks where speed matters). Higher budgets allow the model to reason through more complex problems but increase latency and cost.

Multi-Turn Conversations

Real applications often involve multi-turn conversations where the model needs to remember previous exchanges. Both APIs support this by passing conversation history with each request.

 1 # gemini_conversation.py - Multi-turn conversation with Gemini\n 2 \n 3 import os\n 4 from google import genai\n 5 from google.genai import types\n 6 \n 7 client = genai.Client(api_key=os.getenv(\"GOOGLE_API_KEY\"))\n 8 \n 9 # Build a conversation as a list of content parts\n10 conversation = []\n11 \n12 def chat(user_message):\n13     \"\"\"Send a message and get a response, maintaining conversation history.\"\"\"\n14     conversation.append(\n15         types.Content(role=\"user\", parts=[types.Part(text=user_message)])\n16     )\n17     response = client.models.generate_content(\n18         model=\"gemini-3-flash-preview\",\n19         contents=conversation\n20     )\n21     conversation.append(\n22         types.Content(role=\"model\", parts=[types.Part(text=response.text)])\n23     )\n24     return response.text\n25 \n26 # A multi-turn conversation\n27 print(chat(\"What is the capital of France?\"))\n28 print(chat(\"What is its population?\"))  # \"its\" refers to Paris from context\n29 print(chat(\"What are the top 3 tourist attractions there?\"))\n

\n\n

Notice that the second and third messages use pronouns (“its”, “there”) that only make sense given the conversation history. The model resolves these references correctly because it sees the full conversation with each request.

Multimodal Input: Analyzing Images

Modern LLMs can process images alongside text. This enables applications like image description, document analysis, chart reading, and visual question answering.

 1 # gemini_image.py - Analyzing an image with Gemini\n 2 \n 3 from google import genai\n 4 from google.genai import types\n 5 from PIL import Image\n 6 import os\n 7 \n 8 client = genai.Client(api_key=os.getenv(\"GOOGLE_API_KEY\"))\n 9 \n10 # Load an image from disk\n11 image = Image.open(\"photo.jpg\")\n12 \n13 prompt = \"Describe what you see in this image. Be specific about people, objects, and setting.\"\n14 \n15 response = client.models.generate_content(\n16     model=\"gemini-3-flash-preview\",\n17     contents=[prompt, image],\n18     config=types.GenerateContentConfig(\n19         thinking_config=types.ThinkingConfig(\n20             thinking_budget=0  # no thinking needed for simple description\n21         )\n22     )\n23 )\n24 \n25 print(response.text)\n

\n\n

The key detail is that contents accepts a list containing both text and image objects. The model processes them together, understanding the image in the context of the text prompt.

Web Search with LLMs

Some API providers allow the model to search the web as part of generating a response, which gives it access to current information beyond its training data.

Here is an example using OpenAI’s web search tool:

 1 # openai_search.py - Web search with OpenAI\n 2 \n 3 from openai import OpenAI\n 4 \n 5 client = OpenAI()\n 6 \n 7 response = client.responses.create(\n 8     model=\"gpt-5.4-nano\",\n 9     tools=[{\"type\": \"web_search_preview\"}],\n10     input=\"What were the major AI announcements at Google I/O 2025?\"\n11 )\n12 \n13 output_items = list(response.output)\n14 for item in reversed(output_items):\n15     kind = getattr(type(item), \"__name__\", \"\")\n16     if kind == \"ResponseOutputMessage\" and getattr(item, \"role\", None) == \"assistant\":\n17         for content in item.content:\n18             if type(content).__name__ == \"ResponseOutputText\":\n19                 print(content.text)\n20                 break\n

\n\n

The tools parameter tells the model it can use web search to answer the question. The model decides whether to search based on the query — factual questions about recent events will trigger a search, while questions about well-known topics may not.

Google’s Gemini also supports grounding with Google Search through a similar mechanism. Refer to the Google AI documentation for the current syntax, as this feature is actively evolving.

Structured Output

For many applications you need the model to return data in a specific format — JSON, CSV, or a particular schema. LLMs can be instructed to produce structured output through careful prompting.

 1 # gemini_structured.py - Getting structured JSON output from Gemini\n 2 \n 3 import os\n 4 import json\n 5 from google import genai\n 6 from google.genai import types\n 7 \n 8 client = genai.Client(api_key=os.getenv(\"GOOGLE_API_KEY\"))\n 9 \n10 prompt = \"\"\"Extract the following information from the text below and return\n11 it as a JSON object with keys: \"name\", \"company\", \"role\", \"years_experience\".\n12 \n13 Text: \"Jane Smith has been working as a Senior Data Scientist at Acme Corp\n14 for the past 7 years. She specializes in NLP and recommendation systems.\"\n15 \"\"\"\n16 \n17 response = client.models.generate_content(\n18     model=\"gemini-3-flash-preview\",\n19     contents=prompt,\n20     config=types.GenerateContentConfig(temperature=0.0)\n21 )\n22 \n23 # Parse the JSON from the response\n24 result = json.loads(response.text.strip().removeprefix(\"```json\").removesuffix(\"```\").strip())\n25 print(json.dumps(result, indent=2))\n

\n\n

Using temperature 0.0 is important for structured output — you want the model to be deterministic and precise rather than creative. Some APIs also support specifying a JSON schema directly in the request, which guarantees the output conforms to a specific structure.

Practical Considerations

Cost

API calls are billed per token. Input tokens (your prompt) and output tokens (the model’s response) are priced separately, with output tokens typically costing 2-4x more. Prices vary significantly between providers and models:

\nSmaller, faster models (Gemini 2.5 Flash, GPT-5.4-nano) are very inexpensive — often under $0.10 per million input tokens\n
\nFrontier models (Gemini 2.5 Pro, GPT-5.4, Claude Opus) cost 10-50x more but offer superior reasoning\n

For most applications, start with a fast, inexpensive model and only upgrade to a frontier model for tasks that require it.

Rate Limits

All API providers enforce rate limits — maximum requests per minute, tokens per minute, and tokens per day. Free tiers have lower limits. If you’re building a production application, you’ll need to implement retry logic with exponential backoff and consider batching requests where possible.

Latency

API calls involve network round-trips and model inference time. Simple completions with small models return in under a second. Complex reasoning with frontier models can take 10-30 seconds or more. For interactive applications, consider streaming responses (both Gemini and OpenAI support this) so users see output as it’s generated rather than waiting for the complete response.

Privacy

Any data you send to an API is transmitted to the provider’s servers. For sensitive data — medical records, financial information, proprietary code — review the provider’s data usage policies carefully. Some providers offer data residency guarantees and opt-out options for training. For maximum privacy, consider using local models instead, as covered in the next chapter.

Error Handling

API calls can fail for many reasons: network errors, rate limiting, content filtering, malformed requests, or service outages. Production code should handle these gracefully:

 1 import time\n 2 \n 3 def generate_with_retry(client, prompt, max_retries=3):\n 4     \"\"\"Call the Gemini API with exponential backoff on failure.\"\"\"\n 5     for attempt in range(max_retries):\n 6         try:\n 7             response = client.models.generate_content(\n 8                 model=\"gemini-3-flash-preview\",\n 9                 contents=prompt\n10             )\n11             return response.text\n12         except Exception as e:\n13             if attempt < max_retries - 1:\n14                 wait = 2 ** attempt  # 1s, 2s, 4s\n15                 print(f\"Attempt {attempt + 1} failed: {e}. Retrying in {wait}s...\")\n16                 time.sleep(wait)\n17             else:\n18                 raise\n

\n\n

Summary

Using LLMs through public APIs is the fastest path from idea to working application. The core pattern is simple across all providers: create a client, send a prompt, process the response. The richness comes from features like multi-turn conversations, multimodal input, web search, structured output, and thinking modes.

The main tradeoffs of the API approach are cost (per-token pricing), privacy (data leaves your machine), and dependence on the provider’s availability. For applications where these tradeoffs are acceptable, public APIs give you access to the most capable models available.

In the next chapter we cover the alternative approach: running open-weights models locally on your own hardware, which offers privacy, no per-token cost, and offline operation at the expense of model capability and the need for suitable hardware.

LLMs with Local Models

Running language models on your own hardware gives you privacy, zero per-token cost, and the ability to work offline. The tradeoff is that local models are generally smaller and less capable than the frontier models available through cloud APIs, and running larger models requires significant GPU memory or Apple Silicon unified memory.

In this chapter we focus on Ollama, the most popular tool for running local models. Ollama handles model downloading, quantization, GPU acceleration, and exposes a simple API — you can go from zero to a running local LLM in minutes. We also briefly mention alternative tools at the end of the chapter.

If you want to go deeper into Ollama, including tool use, agents, RAG, and advanced configuration, see my book Ollama in Action.

The examples for this chapter are in the directory source-code/llm_local_models.

Installing Ollama

Ollama is available for macOS, Linux, and Windows. On macOS:

1 brew install ollama\n

\n\n

Or download the installer from ollama.com. After installation, start the Ollama service:

1 ollama serve\n

\n\n

This starts a local server on port 11434. The service runs in the background and manages model loading, GPU memory, and request handling.

Downloading and Running Models

Ollama uses a Docker-like model for pulling and running models. To download a model:

1 ollama pull llama3.2:3b\n

\n\n

This downloads Meta’s Llama 3.2 with 3 billion parameters, quantized to about 2 GB. You can interact with it immediately from the command line:

1 ollama run llama3.2:3b \"What is the capital of France?\"\n

\n\n

Some recommended models to start with:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

Model	Size	Strengths
llama3.2:3b	2 GB	Fast, good general purpose
gemma3:4b	3 GB	Google’s small model, strong reasoning
qwen3:4b	2.6 GB	Excellent multilingual and coding
deepseek-r1:7b	4.7 GB	Strong reasoning with explicit chain-of-thought
llava:7b	4.7 GB	Vision model — can analyze images

Using Ollama from Python

The ollama Python SDK provides a clean interface to the local Ollama service.

1 uv pip install ollama\n

\n\n

Basic Text Generation

The simplest use of the Ollama SDK — send a prompt and print the response:

 1 # ollama_text.py - Basic text generation with a local model\n 2 \n 3 import ollama\n 4 \n 5 response = ollama.chat(\n 6     model=\"llama3.2:3b\",\n 7     messages=[\n 8         {\"role\": \"user\", \"content\": \"Briefly explain what a neural network is.\"}\n 9     ]\n10 )\n11 \n12 print(response.message.content)\n

\n\n

This is similar in structure to the cloud API examples from the previous chapter, but the request never leaves your machine.

Streaming Responses

For interactive applications, streaming lets users see output as it’s generated rather than waiting for the complete response:

 1 # ollama_streaming.py - Streaming responses for real-time output\n 2 \n 3 import ollama\n 4 \n 5 stream = ollama.chat(\n 6     model=\"llama3.2:3b\",\n 7     messages=[\n 8         {\"role\": \"user\", \"content\": \"Write a short poem about programming.\"}\n 9     ],\n10     stream=True\n11 )\n12 \n13 for chunk in stream:\n14     print(chunk.message.content, end=\"\", flush=True)\n15 print()  # final newline\n

\n\n

Each chunk contains a small piece of the response. The flush=True argument ensures text appears immediately rather than being buffered.

Reasoning with Local Models

Some local models support explicit chain-of-thought reasoning, where the model shows its thinking process before providing a final answer. DeepSeek-R1 is particularly good at this.

First pull the model:

1 ollama pull deepseek-r1:7b\n

\n\n

Here is an example that extracts both the reasoning trace and the final answer:

 1 # ollama_reasoning.py - Chain-of-thought reasoning with DeepSeek-R1\n 2 \n 3 import ollama\n 4 import json\n 5 \n 6 def reason_about(question: str, model: str = \"deepseek-r1:7b\") -> dict:\n 7     \"\"\"Ask a question and extract both reasoning and final answer.\"\"\"\n 8     response = ollama.chat(\n 9         model=model,\n10         messages=[\n11             {\"role\": \"user\", \"content\": question}\n12         ]\n13     )\n14     content = response.message.content\n15 \n16     # DeepSeek-R1 wraps reasoning in <think>...</think> tags\n17     reasoning = \"\"\n18     answer = content\n19     if \"<think>\" in content and \"</think>\" in content:\n20         reasoning = content.split(\"<think>\")[1].split(\"</think>\")[0].strip()\n21         answer = content.split(\"</think>\")[1].strip()\n22 \n23     return {\"reasoning\": reasoning, \"answer\": answer}\n24 \n25 \n26 question = (\n27     \"A bakery sells 3 types of bread. Each type comes in 2 sizes. \"\n28     \"How many different bread options are available? \"\n29     \"Respond with just the number and a brief explanation.\"\n30 )\n31 \n32 result = reason_about(question)\n33 \n34 if result[\"reasoning\"]:\n35     print(\"=== Reasoning ===\")\n36     print(result[\"reasoning\"])\n37     print()\n38 \n39 print(\"=== Answer ===\")\n40 print(result[\"answer\"])\n

\n\n

The model’s reasoning trace shows each step of its thinking, making the output more transparent and debuggable than a black-box answer. This is especially valuable for math, logic, and planning tasks.

Conversation Memory with Ollama

Cloud APIs handle conversation history by passing the full message list with each request. With local models the same pattern applies, but since there are no per-token costs, you can maintain longer conversations without worrying about expense.

Here is an example that maintains a conversation with memory across multiple exchanges and uses a system prompt to shape the assistant’s personality:

 1 # ollama_memory.py - Conversation with persistent memory\n 2 \n 3 import ollama\n 4 \n 5 class LocalAssistant:\n 6     \"\"\"A simple conversational assistant that maintains message history.\"\"\"\n 7 \n 8     def __init__(self, model: str = \"llama3.2:3b\", system_prompt: str = \"\"):\n 9         self.model = model\n10         self.messages = []\n11         if system_prompt:\n12             self.messages.append({\"role\": \"system\", \"content\": system_prompt})\n13 \n14     def chat(self, user_message: str) -> str:\n15         \"\"\"Send a message and get a response, maintaining conversation history.\"\"\"\n16         self.messages.append({\"role\": \"user\", \"content\": user_message})\n17         response = ollama.chat(model=self.model, messages=self.messages)\n18         reply = response.message.content\n19         self.messages.append({\"role\": \"assistant\", \"content\": reply})\n20         return reply\n21 \n22     def message_count(self) -> int:\n23         \"\"\"Return the number of messages in the conversation history.\"\"\"\n24         return len(self.messages)\n25 \n26 \n27 # Create an assistant with a specific personality\n28 assistant = LocalAssistant(\n29     system_prompt=\"You are a concise technical writing assistant. \"\n30                   \"Keep answers under 3 sentences.\"\n31 )\n32 \n33 # Multi-turn conversation — the model remembers prior context\n34 print(\"Q:\", \"What is gradient descent?\")\n35 print(\"A:\", assistant.chat(\"What is gradient descent?\"))\n36 print()\n37 \n38 print(\"Q:\", \"How does the learning rate affect it?\")\n39 print(\"A:\", assistant.chat(\"How does the learning rate affect it?\"))\n40 print()\n41 \n42 print(\"Q:\", \"What happens if I set it too high?\")\n43 print(\"A:\", assistant.chat(\"What happens if I set it too high?\"))\n44 print()\n45 \n46 print(f\"(Conversation has {assistant.message_count()} messages)\")\n

\n\n

Note that unlike cloud APIs, keeping long conversation histories in local models is free — there are no per-token costs. The main constraint is the model’s context window size, which varies by model (typically 4K to 128K tokens).

Prompt Caching for Performance

When you send the same long context (a document, a knowledge base, or a detailed system prompt) with multiple questions, Ollama can cache the prompt processing to dramatically speed up subsequent requests. This happens automatically when the prefix of the prompt is identical across requests.

Here is an example that demonstrates the speedup:

 1 # ollama_caching.py - Prompt caching benchmark\n 2 \n 3 import requests\n 4 import time\n 5 \n 6 OLLAMA_URL = \"http://localhost:11434/api/generate\"\n 7 MODEL = \"llama3.2:3b\"\n 8 \n 9 # A long static context that stays the same across queries\n10 CONTEXT = \"\"\"\n11 The Python programming language was created by Guido van Rossum and first\n12 released in 1991. Python's design philosophy emphasizes code readability\n13 with its notable use of significant whitespace. Python is dynamically typed\n14 and garbage-collected. It supports multiple programming paradigms, including\n15 structured, object-oriented, and functional programming.\n16 \n17 Python consistently ranks as one of the most popular programming languages.\n18 It is widely used in web development, data science, machine learning,\n19 automation, and scientific computing. The language's large standard library\n20 and extensive ecosystem of third-party packages make it suitable for a\n21 wide range of applications.\n22 \"\"\" * 20  # repeat to create a substantial context\n23 \n24 \n25 def timed_query(question: str, label: str) -> float:\n26     \"\"\"Send a query with the shared context and measure prompt processing time.\"\"\"\n27     payload = {\n28         \"model\": MODEL,\n29         \"keep_alive\": \"60m\",  # keep model and cache in memory\n30         \"prompt\": f\"{CONTEXT}\\n\\nQuestion: {question}\",\n31         \"stream\": False,\n32         \"options\": {\"num_ctx\": 4096}\n33     }\n34     start = time.time()\n35     resp = requests.post(OLLAMA_URL, json=payload)\n36     elapsed = time.time() - start\n37     data = resp.json()\n38 \n39     # prompt_eval_duration is in nanoseconds\n40     eval_ms = data.get(\"prompt_eval_duration\", 0) / 1_000_000\n41     print(f\"[{label}] Wall time: {elapsed:.2f}s | Prompt eval: {eval_ms:.0f}ms\")\n42     return eval_ms\n43 \n44 \n45 # First request: cold start, processes the full context\n46 time_a = timed_query(\"When was Python created?\", \"Cold start\")\n47 \n48 # Second request: same context prefix, different question — cache hit\n49 time_b = timed_query(\"What paradigms does Python support?\", \"Cache hit\")\n50 \n51 if time_a > 0 and time_b > 0:\n52     print(f\"\\nSpeedup: {time_a / time_b:.1f}x faster on cached prompt\")\n

\n\n

The key settings for prompt caching:

\nkeep_alive: Set to a long duration (e.g., “60m”) so the model and its KV cache stay in memory between requests.\n
\nIdentical prefix: The cached portion must be exactly the same. If even one character of the context changes, the cache is invalidated.\n
\nConsistent num_ctx: The context window size must match between requests.\n

Prompt caching is especially valuable for applications like document Q&A, where you load a long document once and then answer many questions about it.

OpenAI-Compatible API

Ollama exposes an OpenAI-compatible API endpoint, which means you can use the standard openai Python library to talk to local models. This is useful if you want to write code that can switch between cloud and local models by changing only the base URL:

 1 # ollama_openai_compat.py - Using local Ollama with the OpenAI SDK\n 2 \n 3 from openai import OpenAI\n 4 \n 5 # Point the OpenAI client at the local Ollama server\n 6 client = OpenAI(\n 7     base_url=\"http://localhost:11434/v1\",\n 8     api_key=\"not-needed\"  # Ollama doesn't require authentication locally\n 9 )\n10 \n11 response = client.chat.completions.create(\n12     model=\"llama3.2:3b\",\n13     messages=[\n14         {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n15         {\"role\": \"user\", \"content\": \"What is the difference between a list and a tuple in Python?\"}\n16     ],\n17     temperature=0.7\n18 )\n19 \n20 print(response.choices[0].message.content)\n

\n\n

This compatibility layer means you can prototype with local models and then switch to OpenAI, Gemini, or another provider by changing the client configuration — the rest of your code stays the same.

Alternative Tools for Running Local Models

While Ollama is the system I usually use for running local models, several alternatives exist:

\n
llama.cpp: The C++ inference engine that Ollama is built on. Use it directly if you need maximum control over quantization, batching, or want to embed inference in a C/C++ application. Available at github.com/ggerganov/llama.cpp.
\n
\n
LM Studio: A desktop application with a graphical interface for downloading, managing, and chatting with local models. Good for non-programmers or for quickly trying different models. Available at lmstudio.ai.
\n
\n
vLLM: A high-performance inference server optimized for throughput. Best suited for serving models to multiple users in production. Requires more GPU memory but can handle many concurrent requests efficiently. Available at github.com/vllm-project/vllm.
\n
\n
Hugging Face Transformers: The transformers Python library can load and run models directly. This gives you the most flexibility for fine-tuning and custom inference pipelines, but requires more setup and GPU memory management. Best for researchers and advanced users.
\n

For most developers getting started with local models, Ollama provides the best balance of simplicity and capability.

Hardware Considerations

The amount of memory you need depends on the model size:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

Model Parameters	Quantized Size	Minimum RAM/VRAM
1-3B	1-2 GB	8 GB RAM
7-8B	4-5 GB	16 GB RAM
14B	8-9 GB	16 GB RAM
32-70B	18-40 GB	32-64 GB RAM

On macOS with Apple Silicon (M1/M2/M3/M4), models run on the GPU using unified memory, which means your total system RAM is also your GPU memory. A MacBook with 16 GB of RAM can comfortably run 7-8B parameter models, and 32 GB or more enables larger models.

On Linux and Windows, a dedicated NVIDIA GPU with sufficient VRAM provides the best performance. Models can also run on CPU only, but inference is significantly slower (roughly 5-10x slower than GPU for most models).

Summary

Running LLMs locally with Ollama gives you a private, cost-free, offline-capable alternative to cloud APIs. The setup is straightforward — install Ollama, pull a model, and start making API calls from Python. Features like streaming, conversation memory, prompt caching, and reasoning models make local models practical for many real applications.

The main tradeoff is capability: the largest models that run locally (7-14B parameters on typical hardware) are less capable than frontier cloud models with hundreds of billions of parameters. For many tasks — code assistance, text summarization, data extraction, conversational interfaces — local models perform well enough, and the privacy and cost benefits make them the better choice.

Part 4 - Symbolic AI and Knowledge Representation

While deep learning and LLMs dominate modern AI, symbolic AI techniques remain valuable for problems that require explicit reasoning, verifiable logic, and structured knowledge. In this part we explore both classical symbolic AI and practical knowledge representation.

We cover Prolog for logic programming and inference, MiniZinc for constraint satisfaction, the Soar cognitive architecture, and frame-based knowledge representation. We then turn to modern knowledge representation using graph databases, relational databases, and Semantic Web technologies including RDF, SPARQL, and linked data — tools that are widely used in enterprise AI systems today.

Symbolic AI

When I started my paid career as an AI practitioner in 1982 my company bought me a Xerox 1108 Lisp Machine and I spent every spare moment I had working through two books by Patrick Winston that I had purchased a few years earlier: “Lisp” and “Artificial Intelligence.” This material was mostly what is now called symbolic AI or good old fashioned AI (GOFAI). The material in this chapter is optional for modern AI developers but I recently wrote the Python examples listed below when I was thinking of how different knowledge representation is today compared to 40 years ago. Except for the material using Python + Swi-Prolog, and Python + the MiniZinc constraint satisfaction system there is nothing in this chapter that I would consider using today for work but you might enjoy the examples anyway. After this chapter we will bear down on deep learning, information organization using RDF and property graph data stores.

I do not implement three examples in this chapter in “pure Python,” rather, I use the Python bindings for three well known tools that are implemented in C/C++:

\nSwi-Prolog is a Prolog system that has many available libraries for a wide variety of tasks.\n
\nSoar Cognitive Architecture is a flexible and general purpose reasoning and knowledge management system for building intelligent software agents.\n
\nMiniZinc is a powerful Constraint Satisfaction System.\n

The material in this chapter is optional for the modern AI practitioner but I hope you find it interesting.

The examples for this chapter are in the directory source-code/symbolic-AI.

Comparison of Symbolic AI and Deep Learning

Symbolic AI, also known as “good old-fashioned AI” (GOFAI), is a form of artificial intelligence that uses symbolic representations and logical reasoning to solve problems. It is based on the idea that a computer can be programmed to manipulate symbols in the same way that humans manipulate symbols in their minds: an example of this is the process of logical reasoning.

Symbolic AI systems can consist of sets of rules, facts, and procedures that are used to represent knowledge and a reasoning engine that uses these symbolic representations to make inferences and decisions. Some examples of symbolic AI systems include expert systems, other types of rule-based systems, and decision trees. These systems are typically based on a set of predefined rules and the performance of the system is based on the knowledge manually encoded (or it can be learned) in these rules. Symbolic AI is largely non-numerical.

In comparison deep learning, which uses numerical representations (high dimensional matrices, or tensors, containing floating point data), is a subset of machine learning that uses neural networks with many multiple layers (the term “deep” comes from the idea of having dozens or even hundreds of layers, which differs from early neural network models comprised of just a few layers) to learn from data and make predictions or decisions. The basic building blocks of both simple neural models and deep learning models are artificial neurona, which are a simple mathematical function that receives input, applies a transformation, and produces an output. Neural networks can learn to perform a wide variety of tasks, such as image recognition, natural language processing, and game playing, by adjusting the weights of the neurons.

The key difference is that, while Symbolic AI relies on hand-coded rules and logical reasoning, deep learning relies on learning from data. Symbolic AI systems typically have a high level of interpretability and transparency, as the rules and knowledge are explicitly encoded and can be inspected by humans, while deep learning models are often considered “black boxes” due to their complexity and the difficulty of understanding how they arrived at a decision.

So, Symbolic AI uses symbolic representations and logical reasoning while deep learning uses neural networks to learn from data. The first one is more interpretable but less flexible, while the second one is more flexible, much more powerful for most applications, but is less interpretable.

We will start with one “pure Python” example in the next section.

Implementing Frame Data Structures in Python

Most of my learning experiments and AI projects in the early 1980s were built from scratch in Common Lisp and nested frame data structures were a common building block. Here we allow three types of data to be stored in frames:

\nNumbers\n
\nStrings\n
\nOther frames\n

We write a general Python class Frame that supports creating frames and converting a frame, including deeply nested frames, into a string representation. We also write a simple Python class BookShelf as a container for frames that supports searching for any frames containing a string value.

 1 # Implement Lisp-like frames in Python\n 2 \n 3 class Frame():\n 4     frame_counter = 0\n 5     def __init__(self, name = ''):\n 6         Frame.frame_counter += 1\n 7         self.objects = []\n 8         self.depth = 0\n 9         if (len(name)) == 0:\n10             self.name = f\"Frame:{Frame.frame_counter}\"\n11         else:\n12             self.name = f'\"{name}\"'\n13 \n14     def add_subframe(self, a_frame):\n15         a_frame.depth = self.depth + 1\n16         self.objects.append(a_frame)\n17 \n18     def add_number(self, a_number):\n19         self.objects.append(a_number)\n20 \n21     def add_string(self, a_string):\n22         self.objects.append(a_string)\n23 \n24     def __str__(self):\n25         indent = \" \" * self.depth * 2\n26         ret = indent + f\"<Frame {self.name}>\\n\"\n27         for frm in self.objects:\n28             if isinstance(frm, (int, float)):\n29                 ret = ret + indent + '  ' + f\"<Number {frm}>\\n\"\n30             if isinstance(frm, str):\n31                 ret = ret + indent + '  ' + f'<String \"{frm}\">\\n'\n32             if isinstance(frm, Frame):\n33                 ret = ret + frm.__str__()\n34         return ret\n35 \n36 f1 = Frame()\n37 f2 = Frame(\"a sub-frame\")\n38 f1.add_subframe(f2)\n39 f1.add_number(3.14)\n40 f2.add_string(\"a string\")\n41 print(f1)\n42 f2.add_subframe(Frame('a sub-sub-frame'))\n43 print(f1)\n44 \n45 class BookShelf():\n46 \n47     def __init__(self, name = ''):\n48         self.frames = []\n49     \n50     def add_frame(self, a_frame):\n51         self.frames.append(a_frame)\n52     \n53     def search_text(self, search_string):\n54         ret = []\n55         for frm in self.frames:\n56             if frm.__str__().index(search_string):\n57                 ret.append(frm)\n58         return ret\n59     \n60 bookshelf = BookShelf()\n61 bookshelf.add_frame(f1)\n62 search_results = bookshelf.search_text('sub')\n63 print(\"Search results: all frames containing 'sub':\")\n64 for rs in search_results:\n65     print(rs)\n

\n\n

The implementation of the class Frame is straightforward. The init method init defines a list of contained objects (or frames), sets the default frame nesting depth to zero, and assigns a readable name. There are three separate methods to add subframes, numbers, and strings.

The method str ensures that when we print a frame that the output is human readable and visualizes frame nesting.

Here is some output:

 1 $ python\n 2 >>> from frame import Frame Bookshelf\n 3 >>> f1 = Frame()\n 4 >>> f2 = Frame(\"a sub-frame\")\n 5 >>> f1.add_subframe(f2)\n 6 >>> f1.add_number(3.14)\n 7 >>> f2.add_subframe(Frame('a sub-sub-frame'))\n 8 >>> print(f1)\n 9 <Frame Frame:4>\n10   <Frame \"a sub-frame\">\n11     <Frame \"a sub-sub-frame\">\n12   <Number 3.14>\n13 >>> bookshelf = BookShelf()\n14 >>> bookshelf.add_frame(f1)\n15 >>> search_results = bookshelf.search_text('sub')\n16 >>> for rs in search_results:\n17 ...   print(rs)\n18 ... \n19 <Frame Frame:4>\n20   <Frame \"a sub-frame\">\n21     <Frame \"a sub-sub-frame\">\n22   <Number 3.14>\n

\n\n

I my early AI experiments in the 1980s I would start with implementing a simple frame library and extend it for the two types of applications that I worked on: Natural Language Processing (NLP) and planning systems.

I no longer use frames, preferring the use of off the shelf graph databases that we will cover in a later chapter. Graphs can represent a wider range of data representations because frames represent tree structured data and graphs are more general purpose than trees.

Use Predicate Logic by Calling Swi-Prolog

Please skip this section if you either don’t know how to program in Prolog or if you have no interest in learning Prolog. I have a writing project for a book titled Prolog for AI applications that is a work in progress. When that book is released I will add a link here. Before my Prolog book is released, please use Sheila McIlraith’s Swi-Prolog tutorial. It was written for her students and is a good starting point. You can also use the official Swi-Prolog manual for specific information. I will make this section self-contained if you just want to read the material without writing your own Python + Prolog applications.

You can start by reading the documentation for setting up Swi-Prolog so it can be called from Python.

I own many books on Prolog but my favorite that I recommend is “The Art Of Prolog” by Leon S. Sterling and Ehud Y. Shapiro. Here are a few benefits to using Prolog:

\nProlog is a natural fit for symbolic reasoning: Prolog is a logic programming language, which means that it is particularly well-suited for tasks that involve symbolic reasoning and manipulation of symbols. This makes it an excellent choice for tasks such as natural language processing, knowledge representation, and expert systems.\n
\nProlog has built-in support for logic and rule-based systems which makes it easy to represent knowledge and perform inferences. The syntax of Prolog is based on first-order logic, which means that it is similar to the way humans think and reason.\n
\nProlog provides a high-level and expressive syntax, which makes it relatively easy to write and understand code. The declarative nature of Prolog also makes it more readable and maintainable than some other languages.\n
\nProlog provides automatic backtracking and search, which makes it easy to implement algorithms that require searching through large spaces of possible solutions.\n
\nProlog has a set of built-in predicates and libraries that support natural language processing, which made it an ideal choice for tasks such as natural language understanding and generation. Now deep learning techniques are the more effective technology or NLP.\n
\nProlog has a large community of users and developers which means that there are many open source libraries and tools available for use in Prolog projects.\n
\nProlog can be easily integrated with other programming languages, such as Python, C, C++, Common Lisp, and Java. This integration with other languages makes Prolog a good choice for projects that require a combination of different languages.\n
\nProlog code is concise, which means less lines of code to write. Also, some programmers find that the declarative nature of the language makes it less prone to errors.\n

Using Swi-Prolog for the Semantic Web, Fetching Web Pages, and Handling JSON

In several ways, reading the historic Scientific American Article from 2001 The Semantic Web - A new form of Web content that is meaningful to computers will unleash a revolution of new possibilities by Tim Berners-Lee, James Hendler, and Ora Lassila changed my life. I spent a lot of time experimenting with the Swi-Prolog semweb library that I found to be the easiest way to experiment with RDF. We will cover the open source Apache Jena/Fuseki RDF data store and query engine in a later chapter. The Common Lisp and Semantic Web tools company Franz Inc. very kindly subsidized my work writing two Semantic Web books that cover Common Lisp, Java, Clojure, and Scala (available as downloadable PDFs on my web site). Writing these books lead directly to being invited to work at Google for a project using their Knowledge Graph.

Swi-Prolog can be installed from https://www.swi-prolog.org/download/stable. On macOS you can just use:

1 brew install swi-prolog\n

\n\n

Here I mention how to load the semweb library and refer you to the library documentation:

1 ?- use_module(library(semweb/rdf_db)).\n2 ?- use_module(library(semweb/sparql_client)).\n3 ?- sparql_query('select * where { ?s ?p \"Amsterdam\" }',\n4                 Row,\n5                 host('dbpedia.org'), path('/sparql/')]).\n

\n\n

The output is:

1 Row = row('http://dbpedia.org/resource/Anabaptist_riot',\n2           'http://dbpedia.org/ontology/combatant') ;)\n3 Row = row('http://dbpedia.org/resource/Thirteen_Years_War_(1454-1466)',\n4           'http://dbpedia.org/ontology/combatant') ;)\n5 Row = row('http://dbpedia.org/resource/Womens_Euro_Hockey_League',\n6           'http://dbpedia.org/ontology/participant') ;)\n

\n\n

Prolog is a flexible and general purpose language that is used to write compilers, handle text processing, etc. One common thing that I need no matter what programming language I use is fetching content from the web. Here is a simple example you can try to perform a HTTP GET operation and echo the fetched data to standard output:

1 use_module(library(http/http_open)).\n2 http_open('https://markwatson.com', In, []),\n3    copy_stream_data(In, user_output),\n4    close(In).\n

\n\n

Similarly, handling JSON data is a common task so here is an example for doing that:

1 ?- use_module(library(http/json)). % to enable json_read_dict/2\n2 ?- FPath = 'test.json', open(FPath, read, Stream), json_read_dict(Stream, Dicty).\n

\n\n

Python and Swi-Prolog Interop

You need to install the Python bridge library that is supported by the Swi-Prolog developers:

1  uv pip install swiplserver\n

\n\n

I copied a Prolog program from the Swi-Prolog documentation to calculate how to eight queens on a chess board in such a way that no queen can capture another queen. Here I get three results by entering the semicolon key to get another answer or the period key to stop:

 1 $ swipl\n 2 \n 3 ?- use_module(library(clpfd)). /* constraint satisfaction library */\n 4 true.\n 5 \n 6 ?- [n_queens]. /* load the file n_queens.pl */\n 7 true.\n 8 \n 9 ?- n_queens(8, Qs), label(Qs).\n10 Qs = [1, 5, 8, 6, 3, 7, 2, 4] ;\n11 Qs = [1, 6, 8, 3, 7, 4, 2, 5] ;\n12 Qs = [1, 7, 4, 6, 8, 2, 5, 3] .\n

\n\n

The source code to n_queens.pl is included in the examples directory swi-prolog for this book. This example was copied from the Swi-Prolog documentation. Here is Python code to use this Prolog example:

 1 from swiplserver import PrologMQI\n 2 from pprint import pprint\n 3 \n 4 with PrologMQI() as mqi:\n 5     with mqi.create_thread() as prolog_thread:\n 6         prolog_thread.query(\"use_module(library(clpfd)).\")\n 7         prolog_thread.query(\"[n_queens].\")\n 8         result = prolog_thread.query(\"n_queens(8, Qs), label(Qs).\")\n 9         pprint(result)\n10         print(len(result))\n

\n\n

We can run this example to see all 92 possible answers:

1  $ p n_queens.py\n2 [{'Qs': [1, 5, 8, 6, 3, 7, 2, 4]},\n3  {'Qs': [1, 6, 8, 3, 7, 4, 2, 5]},\n4  {'Qs': [1, 7, 4, 6, 8, 2, 5, 3]},\n5  {'Qs': [1, 7, 5, 8, 2, 4, 6, 3]},\n6  {'Qs': [2, 4, 6, 8, 3, 1, 7, 5]},\n7  ...\n8  ]\n9  92\n

\n\n

Here I call the Swi-Prolog system synchronously; that is, each call to prolog_thread.query waits until the answers are ready. If you can also run long running queries asynchronously then please read the instructions online.

In the last example we simply ran an existing Prolog program from Python. Now let’s look at an example for asserting facts and Prolog rules from a Python script. First we look at a simple example of Prolog rules, asserting facts, and applying rules to facts. We will use the Prolog source file family.pl:

1 parent(X, Y) :- mother(X, Y).\n2 parent(X, Y) :- father(X, Y).\n3 grandparent(X, Z) :-\n4   parent(X, Y),\n5   parent(Y, Z).\n

\n\n

Before using a Python script, let’s run an example in the Swi-Prolog REPL (in line 2, running [family]. loads the Prolog source file family.pl):

 1 $ swipl\n 2 ?- [family].\n 3 true.\n 4 \n 5 ?- assertz(mother(irene, ken)).\n 6 true.\n 7 \n 8 ?- assertz(father(ken, ron)).\n 9 true.\n10 \n11 ?- grandparent(A,B).\n12 A = irene,\n13 B = ron ;\n14 false.\n

\n\n

When running Prolog queries you may get zero, one, or many results. The results print one at a time. Typing the semicolon character after a result is printed requests that the next result be printed. Typing a period after a result is printed lets the Prolog system know that you don’t want to see any more of the available results. When I entered a semicolon character in line 13 after the first result is printed, the Prolog system prints false because no further results are available.

Now we can write a Python script family.py that loads the Prolog rules file family.pl, asserts facts, run Prolog queries, and get the results back to the Python script:

 1 from swiplserver import PrologMQI\n 2 from pprint import pprint\n 3 \n 4 with PrologMQI() as mqi:\n 5     with mqi.create_thread() as prolog_thread:\n 6         prolog_thread.query(\"[family].\")\n 7         print(\"Assert a few initial facts:\")\n 8         prolog_thread.query(\"assertz(mother(irene, ken)).\")\n 9         prolog_thread.query(\"assertz(father(ken, ron)).\")\n10         result = prolog_thread.query(\"grandparent(A, B).\")\n11         pprint(result)\n12         print(len(result))\n13         print(\"Assert another test fact:\")\n14         prolog_thread.query(\"assertz(father(ken, sam)).\")\n15         result = prolog_thread.query(\"grandparent(A, B).\")\n16         pprint(result)\n17         print(len(result))\n

\n\n

The output looks like:

1 $ python family.py\n2 Assert a few initial facts:\n3 [{'A': 'irene', 'B': 'ron'}]\n4 1\n5 Assert another test fact:\n6 [{'A': 'irene', 'B': 'ron'}, {'A': 'irene', 'B': 'sam'}]\n7 2\n

\n\n

Swi-Prolog is still under active development (the project was started in 1985) and used for new projects. If the declarative nature of Prolog programming appeals to you then I urge you to take the time to integrate Swi-Prolog into one of your Python-based projects.

Swi-Prolog and Python Deep Learning Interop

Good use cases for Python and Prolog applications involve using Python code to fetch and process data that is imported to Prolog. Applications can then use Prolog’s reasoning and other capabilities.

Here we look at a simple example that:

\nUses the Firebase Hacker News APIs to fetch most recent stories.\n
\nUses the spaCy library deep learning based NLP model to identify organization and peoples names from articles.\n
\nAssert as Prolog facts terms like organization(Name, URI)*.\n

We will cover the spaCy library in depth later. For the purposes of this example, please consider spaCy as a “black box.”

The following listing shows the Python script hackernews.py:

 1 from urllib.request import Request, urlopen\n 2 import json\n 3 from bs4 import BeautifulSoup\n 4 from pprint import pprint\n 5 \n 6 from swiplserver import PrologMQI\n 7 \n 8 import spacy\n 9 try:\n10     spacy_model = spacy.load(\"en_core_web_sm\")\n11 except:\n12   from os import system\n13   system(\"python -m spacy download en_core_web_sm\")\n14   spacy_model = spacy.load('en_core_web_sm')\n15 \n16 LEN = 100 # larger amount of text is more expensive for OpenAI APIs\n17 \n18 def get_new_stories(anAgent={'User-Agent': 'PythonAiBook/1.0'}):\n19     req = Request(\"https://hacker-news.firebaseio.com/v0/newstories.json\",\n20                   headers=anAgent)\n21     httpResponse = urlopen(req)\n22     data = httpResponse.read()\n23     #print(data)\n24     ids = json.loads(data)\n25     #print(ids)\n26     # just return the most recent 3 stories:\n27     return ids[0:3]\n28 \n29 ids = get_new_stories()\n30 \n31 def get_story_data(id, anAgent={'User-Agent': 'PythonAiBook/1.0'}):\n32     req = Request(f\"https://hacker-news.firebaseio.com/v0/item/{id}.json\",\n33                   headers=anAgent)\n34     httpResponse = urlopen(req)\n35     return json.loads(httpResponse.read())\n36 \n37 def get_story_text(a_uri, anAgent={'User-Agent': 'PythonAiBook/1.0'}):\n38     req = Request(a_uri, headers=anAgent)\n39     httpResponse = urlopen(req)\n40     soup = BeautifulSoup(httpResponse.read(), \"html.parser\")\n41     return soup.get_text()\n42 \n43 def find_entities_in_text(some_text):\n44     def clean(s):\n45         return s.replace('\\n', ' ').strip()\n46     doc = spacy_model(some_text)\n47     return map(list, [[clean(entity.text), entity.label_] for entity in doc.ents])\n48 \n49 import re\n50 regex = re.compile('[^a-z_]')\n51 \n52 def safe_prolog_text(s):\n53     s = s.lower().replace(' ','_').replace('&','and').replace('-','_')\n54     return regex.sub('', s)\n55 \n56 for id in ids:\n57     story_json_data = get_story_data(id)\n58     #pprint(story_json_data)\n59     if story_json_data != None and 'url' in story_json_data:\n60         print(f\"Processing {story_json_data['url']}\\n\")\n61         story_text = get_story_text(story_json_data['url'])\n62         entities = list(find_entities_in_text(story_text))\n63         #print(entities)\n64         organizations =\n65             set([safe_prolog_text(name)\n66                  for [name, entity_type] in entities if entity_type == \"ORG\"])\n67         people =\n68             set([safe_prolog_text(name)\n69                  for [name, entity_type] in entities if entity_type == \"PERSON\"])\n70         with PrologMQI() as mqi:\n71             with mqi.create_thread() as prolog_thread:\n72                 for person in people:\n73                     s = f\"assertz(person({person},\n74                                   '{story_json_data['url']}')).\"\n75                     #print(s)\n76                     try:\n77                         prolog_thread.query(s)\n78                     except:\n79                         print(f\"Error with term: {s}\")\n80                 for organization in organizations:\n81                     s = f\"assertz(organization({organization}, '{story_json_data['url']}')).\"\n82                     #print(s)\n83                     try:\n84                         prolog_thread.query(s)\n85                     except:\n86                         print(f\"Error with term: {s}\")\n87                 try:\n88                     result = prolog_thread.query(\"organization(Organization, URI).\")\n89                     pprint(result)\n90                 except:\n91                     print(\"No results for organizations.\")\n92                 try:\n93                     result = prolog_thread.query(\"person(Person, URI).\")\n94                     pprint(result)\n95                 except:\n96                     print(\"No results for people.\")\n

\n\n

Here is some example output:

 1 {'Organization': 'bath_and_beyond_inc',\n 2   'URI': 'https://bedbathandbeyond.gcs-web.com/news-releases/news-release-details/bed-bath-beyond-inc-provides-business-update'},\n 3 {'Person': 'ryan_holiday',\n 4   'URI': 'https://www.parttimetech.io/p/what-do-you-really-want'},\n 5 {'Organization': 'nasa_satellite',\n 6  {'Organization': 'nih',\n 7   'URI': 'https://nap.nationalacademies.org/catalog/26424/measuring-sex-gender-identity-and-sexual-orientation'},\n 8   'URI': 'https://landsat.gsfc.nasa.gov/article/now-then-landsat-u-s-mosaics/'},\n 9 {'Organization': 'the_national_academies',\n10 'URI': 'https://nap.nationalacademies.org/catalog/26424/measuring-sex-gender-identity-and-sexual-orientation'},\n11 {'Organization': 'microsoft',\n12   'URI': 'https://invention.si.edu/susan-kare-iconic-designer'},\n13 {'Person': 'susan_kare',\n14   'URI': 'https://invention.si.edu/susan-kare-iconic-designer'},\n

\n\n

Later we will use deep learning models to summarize text and other NLP tasks. This example could be extended for defining Prolog terms in the form summary(URI, “…”). Other application ideas might be to use Python scripts for:

\nCollect stock market data for a rule-based Prolog reasoner for stock purchase selections.\n
\nA customer service chatbot that is mostly written in Python could be extended by using a Prolog based reasoning system.\n

Soar Cognitive Architecture

Soar is a flexible and general purpose reasoning and knowledge management system for building intelligent software agents. The Soar project is a classic AI tool and has the advantage of being kept up to date. As I write this the Soar GitHub repository was just updated a few days ago. Here is a bit of history:

Soar is a cognitive architecture that was originally developed at Carnegie Mellon University. It is a rule-based system that simulates the problem-solving abilities of human cognition. The architecture is composed of several interacting components, including a production system (a rule-based system), episodic memory (remembers past experiences), and working memory (for current state of the world).

The production system is the core component of Soar and is responsible for making decisions and performing actions. It is based on the idea of a production rule, which is a rule that describes a condition and an action to be taken when that condition is met. Production systems use rules to make decisions and to perform actions based on the current state of the system.

The episodic memory is a component that stores information about past events and experiences. It allows a system built with Soar to remember past events and use that information to make decisions affecting the future state of the world. The working memory is a component that stores information about the current state of the system and is used by the production system to make decisions.

Soar also includes several other components, such as an explanation facility, which allows the system to explain its decisions and actions, and a learning facility, which allows the system to learn from its experiences.

Soar is a general-purpose architecture that can be applied to a wide range of tasks and domains. In the past it has been particularly well-suited for tasks that involve planning, problem-solving, and decision-making. It has been used in a variety of applications, including robotics, NLP, and intelligent tutoring systems.

I am writing this material many years after my original use of the Soar project. My primary reference for preparing the following material is the short paper Introduction to the Soar Cognitive Architecture by John E. Laird. For self-study you can start at the Soar Tutorial Page that provides a download for an eight part tutorial in separate PDF files, binary Soar executables for Linux, Windows, and macOS, and all of the code for the tutorials.

I consider Soar to be important because it proposes and implements a general purpose cognitive architecture. A warning to the reader: Soar has a steep learning curve and there are simpler frameworks for solving practical problems. Later we will look at an example from the Soar Tutorial for the classic “blocks world” problem of moving blocks on a table subject to constraints like not being allowed to move a block if it has another object on top of it. Solving this fairly simple problem requires about 400 lines of Soar source code.

Background Theory

The design goals for the Soar Cognitive Architecture (which I will usually refer to as Soar) is to provide “fixed structures, mechanisms, and representations” to develop human level behavior across a wide range of tasks. There is a commercial company Soar.com that uses Soar for commercial and government projects.

We will cover Reinforcement Learning (which I will usually refer to as RL) in a later chapter but there is similar infrastructure supported by both Soar and RL: a simulated environment, data representing the state of the environment, and possible actions that can be performed in the environment that change the state.

As we have disused, there are two main types of memory of the Soar architecture:

\nWorking Memory - this is the data that specifies the current state of the environment. Actions in the environment change the data in working memory, either by modification, addition, or deletion. At the risk of over-anthropomorphism, consider this like human short term memory.\n
\nProduction Memory - this data is a form of production rules where the left-hand side of rules consist of patterns that if matched against working memory, the the actions on the right-hand side of a rule are executed. Consider these production rules as long-term memory.\n

Both Soar working memory and production memory are symbolic data. As a contrast, data in deep learning is numeric, mostly tensors. This symbolic data comprises goals (G), problem spaces (PS), states (S) and operators (O).

Soar Operator transitioning from one state to another (figure is from the Soar Tutorial):

Setup Python and Soar Development Environment

The Soar Python bindings are available as a pip package, making installation straightforward:

1 uv pip install soar-sml\n

\n\n

This installs prebuilt bindings for the Soar kernel — no need to clone the repository or compile from source.

I will present here a simple example and explain a subset of the capabilities of Soar. When we are done here you can reference a recent paper by Neha Rajan and 2Sunderrajan Srinivasan Exploring Learning Capability of an Agent in SOAR: Using 8- Queens Problem for a complete example using Soar for cognitive modeling and a more complex example.

A minimal Soar Tutorial

I am presenting a minimal introduction to Soar and we will later provide an example of Python and Soar interop for the purpose of introducing you to Soar. If this material looks interesting then I encourage you to work through the Soar Tutorial Page.

Soar supports a rule language that uses the highly efficient Rete algorithm (optimized for huge numbers of rules, less optimized for large working memories). Let’s look at a sample rule from Chapter 1 (first PDF file) of the Soar tutorial:

1 sp {hello-world\n2    (state <s> ^type state)\n3 -->\n4    (write |Hello World|)\n5    (halt)\n6 }\n

\n\n

The token sp on line 1 stands for Soar Production. Rules are enclosed in { and }. The name of this rule is the symbol hello-world. In the tutorial you will usually see rule names partitioned using the characters * and -. Rules have a “left side” and a “right side”, separated by —>. If all of the left side patterns match working memory elements then the right-hand side actions are executed.

The following figure is from the Soar tutorial and shows two blocks stacked on top of each other. The bottom block rests on a table:

This figure represents state s1 that is a root of the graph also containing blocks named b1 and b2 as well as the table named t1. The blocks and table all have attributes ^color, ^name, and ^type. The blocks also have the optional attribute ^ontop.

Rule right-hand side actions can modify, delete, or add working memory data. For example, a left-hand side matching the attribute values for block b1 could modify its ^ontop attribute from the value b2 to the table named t1.

Example Soar System With Python Interop

We will use the blocks world example from the official Soar Tutorial. The agent starts with three blocks (A, B, C) on the table and moves them randomly until they are stacked A on B on C. The Soar production rules are in the file bw.soar in this book’s GitHub repository.

 1 import soar_sml as sml\n 2 \n 3 def callback_debug(mid, user_data, agent, message):\n 4     print(message)\n 5 \n 6 if __name__ == \"__main__\":\n 7     soar_kernel = sml.Kernel.CreateKernelInCurrentThread()\n 8     soar_agent = soar_kernel.CreateAgent(\"agent\")\n 9     soar_agent.RegisterForPrintEvent(sml.smlEVENT_PRINT, callback_debug, None) # no user data\n10     soar_agent.ExecuteCommandLine(\"source bw.soar\")\n11     run_result=soar_agent.RunSelf(50)\n12     soar_kernel.DestroyAgent(soar_agent)\n13     soar_kernel.Shutdown()\n

\n\n

Run this example:

 1 $ uv run bw.py\n 2 \n 3 Initial state has A, B, and C on the table.\n 4 Moving Block: A to: B\n 5 Moving Block: C to: A\n 6 Moving Block: C to: TABLE\n 7 Moving Block: A to: C\n 8 Moving Block: B to: A\n 9 Moving Block: A to: TABLE\n10 Moving Block: B to: C\n11 Moving Block: A to: B\n12 Achieved A, B, C\n

\n\n

Because the blocks are moved at random, the exact sequence of moves will vary between runs, but the agent always reaches the goal state (A on B on C on TABLE) and halts.

I consider Soar to be of historic interest and is an important example of a large multiple-decade research project in building large scale reasoning systems.

Constraint Programming with MiniZinc and Python

As with Soar, our excursion into constraint programming will be brief, hopefully enough to introduce you to a new style of programming though a few examples. I still use constraint programming and hope you might find the material in this section useful.

While I attempt to make this material self-contained reading, you may want to use the MiniZinc Python documentation as a reference for the Python interface and The MiniZinc Handbook as a reference to the MiniZinc language and its use for your own projects.

Constraint Programming (CP) is a paradigm of problem-solving that involves specifying the constraints of a problem, and then searching for solutions that satisfy those constraints. MiniZinc is a high-level modeling language for constraint programming that allows users to specify constraints and objectives in a compact and expressive way.

In MiniZinc a model is defined by a set of variables and a set of constraints that restrict the possible values of those variables. The variables can be of different types, such as integers, booleans, and sets, and the constraints can be specified using a variety of built-in predicates and operators. The model can also include an objective function that the solver tries to optimize.

Once a model is defined in MiniZinc, it can be solved using a constraint solver which is a software program that takes the model as input and returns solutions that satisfy the constraints. MiniZinc supports several constraint solvers, including Gecode, Chuffed, and OR-Tools, each of which has its own strengths and weaknesses.

MiniZinc provides a feature called “Annotations”, which allows the user to specify additional information to the solver, as the way to search for solutions, or setting the maximum time for the solver to run.

Installation and Setup for MiniZinc and Python

You need to first install the MiniZinc system. For macOS this can be done with brew install minizinc or can be installed from source code on macOs and Linux. The Python interface can be installed with uv pip install minizinc.

The following figure shows the MiniZincIDE with simple constraint satisfaction problem:

When I installed minizinc on macOS with brew, the solver coinbc was installed automatically so that is what we use here. Here is the MiniZinc source file test_mzn.mzn that you also see in the last figure:

1 int: n;\n2 int: m;\n3 var 1..n: x;\n4 var 1..n: y;\n5 constraint x+y = n;\n6 constraint x*y = m;\n

\n\n

There are several possible solvers to use with MiniZinc. When I install on macOS using brew the solver “coinbc” is available. When I install sudo apt install minizinc on Ubuntu Linux, the solver “gecode” is available.

Notice that we don’t set values for the constants n and m as we did when using MiniZincIDE. We instead set them in Python code before calling the solver in lines 7 and 8:

 1 from minizinc import Instance, Model, Solver\n 2 \n 3 coinbc = Solver.lookup(\"coinbc\")\n 4 \n 5 test1 = Model(\"./test_mzn.mzn\")\n 6 instance = Instance(coinbc, test1)\n 7 instance[\"n\"] = 30\n 8 instance[\"m\"] = 200\n 9 \n10 result = instance.solve()\n11 print(result)\n12 print(result[\"x\"])\n13 print(result[\"y\"])\n

\n\n

The result is:

1 $ python test_mzn.py\n2 Solution(x=20, y=10, _checker='')\n3 20\n4 10\n

\n\n

Let’s look at a more complex example: on the map of the USA, the states neighboring each other are colored differently than their adjoining states in such a way that no states with touching edges are the same color. We use integers to represent colors and the mapping of numbers to colors is unimportant. Here is a partial listing of us_states.mzn:

 1 int: nc = 3; %% needs to be 4 to solve this problem\n 2 \n 3 var 1..nc: alabama;\n 4 var 1..nc: alaska;\n 5 var 1..nc: arizona;\n 6 var 1..nc: arkansas;\n 7 var 1..nc: california;\n 8  ...\n 9 constraint alabama != florida;\n10 constraint alabama != georgia;\n11 constraint alabama != mississippi;\n12 constraint alabama != tennessee;\n13 constraint arizona != california;\n14 constraint arizona != colorado;\n15 constraint arizona != nevada;\n16 constraint arizona != new_mexico;\n17 constraint arizona != utah;\n18  ...\n19 solve satisfy;\n

\n\n

In line 9, as an example, the statement constraint alabama != florida; means that since Alabama and Florida share a border, they must have different color indices. Initially we set the number of allowed color to 3 on line 1 and with just three colors allowed this problem is unsolvable.

The output is:

1  $ minizinc --solver coinbc us_states.mzn\n2 =====UNSATISFIABLE=====\n

\n\n

So we need more than three colors. Let’s try int: nc = 4;:

 1 $ minizinc --solver coinbc us_states.mzn\n 2 alabama = 2;\n 3 alaska = 1;\n 4 arizona = 3;\n 5 arkansas = 4;\n 6 california = 4;\n 7 colorado = 4;\n 8 connecticut = 2;\n 9 delaware = 4;\n10  ...\n

\n\n

Here is a Python script us_states.py that uses this model and picks out the assigned color indices from the solution object:

 1 from minizinc import Instance, Model, Solver\n 2 \n 3 coinbc = Solver.lookup(\"coinbc\")\n 4 \n 5 model = Model(\"./us_states.mzn\")\n 6 instance = Instance(coinbc, model)\n 7 instance[\"nc\"] = 4 # solve for a maximum of 4 colors\n 8 \n 9 result = instance.solve()\n10 print(result)\n11 all_states = list(result.__dict__['solution'].__dict__.keys())\n12 all_states.remove('_checker')\n13 print(all_states)\n14 for state in all_states:\n15     print(f\" {state} \\t: \\t{result[state]}\")\n

\n\n

Here is some of the output:

1 $ python us_states.py\n2 Solution(alabama=2, alaska=1, arizona=3, arkansas=4, california=4, colorado=4, connecticut=2, delaware=4, florida=1, georgia=4, hawaii=1, idaho=4, ... ]\n3  alabama    :   2\n4  alaska     :   1\n5  arizona    :   3\n6  arkansas   :   4\n7  ...\n8  wisconsin  :   1\n9  wyoming    :   3\n

\n\n

Good Old Fashioned Symbolic AI Wrap-up

As a practical matter almost all of my work in the last ten years used either deep learning or was comprised of a combination of semantic web and linked data with deep learning projects. While the material in this chapter is optional for the modern AI practitioner, I still find using MiniZinc for constraint programming and Prolog to be useful. I included the material for the Soar cognitive architecture because I both find it interesting and I believe the any future development of “real AI” (or AGI) will involve hybrid approaches and there are many good ideas in the Soar implementation.

Part 5 - Knowledge Representation

In the next two chapters we will look at a variety of techniques for encoding information and knowledge for automated processing by AI systems. We dive into using graph data stores and relational databases. We finish this part of the book by reviewing the theory behind the Semantic Web and Linked Data and look at some practical applications.

We touched on symbolic knowledge representation in a previous chapter. Here and in the next two chapters we cover organizing and using “knowledge as data” using graph and relational data stores. Generally in software development we start any project by evaluating sources of data, format and schemas, and what operations we need to accomplish.

Getting Setup To Use Graph and Relational Databases

I use several types of data stores in my work but for the purposes of this book we can explore knowledge representation using two key platforms:

\nRDF data via Python’s rdflib library and public SPARQL endpoints like Wikidata and DBPedia for graph-based knowledge representation.\n
\nSQLite for relational knowledge representation using the SQL query language.\n

The next chapter covers RDF and the SPARQL query language in more detail.

The examples for this chapter are in the directory source-code/knowledge_representation.

In technical terms, knowledge representation using graph and relational databases involves the use of graph structures and relational data models to represent and organize knowledge in a structured, computationally efficient, and easily accessible way.

A graph structure is a collection of nodes (also known as vertices) and edges (also known as arcs) that connect the nodes. Each node and edge in a graph can have properties, such as labels and attributes which provide information about the entities they represent. Graphs can be used to represent knowledge in a variety of ways, such as through semantic networks and using ontologies to define terms, classes, types, etc.

Relational databases, on the other hand, use a tabular data model to represent knowledge. The basic building block of a relational database is the table, which is a collection of rows (also known as tuples) and columns (also known as attributes). Each row represents an instance of an entity, and the columns provide information about the properties of that entity. Relationships between entities can also be represented by foreign keys, which link one table to another.

Combining these two technologies, knowledge can be represented as a graph of interconnected entities, where each entity is stored in a relational database table and connected to other entities through relationships represented by edges in the graph. This allows for efficient querying and manipulation of knowledge, as well as the ability to integrate and reason over large amounts of information.

Querying Wikidata with SPARQL and Python

Wikidata is a free, open knowledge base maintained by the Wikimedia Foundation. It contains structured data about millions of entities — people, places, organizations, scientific concepts, and more — all accessible through a public SPARQL endpoint. Unlike DBPedia, which extracts structured data from Wikipedia infoboxes, Wikidata is a curated knowledge base where the data is entered and maintained directly.

The Python SPARQLWrapper library makes it straightforward to query any SPARQL endpoint, including Wikidata:

1 uv pip install sparqlwrapper\n

\n\n

Finding Information About a Person

Let’s query Wikidata for information about a specific person. Wikidata uses numeric entity identifiers (like Q937 for Albert Einstein) and property identifiers (like P19 for “place of birth”):

 1 # wikidata_person.py - Query Wikidata for information about a person\n 2 \n 3 from SPARQLWrapper import SPARQLWrapper, JSON\n 4 from pprint import pprint\n 5 \n 6 sparql = SPARQLWrapper(\"https://query.wikidata.org/sparql\")\n 7 sparql.addCustomHttpHeader(\"User-Agent\", \"PythonAIBook/1.0\")\n 8 \n 9 queryString = \"\"\"\n10 SELECT ?personLabel ?birthPlaceLabel ?birthDate ?occupationLabel\n11 WHERE {\n12     ?person wdt:P31 wd:Q5 .            # instance of human\n13     ?person rdfs:label \"Albert Einstein\"@en .\n14     OPTIONAL { ?person wdt:P19 ?birthPlace . }\n15     OPTIONAL { ?person wdt:P569 ?birthDate . }\n16     OPTIONAL { ?person wdt:P106 ?occupation . }\n17     SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\" . }\n18 }\n19 LIMIT 10\n20 \"\"\"\n21 \n22 sparql.setQuery(queryString)\n23 sparql.setReturnFormat(JSON)\n24 results = sparql.queryAndConvert()\n25 \n26 for r in results[\"results\"][\"bindings\"]:\n27     print(f\"  Name: {r['personLabel']['value']}\")\n28     if 'birthPlaceLabel' in r:\n29         print(f\"  Born: {r['birthPlaceLabel']['value']}\")\n30     if 'birthDate' in r:\n31         print(f\"  Date: {r['birthDate']['value'][:10]}\")\n32     if 'occupationLabel' in r:\n33         print(f\"  Occupation: {r['occupationLabel']['value']}\")\n34     print()\n

\n\n

The output shows Wikidata returning multiple results — one per occupation — for the same person:

 1 $ uv run wikidata_person.py\n 2   Name: Albert Einstein\n 3   Born: Ulm\n 4   Date: 1879-03-14\n 5   Occupation: scientist\n 6 \n 7   Name: Albert Einstein\n 8   Born: Ulm\n 9   Date: 1879-03-14\n10   Occupation: physicist\n11 \n12   Name: Albert Einstein\n13   Born: Ulm\n14   Date: 1879-03-14\n15   Occupation: mathematician\n16 \n17   Name: Albert Einstein\n18   Born: Ulm\n19   Date: 1879-03-14\n20   Occupation: inventor\n21   ...\n

\n\n

Key things to notice about Wikidata’s SPARQL:

\nwdt: properties represent direct claims (e.g., wdt:P19 is “place of birth”)\n
\nwd: entities are Wikidata items (e.g., wd:Q5 is “human”)\n
\nThe SERVICE wikibase:label clause automatically resolves entity IDs to human-readable labels\n
\nOPTIONAL prevents the query from failing when a property is missing\n

Querying Cities and Their Properties from DBPedia

DBPedia mirrors much of Wikipedia’s structured content as RDF triples. It uses different ontology conventions than Wikidata but is equally useful for knowledge representation tasks. Here we query DBPedia’s public SPARQL endpoint for cities and their populations:

 1 # dbpedia_cities.py - Query DBPedia for city data\n 2 \n 3 from SPARQLWrapper import SPARQLWrapper, JSON\n 4 from pprint import pprint\n 5 \n 6 queryString = \"\"\"\n 7 SELECT ?city_uri ?dbpedia_label ?population ?country_label\n 8 WHERE {\n 9     ?city_uri\n10         <http://dbpedia.org/ontology/type>\n11         <http://dbpedia.org/resource/City> .\n12     ?city_uri\n13         <http://dbpedia.org/property/populationEst>\n14         ?population .\n15     ?city_uri\n16          <http://www.w3.org/2000/01/rdf-schema#label>\n17          ?dbpedia_label FILTER (lang(?dbpedia_label) = 'en') .\n18     OPTIONAL {\n19         ?city_uri <http://dbpedia.org/ontology/country> ?country .\n20         ?country <http://www.w3.org/2000/01/rdf-schema#label>\n21                  ?country_label FILTER (lang(?country_label) = 'en') .\n22     }\n23 }\n24 ORDER BY DESC(?population)\n25 LIMIT 10\n26 \"\"\"\n27 \n28 sparql = SPARQLWrapper(\"http://dbpedia.org/sparql\")\n29 sparql.setQuery(queryString)\n30 sparql.setReturnFormat(JSON)\n31 results = sparql.queryAndConvert()\n32 \n33 for r in results[\"results\"][\"bindings\"]:\n34     city = r['dbpedia_label']['value']\n35     pop = int(r['population']['value'])\n36     country = r.get('country_label', {}).get('value', 'unknown')\n37     print(f\"  {city} ({country}): population {pop:,}\")\n

\n\n

The output (results may vary as DBPedia data is updated):

 1 $ uv run dbpedia_cities.py\n 2   Fort Worth, Texas (United States): population 1,008,106\n 3   Charlotte, North Carolina (unknown): population 911,311\n 4   Detroit (unknown): population 645,705\n 5   Gombe, Nigeria (Nigeria): population 446,800\n 6   Ilesa (Nigeria): population 416,000\n 7   Pittsburgh (unknown): population 307,668\n 8   Durham, North Carolina (United States): population 296,186\n 9   Toledo, Ohio (United States): population 265,638\n10   Winston-Salem, North Carolina (United States): population 252,975\n11   Huntsville, Alabama (unknown): population 249,102\n

\n\n

When I use RDF data from public SPARQL endpoints like DBPedia or Wikidata in applications, I start by using the web-based SPARQL clients for these services, find useful entities, manually look to see what properties are defined for those entities, and then write custom SPARQL queries to fetch the data I need. The web-based query editors at query.wikidata.org and dbpedia.org/sparql are invaluable for this exploratory process.

We will use more SPARQL queries in the next chapter.

The SQLite Relational Database for Knowledge Representation

The SQLite database is included in the standard Python distribution, making it the zero-setup option for persistent data storage. While graph databases naturally express relationships between entities, relational databases can also serve as effective knowledge representations when the schema is designed to capture entity types, attributes, and relationships.

A Reusable SQLite Library

We start with a simple reusable library for SQLite using the standard library sqlite3:

 1 # sqlite_lib.py - Reusable SQLite helper functions\n 2 \n 3 from sqlite3 import connect, version\n 4 \n 5 def create_db(db_file_path):\n 6     \"\"\"Create a database and return the connection.\"\"\"\n 7     conn = connect(db_file_path)\n 8     return conn\n 9 \n10 def connection(db_file_path):\n11     \"\"\"Create and return a database connection.\"\"\"\n12     return connect(db_file_path)\n13 \n14 def query(conn, sql, variable_bindings=None):\n15     \"\"\"Execute a SQL query and return all results.\"\"\"\n16     cur = conn.cursor()\n17     if variable_bindings:\n18         cur.execute(sql, variable_bindings)\n19     else:\n20         cur.execute(sql)\n21     conn.commit()\n22     return cur.fetchall()\n

\n\n

Modeling a Knowledge Graph in SQLite

Relational databases become knowledge representation tools when we design tables to capture entities, their types, their attributes, and the relationships between them. Here is an example that builds a simple knowledge base about scientists, their fields, and their discoveries:

  1 # sqlite_knowledge.py - Knowledge representation with SQLite\n  2 \n  3 import sqlite3\n  4 \n  5 def build_knowledge_base():\n  6     \"\"\"Build a relational knowledge base about scientists and their work.\"\"\"\n  7     conn = sqlite3.connect(\":memory:\")\n  8     cur = conn.cursor()\n  9 \n 10     # Entity tables: each table represents a type of entity\n 11     cur.execute(\"\"\"\n 12         CREATE TABLE scientists (\n 13             id INTEGER PRIMARY KEY,\n 14             name TEXT NOT NULL,\n 15             birth_year INTEGER,\n 16             nationality TEXT\n 17         )\n 18     \"\"\")\n 19 \n 20     cur.execute(\"\"\"\n 21         CREATE TABLE fields (\n 22             id INTEGER PRIMARY KEY,\n 23             name TEXT NOT NULL,\n 24             description TEXT\n 25         )\n 26     \"\"\")\n 27 \n 28     cur.execute(\"\"\"\n 29         CREATE TABLE discoveries (\n 30             id INTEGER PRIMARY KEY,\n 31             name TEXT NOT NULL,\n 32             year INTEGER,\n 33             description TEXT\n 34         )\n 35     \"\"\")\n 36 \n 37     # Relationship tables: capture how entities are connected\n 38     cur.execute(\"\"\"\n 39         CREATE TABLE scientist_field (\n 40             scientist_id INTEGER REFERENCES scientists(id),\n 41             field_id INTEGER REFERENCES fields(id),\n 42             PRIMARY KEY (scientist_id, field_id)\n 43         )\n 44     \"\"\")\n 45 \n 46     cur.execute(\"\"\"\n 47         CREATE TABLE scientist_discovery (\n 48             scientist_id INTEGER REFERENCES scientists(id),\n 49             discovery_id INTEGER REFERENCES discoveries(id),\n 50             PRIMARY KEY (scientist_id, discovery_id)\n 51         )\n 52     \"\"\")\n 53 \n 54     # Populate with knowledge\n 55     cur.executemany(\"INSERT INTO scientists VALUES (?, ?, ?, ?)\", [\n 56         (1, \"Albert Einstein\", 1879, \"German\"),\n 57         (2, \"Marie Curie\", 1867, \"Polish\"),\n 58         (3, \"Richard Feynman\", 1918, \"American\"),\n 59     ])\n 60 \n 61     cur.executemany(\"INSERT INTO fields VALUES (?, ?, ?)\", [\n 62         (1, \"Physics\", \"Study of matter, energy, and their interactions\"),\n 63         (2, \"Chemistry\", \"Study of the composition and properties of matter\"),\n 64         (3, \"Quantum Mechanics\", \"Physics of atomic and subatomic systems\"),\n 65     ])\n 66 \n 67     cur.executemany(\"INSERT INTO discoveries VALUES (?, ?, ?, ?)\", [\n 68         (1, \"Special Relativity\", 1905, \"Time and space are relative\"),\n 69         (2, \"Radioactivity\", 1898, \"Discovery of radium and polonium\"),\n 70         (3, \"Quantum Electrodynamics\", 1948, \"Quantum theory of light and matter\"),\n 71     ])\n 72 \n 73     cur.executemany(\"INSERT INTO scientist_field VALUES (?, ?)\", [\n 74         (1, 1), (1, 3),  # Einstein: Physics, Quantum Mechanics\n 75         (2, 1), (2, 2),  # Curie: Physics, Chemistry\n 76         (3, 1), (3, 3),  # Feynman: Physics, Quantum Mechanics\n 77     ])\n 78 \n 79     cur.executemany(\"INSERT INTO scientist_discovery VALUES (?, ?)\", [\n 80         (1, 1),  # Einstein -> Special Relativity\n 81         (2, 2),  # Curie -> Radioactivity\n 82         (3, 3),  # Feynman -> QED\n 83     ])\n 84 \n 85     conn.commit()\n 86     return conn\n 87 \n 88 \n 89 def query_knowledge_base(conn):\n 90     \"\"\"Demonstrate knowledge queries against the relational schema.\"\"\"\n 91     cur = conn.cursor()\n 92 \n 93     # Query 1: Who works in Quantum Mechanics?\n 94     print(\"Scientists in Quantum Mechanics:\")\n 95     cur.execute(\"\"\"\n 96         SELECT s.name, s.nationality\n 97         FROM scientists s\n 98         JOIN scientist_field sf ON s.id = sf.scientist_id\n 99         JOIN fields f ON sf.field_id = f.id\n100         WHERE f.name = 'Quantum Mechanics'\n101     \"\"\")\n102     for row in cur.fetchall():\n103         print(f\"  {row[0]} ({row[1]})\")\n104 \n105     # Query 2: What did each scientist discover?\n106     print(\"\\nDiscoveries by scientist:\")\n107     cur.execute(\"\"\"\n108         SELECT s.name, d.name, d.year, d.description\n109         FROM scientists s\n110         JOIN scientist_discovery sd ON s.id = sd.scientist_id\n111         JOIN discoveries d ON sd.discovery_id = d.id\n112         ORDER BY d.year\n113     \"\"\")\n114     for row in cur.fetchall():\n115         print(f\"  {row[0]}: {row[1]} ({row[2]}) — {row[3]}\")\n116 \n117     # Query 3: Which fields overlap between scientists?\n118     print(\"\\nScientists who share a field:\")\n119     cur.execute(\"\"\"\n120         SELECT s1.name, s2.name, f.name\n121         FROM scientist_field sf1\n122         JOIN scientist_field sf2 ON sf1.field_id = sf2.field_id\n123                                 AND sf1.scientist_id < sf2.scientist_id\n124         JOIN scientists s1 ON sf1.scientist_id = s1.id\n125         JOIN scientists s2 ON sf2.scientist_id = s2.id\n126         JOIN fields f ON sf1.field_id = f.id\n127     \"\"\")\n128     for row in cur.fetchall():\n129         print(f\"  {row[0]} & {row[1]}: {row[2]}\")\n130 \n131 \n132 conn = build_knowledge_base()\n133 query_knowledge_base(conn)\n134 conn.close()\n

\n\n

The output shows how SQL JOIN queries traverse the relationships between entities, much like following edges in a graph:

 1 Scientists in Quantum Mechanics:\n 2   Albert Einstein (German)\n 3   Richard Feynman (American)\n 4 \n 5 Discoveries by scientist:\n 6   Marie Curie: Radioactivity (1898) — Discovery of radium and polonium\n 7   Albert Einstein: Special Relativity (1905) — Time and space are relative\n 8   Richard Feynman: Quantum Electrodynamics (1948) — Quantum theory of light and matter\n 9 \n10 Scientists who share a field:\n11   Albert Einstein & Marie Curie: Physics\n12   Albert Einstein & Richard Feynman: Physics\n13   Albert Einstein & Richard Feynman: Quantum Mechanics\n14   Marie Curie & Richard Feynman: Physics\n

\n\n

The key insight is that the relationship tables (scientist_field, scientist_discovery) transform a flat relational database into a knowledge representation. Each relationship table captures a specific type of connection between entity types, and SQL JOINs let you traverse these connections to answer knowledge queries. While not as natural as a graph database for highly connected data, this pattern works well for structured knowledge with well-defined entity types and relationships.

We will combine the use of SQLite, RDF, SPARQL, and deep learning Natural Language Processing (NLP) libraries later in the book.

If you want to deepen your understanding of the standards behind the SPARQL queries we used in this chapter, the next chapter provides optional reference material on RDF data formats, RDFS sub-property hierarchies, the SPARQL query language in detail, and OWL reasoning. That background will help you write more sophisticated queries against Wikidata, DBPedia, and any other SPARQL endpoint.

Optional Material: A Deeper Dive Into Semantic Web and Linked Data

In the previous chapter we used Python to query Wikidata and DBPedia with SPARQL and built a knowledge base in SQLite. Those examples gave you working code for practical knowledge representation tasks. This optional chapter provides the theoretical background behind those tools — if you understand the standards that underpin SPARQL endpoints, you will be able to write more sophisticated queries, design your own RDF schemas, and use reasoning to infer new knowledge from existing data.

Specifically, this chapter covers:

\nRDF — The data format behind Wikidata, DBPedia, and every SPARQL endpoint. Understanding RDF triples, namespaces, and serialization formats will help you design better queries and interpret results from the examples in the previous chapter.\n
\nRDFS — How to define sub-properties and class hierarchies so that a single SPARQL query can match related predicates automatically (e.g., querying for “containsPlace” also matches “containsCity” and “containsCountry”).\n
\nSPARQL in depth — The query patterns, filters, OPTIONAL clauses, and ordering that let you write expressive queries against public knowledge graphs.\n
\nOWL — The Web Ontology Language, which extends RDFS with transitive properties, inverse properties, and description logic reasoning.\n

This material is not required to use the Python examples from the previous chapter, but it will deepen your understanding and help you build more powerful knowledge representation applications.

The scope of the Semantic Web is comprised of all public data sources on the Internet that follow specific standards like RDF. Knowledge Graphs may be large scale, as the graphs that drive Google’s and Facebook’s businesses, or they can be specific to an organization. Knowledge Graphs may be in customer-facing applications or part of internal engineering infrastructure.

The Semantic Web is a vision for the future of the World Wide Web, where data on the web is represented in a way that machines can understand, reason about, and use to perform tasks. It is built on top of the existing web and is designed to be backward-compatible. The Semantic Web relies on the use of ontologies, which are formal representations of a domain of knowledge, and the Resource Description Framework (RDF), which is a standard for expressing ontologies in a machine-readable format.

Linked Data is a set of best practices for publishing and linking data on the web using RDF. It involves the use of URIs (Uniform Resource Identifiers) to identify resources on the web and the use of links (also known as triples) to connect resources. This allows for the creation of a web of interconnected data, where each resource can be linked to other resources in a way that machines can understand and follow.

Together, the Semantic Web and Linked Data provide a framework for representing and interlinking data on the web in a structured and machine-readable format, allowing for the integration, querying, and reasoning over large amounts of information. This helps to make data on the web more accessible and useful to both humans and machines, enabling more powerful and intelligent applications and services.

As a developer you are likely familiar with the term data lake for enterprise-wide relational and other types of databases. Graph databases and RDF data stores are another tool to implement data lakes but I prefer using the term Knowledge Graph for RDF/RDFS/OWL/SPARQL based data stores because of the ability to infer data that is not explicitly stated as well as providing abstractions for merging different data sources in-place, that is, without the requirement to convert data to other formats or database infrastructure.

Overview and Theory

You will learn how to do the following:

\nUnderstand RDF data formats (the foundation behind the Wikidata and DBPedia queries from the previous chapter).\n
\nSee more use cases for SPARQL queries that go beyond the simple SELECT patterns we used earlier.\n
\nLearn about RDFS sub-property hierarchies and OWL reasoning, which enable more powerful knowledge queries.\n

The Semantic Web is intended to provide a massive linked set of data for use by software systems just as the World Wide Web provides a massive collection of linked web pages for human reading and browsing. The Semantic Web is like the web in that anyone can generate any content that they want. This freedom to publish anything works for the web because we use our ability to understand natural language to interpret what we read – and often to dismiss material that based upon our own knowledge we consider to be incorrect.

Semantic Web and linked data technologies are also useful for smaller amounts of data, an example being a Knowledge Graph containing information for a business. We will further explore Knowledge Graph use cases.

The core concept for the Semantic Web is data integration and use from different sources. As we will soon see, the tools for implementing the Semantic Web are designed for encoding data and sharing data from many different sources.

I cover the Semantic Web in this book because I believe that Semantic Web technologies are complementary to AI systems for gathering and processing data on the web. As more web pages are generated by applications (as opposed to simply showing static HTML files) it becomes easier to produce both HTML for human readers and semantic data for software agents.

RDF: The Universal Data Format

The Resource Description Framework (RDF) is used to encode information and the RDF Schema (RDFS) facilitates using data with different RDF encodings without the need to convert one set of schemas to another. Later, using OWL, we can simply declare that one predicate (or property) is the same as another; that is, one predicate is a sub-predicate of another (e.g., a property containsCity can be declared to be a sub-property of containsPlace so if something contains a city then it also contains a place), etc. The predicate part of an RDF statement often refers to a property.

RDF data was originally encoded as XML and intended for automated processing. In this chapter we will use two simple to read formats called “N-Triples” and “N3.” Apache Jena can be used to convert between all RDF formats so we might as well use formats that are easier to read and understand. RDF data consists of a set of triple values:

\nsubject\n
\npredicate\n
\nobject\n

Some of my work with Semantic Web technologies deals with processing news stories, extracting semantic information from the text, and storing it in RDF. I will use this application domain for the examples in this chapter when we implement code to automatically generate RDF for Knowledge Graphs. I deal with triples like:

\nSubject: a URL (or URI) of a news article.\n
\nPredicate (or property): a relation like “containsPerson”.\n
\nObject: a literal value like “Bill Clinton” or a URI representing Bill Clinton.\n

In general subjects refer to entities. In the next chapter we will use the entity recognition library we developed in an earlier chapter to create RDF from text input.

We will use either URIs or string literals as values for objects. We will always use URIs for representing subjects and predicates. In any case URIs are usually preferred to string literals. We will see an example of this preferred use but first we need to learn the N-Triple and N3 RDF formats.

I propose that the idea that RDF is more flexible than Object Modeling in programming languages, relational databases, and XML with schemas. If we can tag new attributes on the fly to existing data, how do we prevent what I might call “data chaos” as we modify existing data sources? It turns out that the solution to this problem is also the solution for encoding real semantics (or meaning) with data: we usually use unique URIs for RDF subjects, predicates, and objects, and usually with a preference for not using string literals. The definitions of predicates are tied to a namespace and later with OWL we will state the equivalence of predicates in different namespaces with the same semantic meaning. I will try to make this idea more clear with some examples and for further reference Wikipedia has a good writeup on RDF.

Any part of a triple (subject, predicate, or object) is either a URI or a string literal. URIs encode namespaces. For example, the containsPerson predicate in the last example could be written as:

1 http://knowledgebooks.com/ontology/#containsPerson\n

\n\n

The first part of this URI is considered to be the namespace for this predicate “containsPerson.” When different RDF triples use this same predicate, this is some assurance to us that all users of this predicate understand the same meaning. Furthermore, we will see later that we can use RDFS to state equivalency between this predicate (in the namespace http://knowledgebooks.com/ontology/) with predicates represented by different URIs used in other data sources. In an “artificial intelligence” sense, software that we write does not understand predicates like “containsCity”, “containsPerson”, or “isLocation” in the way that a human reader can by combining understood common meanings for the words “contains”, “city”, “is”, “person”, and “location” but for many interesting and useful types of applications that is fine as long as the predicate is used consistently. We will see that we can define abbreviation prefixes for namespaces which makes RDF and RDFS files shorter and easier to read.

There are many serialization formats for RDF (we will mostly use JSON in our Python examples):

\nTurtle\n
\nN3\n
\nN-Triples\n
\nNQuads\n
\nTriG\n-JSON\n
\nJSON-LD\n
\nRDF/XML\n
\nRDF/JSON\n
\nTriX\n
\nRDF Binary\n

A statement in N-Triple format consists of three URIs (two URIs and a string literals for the object) followed by a period to end the statement. While statements are often written one per line in a source file they can be broken across lines; it is the ending period which marks the end of a statement. The standard file extension for N-Triple format files is *.nt and the standard format for N3 format files is *.n3.

My preference is to use N-Triple format files as output from programs that I write to save data as RDF. N-Triple files don’t use any abbreviations and each RDF statement is self-contained. I often use tools like the command line commands in Jena or RDF4J to convert N-Triple files to N3 or other formats if I will be reading them or even hand editing them. Here is an example using the N3 syntax:

1 @prefix kb:  <http://knowledgebooks.com/ontology#>\n2 \n3 <http://news.com/201234/> kb:containsCountry \"China\" .\n

\n\n

The N3 format adds prefixes (abbreviations) to the N-Triple format. In practice it would be better to use the URI http://dbpedia.org/resource/China instead of the literal value “China.”

Here we see the use of an abbreviation prefix “kb:” for the namespace for my company KnowledgeBooks.com ontologies. The first term in the RDF statement (the subject) is the URI of a news article. The second term (the predicate) is “containsCountry” in the “kb:” namespace. The last item in the statement (the object) is a string literal “China.” I would describe this RDF statement in English as, “The news article at URI http://news.com/201234 mentions the country China.”

This was a very simple N3 example which we will expand to show additional features of the N3 notation. As another example, let’s look at the case of this news article also mentioning the USA. Instead of adding a whole new statement like this we can combine them using N3 notation. Here we have two separate RDF statements:

1 @prefix kb:  <http://knowledgebooks.com/ontology#> .\n2 \n3 <http://news.com/201234/>\n4   kb:containsCountry\n5   <http://dbpedia.org/resource/China>  .\n6   \n7 <http://news.com/201234/>\n8   kb:containsCountry\n9   <http://dbpedia.org/resource/United_States>  .\n

\n\n

We can collapse multiple RDF statements that share the same subject and optionally the same predicate:

1 @prefix kb:  <http://knowledgebooks.com/ontology#> .\n2 \n3 <http://news.com/201234/>\n4   kb:containsCountry\n5     <http://dbpedia.org/resource/China> ,\n6     <http://dbpedia.org/resource/United_States>  .\n

\n\n

The indentation and placement on separate lines is arbitrary - use whatever style you like that is readable. We can also add in additional predicates that use the same subject (I am going to use string literals here instead of URIs for objects to make the following example more concise but in practice prefer using URIs):

 1 @prefix kb:  <http://knowledgebooks.com/ontology#> .\n 2 \n 3 <http://news.com/201234/>\n 4         kb:containsCountry \"China\" ,\n 5                            \"USA\" .\n 6         kb:containsOrganization \"United Nations\" ;\n 7         kb:containsPerson \"Ban Ki-moon\" , \"Gordon Brown\" ,\n 8                           \"Hu Jintao\" , \"George W. Bush\" ,\n 9                           \"Pervez Musharraf\" ,\n10                           \"Vladimir Putin\" , \n11                           \"Mahmoud Ahmadinejad\" .\n

\n\n

This single N3 statement represents ten individual RDF triples. Each section defining triples with the same subject and predicate have objects separated by commas and ending with a period. Please note that whatever RDF storage system you use (we will be using Jena) it makes no difference if we load RDF as XML, N-Triple, or N3 format files: internally subject, predicate, and object triples are stored in the same way and are used in the same way. RDF triples in a data store represent directed graphs that may not all be connected.

I promised you that the data in RDF data stores was easy to extend. As an example, let us assume that we have written software that is able to read online news articles and create RDF data that captures some of the semantics in the articles. If we extend our program to also recognize dates when the articles are published, we can simply reprocess articles and for each article add a triple to our RDF data store using a form like:

1 @prefix kb:  <http://knowledgebooks.com/ontology#> .\n2 \n3 <http://news.com/201234/>\n4   kb:datePublished\n5   \"2008-05-11\" .\n

\n\n

Note that I split one RDF statement across three lines (3-5) but this could have been on one line. The RDF statement on lines 3-5 is legal and will be handled correctly by RDF parsers. Here we just represent the date as a string. We can add a type to the object representing a specific date:

1 @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\n2 @prefix kb:  <http://knowledgebooks.com/ontology#> .\n3  \n4  <http://news.com/201234/>\n5    kb:datePublished\n6    \"2008-05-11\"^^xsd:date .\n

\n\n

Furthermore, if we do not have dates for all news articles, that is often acceptable because when constructing SPARQL queries you can match optional patterns. If for example you are looking up articles on a specific subject then some results may have a publication date attached to the results for that article and some might not. In practice RDF supports types and we would use a date type as seen in the last example, not a string. However, in designing the example programs for this chapter I decided to simplify our representation of URIs and often use string literals as simple Java strings.

Extending RDF with RDF Schema

RDF Schema (RDFS) supports the definition of classes and properties based on set inclusion. In RDFS classes and properties are orthogonal. Let’s start with looking at an example using additional namespaces:

 1 @prefix kb:   <http://knowledgebooks.com/ontology#> .\n 2 @prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .\n 3 @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n 4 @prefix dbo:  <http://dbpedia.org/ontology/> .\n 5 \n 6 <http://news.com/201234/>\n 7   kb:containsCountry\n 8   <http://dbpedia.org/resource/China>  .\n 9   \n10 <http://news.com/201234/>\n11   kb:containsCountry\n12   <http://dbpedia.org/resource/United_States>  .\n13   \n14 <http://dbpedia.org/resource/China>\n15   rdfs:label \"China\"@en,\n16   rdf:type dbo:Place ,\n17   rdf:type dbo:Country .\n

\n\n

Because the Semantic Web is intended to be processed automatically by software systems it is encoded as RDF. There is a problem that must be solved in implementing and using the Semantic Web: everyone who publishes Semantic Web data is free to create their own RDF schemas for storing data. For example, there is usually no single standard RDF schema definition for topics like news stories and stock market data. The SKOS is a namespace containing standard schemas and the most widely used standard is schema.org. Understanding the ways of integrating different data sources using different schemas helps to understand the design decisions behind the Semantic Web applications. In this chapter I often use my own schemas in the knowledgebooks.com namespace for the simple examples you see here. When you build your own production systems part of the work is searching through schema.org and SKOS to use standard name spaces and schemas when possible because this facilitates linking your data to other RDF Data on the web. The use of standard schemas helps when you link internal proprietary Knowledge Graphs used inside your organization with public open data from sources like WikiData and DBPedia.

Let’s consider an example: suppose that your local Knowledge Graph referred to President Joe Biden in which case we could “mint” our own URI like:

1 https://knowledgebooks.com/person#Joe_Biden\n

\n\n

In this case users of the local Knowledge Graph could not take advantage of connected data. For example, the DBPedia and WikiData URIs for Joe Biden are:

1 https://dbpedia.org/resource/Joe_Biden\n2 http://www.wikidata.org/entity/Q6279\n

\n\n

Both of these URIs can be followed by clicking on the links if you are reading a PDF copy of this book. Please “follow your nose” and see how both of these URIs resolve to human-readable web pages.

After telling you, dear reader, to always try to use public and standard URIs like the above examples for Joe Biden, I will now revert to using simple made-up URIs for the following discussion.

We will start with an example that is an extension of the example in the last section that also uses RDFS. We add a few additional RDF statements:

1 @prefix kb:  <http://knowledgebooks.com/ontology#> .\n2 @prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .\n3 \n4 kb:containsCity rdfs:subPropertyOf kb:containsPlace .\n5 kb:containsCountry rdfs:subPropertyOf kb:containsPlace .\n6 kb:containsState rdfs:subPropertyOf kb:containsPlace .\n

\n\n

The last three lines declare that:

\nThe property containsCity is a sub-property of containsPlace.\n
\nThe property containsCountry is a sub-property of containsPlace.\n
\nThe property containsState is a sub-property of containsPlace.\n

Why is this useful? For at least two reasons:

You can query an RDF data store for all triples that use property containsPlace and also match triples with properties equal to containsCity, containsCountry, or containsState. There may not even be any triples that explicitly use the property containsPlace.

Consider a hypothetical case where you are using two different RDF data stores that use different properties for naming cities: cityName and city. You can define cityName to be a sub-property of city and then write all queries against the single property name city. This removes the necessity to convert data from different sources to use the same Schema. You can also use OWL to state property and class equivalency.

In addition to providing a vocabulary for describing properties and class membership by properties, RDFS is also used for logical inference to infer new triples, combine data from different RDF data sources, and to allow effective querying of RDF data stores. We will see examples of all of these features of RDFS when we later when using libraries to perform SPARQL queries.

The SPARQL Query Language

We briefly covered the use of SPARQL in the last chapter. SPARQL is a query language used to query RDF data stores. While SPARQL may initially look like SQL, we will see that there are some important differences like support for RDFS and OWL inferencing and graph-based instead of relational matching operations. We will cover the basics of SPARQL in this section and then see more examples later when we learn how to embed SPARQL queries in Python applications.

We will use the N3 format RDF file test_data/news.n3 for the examples. I created this file automatically by spidering Reuters news stories on the news.yahoo.com web site and automatically extracting named entities from the text of the articles. In this chapter we use these sample RDF files.

You have already seen snippets of this file and I list the entire file here for reference, edited to fit line width: you may find the file news.n3 easier to read if you are at your computer and open the file in a text editor so you will not be limited to what fits on a book page:

 1 @prefix kb:  <http://knowledgebooks.com/ontology#> .\n 2 @prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .\n 3 \n 4 kb:containsCity rdfs:subPropertyOf kb:containsPlace .\n 5 \n 6 kb:containsCountry rdfs:subPropertyOf kb:containsPlace .\n 7 \n 8 kb:containsState rdfs:subPropertyOf kb:containsPlace .\n 9 \n10 <http://yahoo.com/20080616/usa_flooding_dc_16/>\n11         kb:containsCity \"Burlington\" , \"Denver\" ,\n12                         \"St. Paul\" ,\" Chicago\" ,\n13                         \"Quincy\" , \"CHICAGO\" ,\n14                         \"Iowa City\" ;\n15         kb:containsRegion \"U.S. Midwest\" , \"Midwest\" ;\n16         kb:containsCountry \"United States\" , \"Japan\" ;\n17         kb:containsState \"Minnesota\" , \"Illinois\" , \n18                          \"Mississippi\" , \"Iowa\" ;\n19         kb:containsOrganization \"National Guard\" ,\n20                          \"U.S. Department of Agriculture\",\n21                          \"White House\" ,\n22                          \"Chicago Board of Trade\" ,\n23                          \"Department of Transportation\" ;\n24         kb:containsPerson \"Dena Gray-Fisher\" ,\n25                           \"Donald Miller\" ,\n26                           \"Glenn Hollander\" ,\n27                           \"Rich Feltes\" ,\n28                           \"George W. Bush\" ;\n29         kb:containsIndustryTerm \"food inflation\", \"food\",\n30                                 \"finance ministers\" ,\n31                                 \"oil\" .\n32 \n33 <http://yahoo.com/78325/ts_nm/usa_politics_dc_2/>\n34         kb:containsCity \"Washington\" , \"Baghdad\" ,\n35                         \"Arlington\" , \"Flint\" ;\n36         kb:containsCountry \"United States\" ,\n37                            \"Afghanistan\" ,\n38                            \"Iraq\" ;\n39         kb:containsState \"Illinois\" , \"Virginia\" ,\n40                          \"Arizona\" , \"Michigan\" ;\n41         kb:containsOrganization \"White House\" ,\n42                                 \"Obama administration\" ,\n43                                 \"Iraqi government\" ;\n44         kb:containsPerson \"David Petraeus\" ,\n45                           \"John McCain\" ,\n46                           \"Hoshiyar Zebari\" ,\n47                           \"Barack Obama\" ,\n48                           \"George W. Bush\" ,\n49                           \"Carly Fiorina\" ;\n50         kb:containsIndustryTerm \"oil prices\" .\n51 \n52 <http://yahoo.com/10944/ts_nm/worldleaders_dc_1/>\n53         kb:containsCity \"WASHINGTON\" ;\n54         kb:containsCountry \"United States\" , \"Pakistan\" ,\n55                            \"Islamic Republic of Iran\" ;\n56         kb:containsState \"Maryland\" ;\n57         kb:containsOrganization \"University of Maryland\" ,\n58                                 \"United Nations\" ;\n59         kb:containsPerson \"Ban Ki-moon\" , \"Gordon Brown\" ,\n60                           \"Hu Jintao\" , \"George W. Bush\" ,\n61                           \"Pervez Musharraf\" ,\n62                           \"Vladimir Putin\" ,\n63                           \"Steven Kull\" ,\n64                           \"Mahmoud Ahmadinejad\" .\n65 \n66 <http://yahoo.com/10622/global_economy_dc_4/>\n67         kb:containsCity \"Sao Paulo\" , \"Kuala Lumpur\" ;\n68         kb:containsRegion \"Midwest\" ;\n69         kb:containsCountry \"United States\" , \"Britain\" ,\n70                            \"Saudi Arabia\" , \"Spain\" ,\n71                            \"Italy\" , India\" , \n72                            \"\"France\" , \"Canada\" ,\n73                            \"Russia\", \"Germany\", \"China\",\n74                            \"Japan\" , \"South Korea\" ;\n75         kb:containsOrganization \"Federal Reserve Bank\" ,\n76                                 \"European Union\" ,\n77                                 \"European Central Bank\" ,\n78                                 \"European Commission\" ;\n79         kb:containsPerson \"Lee Myung-bak\" , \"Rajat Nag\" ,\n80                           \"Luiz Inacio Lula da Silva\" ,\n81                           \"Jeffrey Lacker\" ;\n82         kb:containsCompany\n83             \"Development Bank Managing\" ,\n84             \"Reuters\" ,\n85             \"Richmond Federal Reserve Bank\";\n86         kb:containsIndustryTerm \"central bank\" , \"food\" ,\n87                                 \"energy costs\" ,\n88                                 \"finance ministers\" ,\n89                                 \"crude oil prices\" ,\n90                                 \"oil prices\" ,\n91                                 \"oil shock\" ,\n92                                 \"food prices\" ,\n93                                 \"Finance ministers\" ,\n94                                 \"Oil prices\" , \"oil\" .\n

\n\n

Please note that in the above RDF listing I took advantage of the free form syntax of N3 and Turtle RDF formats to reformat the data to fit page width.

We will start with a simple SPARQL query for subjects (news article URLs) and objects (matching countries) with the value for the predicate equal to containsCountry. Variables in queries start with a question mark character and can have any names:

1 SELECT ?subject ?object\n2   WHERE {\n3    ?subject\n4    <http://knowledgebooks.com/ontology#containsCountry>\n5    ?object .\n6 }\n

\n\n

It is important for you to understand what is happening when we apply the last SPARQL query to our sample data. Conceptually, all the triples in the sample data are scanned, keeping the ones where the predicate part of a triple is equal to

1 http://knowledgebooks.com/ontology#containsCountry.\n

\n\n

In practice RDF data stores supporting SPARQL queries index RDF data so a complete scan of the sample data is not required. This is analogous to relational databases where indices are created to avoid needing to perform complete scans of database tables.

In practice, when you are exploring a Knowledge Graph like DBPedia or WikiData (that are just very large collections of RDF triples), you might run a query and discover a useful or interesting entity URI in the triple store, then drill down to find out more about the entity. In a later example in this chapter (Knowledge Graph Navigator) we attempt to automate this exploration process using the DBPedia data as a Knowledge Graph.

We will be using the same code to access the small example of RDF statements in our sample data as we will for accessing DBPedia or WikiData.

We can make this last query easier to read and reduce the chance of misspelling errors by using a namespace prefix:

1 PREFIX kb:  <http://knowledgebooks.com/ontology#>\n2 SELECT ?subject ?object\n3   WHERE {\n4       ?subject kb:containsCountry ?object .\n5   }\n

\n\n

We could have filtered on any other predicate, for instance containsPlace. Here is another example using a match against a string literal to find all articles exactly matching the text “Maryland.”

1 PREFIX kb:  <http://knowledgebooks.com/ontology#>\n2 SELECT ?subject WHERE { ?subject kb:containsState \"Maryland\" . }\n

\n\n

The output is:

1 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n

\n\n

We can also match partial string literals against regular expressions:

1 PREFIX kb: <http://knowledgebooks.com/ontology#>\n2 SELECT ?subject ?object\n3        WHERE {\n4          ?subject\n5          kb:containsOrganization\n6          ?object FILTER regex(?object, \"University\") .\n7        }\n

\n\n

The output is:

1 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1\n2   \"University of Maryland\"\n

\n\n

We might want to return all triples matching a property of containing an organization and where the object is a string containing the substring “University.” The matching statement after the FILTER check matches every triple that matches the subject in the first pattern:

 1 PREFIX kb: <http://knowledgebooks.com/ontology#>\n 2 SELECT DISTINCT ?subject ?a_predicate ?an_object\n 3  WHERE {\n 4     ?subject kb:containsOrganization ?object .\n 5        FILTER regex(?object,\"University\") .\n 6     ?subject ?a_predicate ?an_object .\n 7 }\n 8 ORDER BY ?a_predicate ?an_object\n 9 LIMIT 10\n10 OFFSET 5\n

\n\n

When WHERE clauses contain more than one triple pattern to match, this is equivalent to a Boolean “and” operation. The DISTINCT clause removes duplicate results. The ORDER BY clause sorts the output in alphabetical order: in this case first by predicate (containsCity, containsCountry, etc.) and then by object. The LIMIT modifier limits the number of results returned and the OFFSET modifier sets the number of matching results to skip.

The output is:

 1 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n 2     http://knowledgebooks.com/ontology#containsOrganization\n 3    \"University of Maryland\" .\n 4 \n 5 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n 6    http://knowledgebooks.com/ontology#containsPerson,\n 7    \"Ban Ki-moon\" .\n 8     \n 9 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n10    http://knowledgebooks.com/ontology#containsPerson\n11    \"George W. Bush\" .\n12 \n13 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n14    http://knowledgebooks.com/ontology#containsPerson\n15    \"Gordon Brown\" .\n16 \n17 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n18    http://knowledgebooks.com/ontology#containsPerson\n19    \"Hu Jintao\" .\n20 \n21 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n22    http://knowledgebooks.com/ontology#containsPerson\n23    \"Mahmoud Ahmadinejad\" .\n24 \n25 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n26    http://knowledgebooks.com/ontology#containsPerson\n27    \"Pervez Musharraf\" .\n28 \n29 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n30    http://knowledgebooks.com/ontology#containsPerson\n31    \"Steven Kull\" .\n32 \n33 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n34    http://knowledgebooks.com/ontology#containsPerson\n35    \"Vladimir Putin\" .\n36 \n37 http://news.yahoo.com/s/nm/20080616/ts_nm/worldleaders_trust_dc_1/\n38    http://knowledgebooks.com/ontology#containsState\n39    \"Maryland\" .\n

\n\n

We are finished with our quick tutorial on using the SELECT query form. There are three other query forms that I am not covering in this chapter:

\nCONSTRUCT – returns a new RDF graph of query results.\n
\nASK – returns Boolean true or false indicating if a query matches any triples.\n
\nDESCRIBE – returns a new RDF graph containing matched resources.\n

We will later use the OPTIONAL query form that allows complex queries to contain patterns that do not have to match.

OWL: The Web Ontology Language

We have already seen a few examples of using RDFS to define sub-properties in this chapter. The Web Ontology Language (OWL) extends the expressive power of RDFS. We now look at a few OWL examples and then look at parts of the Java unit test showing three SPARQL queries that use OWL reasoning. The following RDF data stores support at least some level of OWL reasoning:

\nProtegeOwlApis - compatible with the Protege Ontology editor.\n
\nPellet - DL reasoner.\n
\nOwlim - OWL DL reasoner compatible with some versions of Sesame.\n
\nJena - General purpose library that is used in Apache Jena Fuseki.\n
\nOWLAPI - a simpler API using many other libraries.\n
\nStardog - a commercial OWL and RDF reasoning system and datastore.\n
\nAllegrograph - a commercial RDF+ and RDF reasoning system and datastore.\n

OWL is more expressive than RDFS in that it supports cardinality, richer class relationships, and Descriptive Logic (DL) reasoning. OWL treats the idea of classes very differently than object oriented programming languages like Java and Smalltalk. In OWL, instances of a class are referred to as individuals and class membership is determined by a set of properties that allow a DL reasoner to infer class membership of an individual (this is called entailment.)

We have been using the RDF file news.n3 in previous examples and we will layer new examples by adding new triples that represent RDF, RDFS, and OWL. We saw in news.n3 the definition of three triples using rdfs:subPropertyOf properties to create a more general kb:containsPlace property:

1 kb:containsCity rdfs:subPropertyOf kb:containsPlace .\n2 kb:containsCountry rdfs:subPropertyOf kb:containsPlace .\n3 kb:containsState rdfs:subPropertyOf kb:containsPlace .\n4 \n5 kb:containsPlace rdf:type owl:transitiveProperty .\n6 \n7 kbplace:UnitedStates kb:containsState kbplace:Illinois .\n8 kbplace:Illinois kb:containsCity kbplace:Chicago .\n

\n\n

We can also infer that:

1 kbplace:UnitedStates kb:containsPlace kbplace:Chicago .\n

\n\n

We can also model inverse properties in OWL. For example, here we add an inverse property kb:containedIn, adding it to the example in the last listing:

1 kb:containedIn owl:inverseOf kb:containsPlace .\n

\n\n

Given an RDF container that supported OWL, we can now execute SPARQL queries matching the property kb:containedIn and “match” triples in the RDF triple store that have never been asserted but are inferred by the OWL reasoner.

You should understand the concept of class membership not by explicitly stating that an object (or individual) is a member of a class, but rather because an individual has properties that can be used to infer class membership.

The World Wide Web Consortium has defined three versions of the OWL language that are in increasing order of complexity: OWL Lite, OWL DL, and OWL Full. OWL DL (supports Description Logic) is the most widely used (and recommended) version of OWL. OWL Full is not computationally decidable since it supports full logic, multiple class inheritance, and other things that probably make it computationally intractable for all but smaller problems.

\n\n

\n\n","metaText","Artificial Intelligence programming with Python. Covers both theory and practical examples of LLMs, deep learning, symbolic AI, and Knowledge Representation","slug","pythonai","showReadLinkString","full","tocStructure",[44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,19,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127],"publicChapterContent",{"_22":23,"_12":293,"_25":-5,"_26":294,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":291,"_25":-5,"_26":292,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":289,"_25":-5,"_26":290,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":287,"_25":-5,"_26":288,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":285,"_25":-5,"_26":286,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":283,"_25":-5,"_26":284,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":281,"_25":-5,"_26":282,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":279,"_25":-5,"_26":280,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":277,"_25":-5,"_26":278,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":275,"_25":-5,"_26":276,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":273,"_25":-5,"_26":274,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":271,"_25":-5,"_26":272,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":269,"_25":-5,"_26":270,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":267,"_25":-5,"_26":268,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":265,"_25":-5,"_26":266,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":263,"_25":-5,"_26":264,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":261,"_25":-5,"_26":262,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":259,"_25":-5,"_26":260,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":257,"_25":-5,"_26":258,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":255,"_25":-5,"_26":256,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":253,"_25":-5,"_26":254,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":251,"_25":-5,"_26":252,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":249,"_25":-5,"_26":250,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":247,"_25":-5,"_26":248,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":245,"_25":-5,"_26":246,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":243,"_25":-5,"_26":244,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":241,"_25":-5,"_26":242,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":239,"_25":-5,"_26":240,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":237,"_25":-5,"_26":238,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":235,"_25":-5,"_26":236,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":233,"_25":-5,"_26":234,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":231,"_25":-5,"_26":232,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":229,"_25":-5,"_26":230,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":227,"_25":-5,"_26":228,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":225,"_25":-5,"_26":226,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":223,"_25":-5,"_26":224,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":221,"_25":-5,"_26":222,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":219,"_25":-5,"_26":220,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":217,"_25":-5,"_26":218,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":215,"_25":-5,"_26":216,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":213,"_25":-5,"_26":214,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":211,"_25":-5,"_26":212,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":209,"_25":-5,"_26":210,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":207,"_25":-5,"_26":208,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":205,"_25":-5,"_26":206,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":203,"_25":-5,"_26":204,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":201,"_25":-5,"_26":202,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":182,"_25":-5,"_26":200,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":198,"_25":-5,"_26":199,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":196,"_25":-5,"_26":197,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":194,"_25":-5,"_26":195,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":192,"_25":-5,"_26":193,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":190,"_25":-5,"_26":191,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":188,"_25":-5,"_26":189,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":186,"_25":-5,"_26":187,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":184,"_25":-5,"_26":185,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":182,"_25":-5,"_26":183,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":159,"_25":-5,"_26":181,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":179,"_25":-5,"_26":180,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":177,"_25":-5,"_26":178,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":175,"_25":-5,"_26":176,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":173,"_25":-5,"_26":174,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":171,"_25":-5,"_26":172,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":169,"_25":-5,"_26":170,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":167,"_25":-5,"_26":168,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":165,"_25":-5,"_26":166,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":163,"_25":-5,"_26":164,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":161,"_25":-5,"_26":162,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":159,"_25":-5,"_26":160,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":157,"_25":-5,"_26":158,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":155,"_25":-5,"_26":156,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":153,"_25":-5,"_26":154,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":151,"_25":-5,"_26":152,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":149,"_25":-5,"_26":150,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":147,"_25":-5,"_26":148,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":145,"_25":-5,"_26":146,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":143,"_25":-5,"_26":144,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":141,"_25":-5,"_26":142,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":139,"_25":-5,"_26":140,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":137,"_25":-5,"_26":138,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":135,"_25":-5,"_26":136,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":133,"_25":-5,"_26":134,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":131,"_25":-5,"_26":132,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":129,"_25":-5,"_26":130,"_28":29,"_30":-5,"_31":-5,"_32":-5},3,"Overview and Theory","#overview-and-theory","Optional Material: A Deeper Dive Into Semantic Web and Linked Data","#optional-material-a-deeper-dive-into-semantic-web-and-linked-data","The SQLite Relational Database for Knowledge Representation","#the-sqlite-relational-database-for-knowledge-representation","Querying Wikidata with SPARQL and Python","#querying-wikidata-with-sparql-and-python","Getting Setup To Use Graph and Relational Databases","#getting-setup-to-use-graph-and-relational-databases","Part 5 - Knowledge Representation","#part-5---knowledge-representation","Good Old Fashioned Symbolic AI Wrap-up","#good-old-fashioned-symbolic-ai-wrap-up","Constraint Programming with MiniZinc and Python","#constraint-programming-with-minizinc-and-python","Soar Cognitive Architecture","#soar-cognitive-architecture","Swi-Prolog and Python Deep Learning Interop","#swi-prolog-and-python-deep-learning-interop","Use Predicate Logic by Calling Swi-Prolog","#use-predicate-logic-by-calling-swi-prolog","Implementing Frame Data Structures in Python","#implementing-frame-data-structures-in-python","Comparison of Symbolic AI and Deep Learning","#comparison-of-symbolic-ai-and-deep-learning","Symbolic AI","#symbolic-ai","Part 4 - Symbolic AI and Knowledge Representation","#part-4---symbolic-ai-and-knowledge-representation","Summary","#summary-1","Hardware Considerations","#hardware-considerations","Alternative Tools for Running Local Models","#alternative-tools-for-running-local-models","OpenAI-Compatible API","#openai-compatible-api","Prompt Caching for Performance","#prompt-caching-for-performance","Conversation Memory with Ollama","#conversation-memory-with-ollama","Reasoning with Local Models","#reasoning-with-local-models","Using Ollama from Python","#using-ollama-from-python","Downloading and Running Models","#downloading-and-running-models","Installing Ollama","#installing-ollama","LLMs with Local Models","#llms-with-local-models","#summary","Practical Considerations","#practical-considerations-1","Structured Output","#structured-output","Web Search with LLMs","#web-search-with-llms","Multimodal Input: Analyzing Images","#multimodal-input-analyzing-images","Multi-Turn Conversations","#multi-turn-conversations","Thinking Models","#thinking-models","Text Generation","#text-generation","Setup and Authentication","#setup-and-authentication","LLMs with Public APIs","#llms-with-public-apis","#practical-considerations","Key Capabilities of Modern LLMs","#key-capabilities-of-modern-llms","From Transformers to Large Language Models","#from-transformers-to-large-language-models","Tokenization","#tokenization","The Transformer Architecture","#the-transformer-architecture","Introduction to Transformers and Large Language Models","#introduction-to-transformers-and-large-language-models","Part 3 - Large Language Models","#part-3---large-language-models","Recommendation Systems Wrap-up","#recommendation-systems-wrap-up","TensorFlow Recommenders","#tensorflow-recommenders","Overview of Recommendation Systems (Optional Material)","#overview-of-recommendation-systems-optional-material","Reinforcement Learning Wrap-up","#reinforcement-learning-wrap-up","A Concrete Example: Q-Learning with Gymnasium","#a-concrete-example-q-learning-with-gymnasium","An Introduction to Markov Decision Process","#an-introduction-to-markov-decision-process","Available RL Tools","#available-rl-tools","Overview","#overview","Overview of Reinforcement Learning (Optional Material)","#overview-of-reinforcement-learning-optional-material","Recommended Reading for Image Generation","#recommended-reading-for-image-generation","Mini-DALL·E: A Lightweight Alternative","#mini-dalle-a-lightweight-alternative","Image Generation Using Stable Diffusion and PyTorch","#image-generation-using-stable-diffusion-and-pytorch","Part IV - Overviews of Image Generation, Reinforcement Learning, and Recommendation Systems","#part-iv---overviews-of-image-generation-reinforcement-learning-and-recommendation-systems","Deep Learning Natural Language Processing Wrap-up","#deep-learning-natural-language-processing-wrap-up","Comparing Sentences for Similarity Using Transformer Models","#comparing-sentences-for-similarity-using-transformer-models","Hugging Face and the Transformers Library","#hugging-face-and-the-transformers-library","Natural Language Processing Using Deep Learning","#natural-language-processing-using-deep-learning","Using PyTorch for Building a Cancer Prediction Model","#using-pytorch-for-building-a-cancer-prediction-model","The Basics of Deep Learning","#the-basics-of-deep-learning","Part 2 - Deep Learning","#part-2---deep-learning","EDA and Feature Engineering Wrap-up","#eda-and-feature-engineering-wrap-up","Feature Engineering","#feature-engineering","Exploratory Data Analysis","#exploratory-data-analysis","Exploratory Data Analysis and Feature Engineering","#exploratory-data-analysis-and-feature-engineering","Regression and Clustering Wrap-up","#regression-and-clustering-wrap-up","Clustering: Discovering Groups in Data","#clustering-discovering-groups-in-data","Regression: Predicting Housing Prices","#regression-predicting-housing-prices","Regression and Clustering","#regression-and-clustering","Classic Machine Learning Wrap-up","#classic-machine-learning-wrap-up","Classification Models using Scikit-learn","#classification-models-using-scikit-learn","Example Material","#example-material","“Classic” Machine Learning","#classic-machine-learning","Part 1 - Machine Learning","#part-1---machine-learning","Managing Python with uv","#managing-python-with-uv","Python Development Environment","#python-development-environment","Acknowledgements","#acknowledgements","Book Cover","#book-cover","Using the Example Code","#using-the-example-code","About the Author","#about-the-author","Preface","#preface","Cover Material, Copyright, and License","#cover-material-copyright-and-license",{"_12":13,"_37":38,"_35":36,"_440":441,"_442":443,"_444":445,"_39":40,"_446":447,"_448":449},"parts",[388],"toc",[303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387],"isUnavailableOnline",false,"unavailableMessage",{"_22":23,"_12":293,"_25":-5,"_26":294,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":291,"_25":-5,"_26":292,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":289,"_25":-5,"_26":290,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":287,"_25":-5,"_26":288,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":285,"_25":-5,"_26":286,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":283,"_25":-5,"_26":284,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":281,"_25":-5,"_26":282,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":279,"_25":-5,"_26":280,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":277,"_25":-5,"_26":278,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":275,"_25":-5,"_26":276,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":273,"_25":-5,"_26":274,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":271,"_25":-5,"_26":272,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":269,"_25":-5,"_26":270,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":267,"_25":-5,"_26":268,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":265,"_25":-5,"_26":266,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":263,"_25":-5,"_26":264,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":261,"_25":-5,"_26":262,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":259,"_25":-5,"_26":260,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":257,"_25":-5,"_26":258,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":255,"_25":-5,"_26":256,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":253,"_25":-5,"_26":254,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":251,"_25":-5,"_26":252,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":249,"_25":-5,"_26":250,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":247,"_25":-5,"_26":248,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":245,"_25":-5,"_26":246,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":243,"_25":-5,"_26":244,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":241,"_25":-5,"_26":242,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":239,"_25":-5,"_26":240,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":237,"_25":-5,"_26":238,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":24,"_25":-5,"_26":27,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":235,"_25":-5,"_26":236,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":233,"_25":-5,"_26":234,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":231,"_25":-5,"_26":232,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":229,"_25":-5,"_26":230,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":227,"_25":-5,"_26":228,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":225,"_25":-5,"_26":226,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":223,"_25":-5,"_26":224,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":221,"_25":-5,"_26":222,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":219,"_25":-5,"_26":220,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":217,"_25":-5,"_26":218,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":215,"_25":-5,"_26":216,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":213,"_25":-5,"_26":214,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":211,"_25":-5,"_26":212,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":209,"_25":-5,"_26":210,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":207,"_25":-5,"_26":208,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":205,"_25":-5,"_26":206,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":203,"_25":-5,"_26":204,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":201,"_25":-5,"_26":202,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":182,"_25":-5,"_26":200,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":198,"_25":-5,"_26":199,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":196,"_25":-5,"_26":197,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":194,"_25":-5,"_26":195,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":192,"_25":-5,"_26":193,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":190,"_25":-5,"_26":191,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":188,"_25":-5,"_26":189,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":186,"_25":-5,"_26":187,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":184,"_25":-5,"_26":185,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":182,"_25":-5,"_26":183,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":159,"_25":-5,"_26":181,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":179,"_25":-5,"_26":180,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":177,"_25":-5,"_26":178,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":175,"_25":-5,"_26":176,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":173,"_25":-5,"_26":174,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":171,"_25":-5,"_26":172,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":169,"_25":-5,"_26":170,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":167,"_25":-5,"_26":168,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":165,"_25":-5,"_26":166,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":163,"_25":-5,"_26":164,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":161,"_25":-5,"_26":162,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":159,"_25":-5,"_26":160,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":157,"_25":-5,"_26":158,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":155,"_25":-5,"_26":156,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":153,"_25":-5,"_26":154,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":151,"_25":-5,"_26":152,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":149,"_25":-5,"_26":150,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":147,"_25":-5,"_26":148,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":145,"_25":-5,"_26":146,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":143,"_25":-5,"_26":144,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":141,"_25":-5,"_26":142,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":139,"_25":-5,"_26":140,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":137,"_25":-5,"_26":138,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":135,"_25":-5,"_26":136,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":133,"_25":-5,"_26":134,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":23,"_12":131,"_25":-5,"_26":132,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_22":128,"_12":129,"_25":-5,"_26":130,"_28":29,"_30":-5,"_31":-5,"_32":-5},{"_12":13,"_25":389,"_28":390,"_391":392},"",0,"chapters",[393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415],{"_22":23,"_12":293,"_26":294,"_28":29,"_416":439,"_32":-5},{"_22":23,"_12":291,"_26":292,"_28":29,"_416":438,"_32":-5},{"_22":23,"_12":281,"_26":282,"_28":29,"_416":437,"_32":-5},{"_22":23,"_12":277,"_26":278,"_28":29,"_416":436,"_32":-5},{"_22":23,"_12":275,"_26":276,"_28":29,"_416":435,"_32":-5},{"_22":23,"_12":267,"_26":268,"_28":29,"_416":434,"_32":-5},{"_22":23,"_12":259,"_26":260,"_28":29,"_416":433,"_32":-5},{"_22":23,"_12":251,"_26":252,"_28":29,"_416":432,"_32":-5},{"_22":23,"_12":249,"_26":250,"_28":29,"_416":431,"_32":-5},{"_22":23,"_12":245,"_26":246,"_28":29,"_416":430,"_32":-5},{"_22":23,"_12":237,"_26":238,"_28":29,"_416":429,"_32":-5},{"_22":23,"_12":24,"_26":27,"_28":29,"_416":428,"_32":-5},{"_22":23,"_12":229,"_26":230,"_28":29,"_416":427,"_32":-5},{"_22":23,"_12":217,"_26":218,"_28":29,"_416":426,"_32":-5},{"_22":23,"_12":211,"_26":212,"_28":29,"_416":425,"_32":-5},{"_22":23,"_12":209,"_26":210,"_28":29,"_416":424,"_32":-5},{"_22":23,"_12":198,"_26":199,"_28":29,"_416":423,"_32":-5},{"_22":23,"_12":179,"_26":180,"_28":29,"_416":422,"_32":-5},{"_22":23,"_12":157,"_26":158,"_28":29,"_416":421,"_32":-5},{"_22":23,"_12":155,"_26":156,"_28":29,"_416":420,"_32":-5},{"_22":23,"_12":139,"_26":140,"_28":29,"_416":419,"_32":-5},{"_22":23,"_12":137,"_26":138,"_28":29,"_416":418,"_32":-5},{"_22":23,"_12":131,"_26":132,"_28":29,"_416":417,"_32":-5},"subsections",[387],[384,385],[],[375,376,377,378,379,380,381],[],[363,364,365,366,367,368,369,370,371,372],[353,354,355,356,357,358,359,360,361],[347,348,349,350,351],[],[343,344],[337,338,339,340,341],[333,334,335],[],[328,329,330],[326],[],[321,322,323],[317,318,319],[313,314,315],[],[310],[305,306,307,308],[],"coverFeatured",{"_459":461},"coverFeatured2x",{"_459":460},"authors",[450],"samplePdfUrl","https://s3.amazonaws.com/samples.leanpub.com/pythonai-sample.pdf?x-amz-checksum-mode=ENABLED&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0XBXHW3Q9GGV69BT7E82%2F20260505%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260505T154042Z&X-Amz-Expires=518400&X-Amz-SignedHeaders=host&X-Amz-Signature=7d3a0a4427837a1f1bcc5d70f741ca9169b24f57b4b80c28382773cb33b5ad72","sampleEpubUrl","https://s3.amazonaws.com/samples.leanpub.com/pythonai-sample.epub?x-amz-checksum-mode=ENABLED&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0XBXHW3Q9GGV69BT7E82%2F20260505%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260505T154042Z&X-Amz-Expires=518400&X-Amz-SignedHeaders=host&X-Amz-Signature=cc4270292281fdec2217a4f8471faf2782468040e0e432e188e9696d08e49d05",{"_451":452,"_453":454,"_455":456},"id","Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6UHJvZHVjdEF1dGhvci81NTgxOA","name","Mark Watson","user",{"_457":458},"username","markwatson","url","https://d2sofvawe08yqg.cloudfront.net/pythonai/s_featured2x?1671052252&1671052252","https://d2sofvawe08yqg.cloudfront.net/pythonai/s_featured?1671052252&1671052252","shoppingCartId","shoppingCart",{"_451":491,"_492":390,"_493":482,"_494":389,"_495":496,"_497":498,"_499":482},"locale","en","readerTheme","light","constants",{"_489":490},"publishers",{"_485":486,"_487":488},"learnerOrganizations",[],"needsLocaleCookieSync","announcements",{"_478":479,"_480":-5},"announcement_store",{"_481":482,"_483":484},"announcement_authors","flag",true,"text","

Kick off your book project in 3 hours! Live workshop on Zoom. You’ll leave with a real book project, progress on your first chapter, and a clear plan to keep going. Saturday, May 16, 2026. Learn more…

\n","companies",[],"universities",[],"paypalClientId","AX53Cwz8jR6rVVzzZiZNWWXl0dGagi-GZrjG-S3Ua14E5nrKMn7DxDQ730O4R8LusLPp49ALn6aRVaEy","no-cart","count","free","updatedAt","paymentGateway","STRIPE","cartItems",[],"loading"]

Part IV - Overviews of Image Generation, Reinforcement Learning, and Recommendation Systems

Table of Contents

Cover Material, Copyright, and License

Preface

About the Author

Using the Example Code

Book Cover

Acknowledgements

Python Development Environment

Managing Python with uv

Setting Up a Project

Managing Python Versions

Code Formatting (Optional)

Part 1 - Machine Learning

“Classic” Machine Learning

Example Material

Classification Models using Scikit-learn

Classic Machine Learning Wrap-up

Regression and Clustering

Regression: Predicting Housing Prices

Linear Regression

Polynomial Regression

Clustering: Discovering Groups in Data

Choosing K with the Silhouette Score

Comparing Clusters to True Labels

Regression and Clustering Wrap-up

Exploratory Data Analysis and Feature Engineering

Exploratory Data Analysis

Loading and Inspecting the Data

Checking for Missing Values

Correlation Analysis

Outlier Detection

Feature Engineering

Creating New Features

Encoding Categorical Features

Handling Missing Data

Feature Scaling

Measuring the Impact

EDA and Feature Engineering Wrap-up

Part 2 - Deep Learning

The Basics of Deep Learning

Using PyTorch for Building a Cancer Prediction Model

Why PyTorch?

Loading and Preparing the Data

Defining the Model

The Training Loop

Running the Example

Natural Language Processing Using Deep Learning

Hugging Face and the Transformers Library

Summarizing Text Using a Pre-trained Model on Your Laptop

Zero Shot Classification Using a Local Model

Comparing Sentences for Similarity Using Transformer Models

Deep Learning Natural Language Processing Wrap-up

Part IV - Overviews of Image Generation, Reinforcement Learning, and Recommendation Systems

Overview of Image Generation

Image Generation Using Stable Diffusion and PyTorch

Understanding the Diffusion Process

Mini-DALL·E: A Lightweight Alternative

Recommended Reading for Image Generation

Overview of Reinforcement Learning (Optional Material)

Overview

Available RL Tools

An Introduction to Markov Decision Process

Solving MDPs with pymdptoolbox

A Concrete Example: Q-Learning with Gymnasium

Other Gymnasium Environments Worth Exploring

Reinforcement Learning Wrap-up

Overview of Recommendation Systems (Optional Material)

TensorFlow Recommenders

Recommendation Systems Wrap-up

Part 3 - Large Language Models

Introduction to Transformers and Large Language Models

The Transformer Architecture

Self-Attention: The Core Innovation

Multi-Head Attention

Positional Encoding

Layer Structure

Encoder-Only, Decoder-Only, and Encoder-Decoder Models

Tokenization

Why Not Just Use Characters or Words?