Preface

The goal of this short book is to promote a “humans-first” approach to designing and building AI systems. Here we address both privacy concerns and look at strategies for not getting locked into any single Large Language Model (LLM) API vendor. Tactically we do this using a Python wrapper library that lets you, dear reader, replace API service backends from Open AI and Hugging Face as well as running open models locally in your applications.

We will use the llmlib library that I wrote for this backend LLM selection flexibility. You can find resources here: https://github.com/mark-watson/llmlib and https://pypi.org/project/llmlib/.

We don’t cover classic AI techniques (symbolic AI, simple machine learning) in this book because I feel that the threats to our collective privacy and control over our data and systems is mostly a threat from LLM based systems.

We take a dive into both general architectures for safe, human-first AI as well as concrete examples of building applications with LLMs with both prompt engineering and fine-tuning. While I favor using self-hosted LLMs for the book technical examples, most can also be used with OpenAI and Hugging Face APIs.

I have been working in the field of artificial intelligence since 1982. For forty years my main career goal was to keep current with AI and apply techniques like expert systems, neural networks, symbolic natural language processing (NLP), deep learning, and chained large language models (LLMs) to solve practical problems.

In recent years my personal interests have drifted to include the impact of AI on society. This book is intended to be “food for thought” for you, dear reader, as you combine work using AI technology with your desires to:

• Design and build “human first” systems.
• Work on projects that have a positive impact on society.
• Respect the rights of users of the systems you build in the areas of privacy, data security, and reduction of false information.
• Develop a personal technology stack that lets you use AI techniques, including chained LLMs on both your own hardware or secure compute leased in the cloud.

Part I of this book is largely non-technical. I am trying to set a mental framework for us to discuss, agree, and disagree on what the tradeoffs are for making maximum use of AI vs. maintaining human values and fairness in the systems we help build and the platforms we chose to use in our personal lives and for work.

Part II of this book contains the fun technical material. We will revisit the material in my last book LangChain and LlamaIndex Projects Lab Book: Hooking Large Language Models Up to the Real Worldthat can be read free online. In my last book I used OpenAI’s GPT APIs and other cloud services to process private data using public platforms and services. In Part II we will reimplement many of these ideas using smaller open LLM models running both on our own computers and secure leased cloud compute. My desire is that the technical material in my last book and Part II of this book will be complementary.

Part III of this book questions what the future may hold for us, and provide ideas for dealing with privacy, exponentially accelerating efficiency driven by AI in general and specifically chained LLMs working with external systems and tooling to solve practical problems. Part III also contains my suggestions for AI-safety respecting architectures for use in education and small and medium size businesses.

Technology is changing rapidly. I had to release four major updates to my last book in the first four months the book was published as an eBook. I expect the material in this book to also evolve as technologies change.

The GitHub repository for examples in this book is https://github.com/mark-watson/safe-for-humans-AI-software. Please note that I usually update the code in the examples repository fairly frequently for library version updates, etc.

In order to run many of the examples in this book, you will need a GPU with 20 to 48 GB of memory. I used a leased server on Lambda Labs with an Nvidia A6000 (48 GB) for development. This server costs $0.80/hour to run (charged by the minute). I tend to develop in two-hour sprints so my dev costs for a sprint is $1.60 and it takes me about two minutes to set up a server.

I also use Google Colab for experimenting with deep learning and Large Language Models (LLMs).

Appendix A contains setup advice for using Colab and Appendix B contains setup advice for using Lambda Labs.

We require two types of Large Language Models (LLMs):

• Embeddings: given any amount of text, calculate vector embeddings.
• Text completion with instruction fine tuning.

We use the FastChat platform that provides both an interactive command line chat interface as well as an API service that is compatible with the OpenAI API interfaces. We use stubs so that in development you can quickly switch between these options.

We also use the LangChain library and the LlamaIndex library.

We will use this technology stack to implement the book examples that are intended as a jumping off point for you, dear reader, to develop your own AI tools for both work and for fun.

When I write code I always have GitHub CoPilot active when using both Emacs and VSCode. I often use Google Colab for writing Python machine learning, deep learning, and LLM code. I run Colab with Google’s Colab AI enabled that does code completions and suggestions. As a result of using these tools, a some of code that I now write is auto-generated code that I may modify and then test.

Similarly when I write book manuscripts, text in Google Docs, and with the applications in Microsoft 365, I also use some auto-generated text created by LLMs. While writing a book manuscript I usually just write markdown text in iA Writer and that is all my own work. I also sometimes write a few paragraphs in iA Writer and then paste this text into one of ChatGPT, Microsoft Bing Chat, or Google Bard using a prompt like:

If I like the generated text, I copy it back into iA Writer and heavily edit it to match my writing style and to remove/add material as I see fit.

All of my recent eBooks are released under a Creative Commons License. I will start by extracting a few hundred pages of text from a few of them for use in implementing two examples:

• Prompt Engineering: I will break the input text into many hundreds of text-chunks and use an embedding LLM to calculate vector embeddings for each chunk. I will then build an application that uses the embedding search index and a text completion LLM to implement an interactive chat application for my book material.
• Fine-Tuning: I will use the input text from my books to fine-tune an existing instruction tuned LLM and use this modified model in a chat application for my book material.

I have written over 20 books, I have over 50 US patents, and I have worked at interesting companies like Google, Capital One, SAIC, and many others. You can find links for reading most of my recent books and about 2000 blog articles I have written over the last 24 years free on my web site https://markwatson.com. If I had to summarize my career the short take would be that I have had a lot of fun and enjoyed my work. I hope that what you learn here will be both enjoyable and help you in your work.

If you would like to support my work please consider purchasing my books on Leanpub and star my git repositories that you find useful on GitHub. You can also interact with me on social media on Mastodon and Twitter. I am also available as a consultant.

I used Midjourney to create the cover art.

This picture shows me and my wife Carol who helps me with book production and editing.

I would also like to thank the following readers who reported errors or typos in this book: none so far.