Portability Library Supporting OpenAI, Hugging Face, and Self Hosted Models

I usually avoid extra abstraction layers in software I design and write. For the use cases for this book’s examples and my own projects I wrote the library llmlib that offers a common API for OpenAI APIs and local local hosted HuggingFace models. My work flow for new LLM applications now usually starts with interactive development using either a local Hugging Face model running on my laptop or OpenAI APIs. I can easily work on my laptop. Later I have the option of switching between self-hosted LLM models and OpenAI APIs without needing to convert my application code to a new LLM backend.

The llmlib is a small library that I wrote for my own use and for use in this book.

For more documentation for llmlib refer to https://github.com/mark-watson/llmlib and https://pypi.org/project/llmlib/. If you like my approach using llmlib then I suggest you fork the GitHub repository and modify it for your own use. You can also install it as a package since I maintain the library on PyPi:

1 pip install llmlib

We will not look at the backend implementation code for using OpenAI APIs and local Hugging Face self hosted models. We will review the public APT in the Python base class:

 1 class BaseModelWrapper():
 2     def __init__(self, embeddings_dir="./db_embeddings"):
 3         llm = None
 4         embeddings = None
 5         embeddings_dir = embeddings_dir
 6     # complete text:
 7     def get_completion(self, prompt, max_tokens=64):
 8         pass
 9     # create local embeddings:
10     def create_local_embeddings_files_in_dir(self, path):
11         " path is a directory "
12         pass
13     # query local embeddings:
14     def query_local_embeddings(self, query, n=10):
15         pass

This is a simple API but is adequate for the purposes of examples developed in the next chapter. For more examples using LLMs, LangChain, and LlamaIndex please see my book LangChain and LlamaIndex Projects Lab Book: Hooking Large Language Models Up to the Real World that you can read free online or purchase.

The rest of this chapter can be considered optional material. We will take a quick look at the implementation of the LLM backends for llmlib.

There are four example programs in the llmlib GitHub repository that you can experiment with. Here we look at two of them: the first uses OpenAI APIs for the backend and the second uses a local Hugging Face model back end.

test_completion_openai.py

1 from llmlib.openai import OpenAiWrapper
2 
3 llm = OpenAiWrapper()
4 
5 gen_text = llm.get_completion("Once upon a time", max_tokens=64)
6 print(gen_text)%                                                                          

test_local_index_hf.py

 1 from llmlib.huggingface import HuggingFaceAiWrapper
 2 
 3 llm = HuggingFaceAiWrapper()
 4 
 5 llm.create_local_embeddings_files_in_dir("./data/")
 6 
 7 a = llm.query_local_embeddings("definition of sports")
 8 print(a)
 9 
10 a = llm.query_local_embeddings("activities in sports")
11 print(a)