Integrating OpenAI
OpenAI is a pioneer in LLM development, hosting models such as GPT-4o and GPT-4o-mini. These models are accessible via a standardized JSON API.
In this chapter, we implement an OpenAI client in Scala 3 using the requests and ujson libraries, demonstrating how to construct payload objects and authenticate requests using headers.
All code is in source-code/openai-client.
The Chat Completions Format
The previous chapter covered how an LLM works: a transformer trained to predict the next token, generating text autoregressively. This chapter is about the API design that has become the industry standard for talking to such a model, the chat completions format that OpenAI introduced and that many other providers, including Ollama in the next chapter, now imitate.
The central idea is that a request is not a single prompt but a list of messages, each tagged with a role. Three roles carry the conversation:
- system: an opening instruction that sets the model’s persona, tone, and rules for the whole conversation. It is how you steer behavior (“You are a terse assistant that answers only in French”).
- user: a turn written by the human.
- assistant: a turn previously generated by the model.
This structure is what turns a next-token predictor into a chatbot. Because the API is stateless, as noted in the last chapter, the model remembers nothing on its own. To hold a multi-turn conversation you resend the entire message list every call, alternating user and assistant turns, so the model can read the history and continue it. Our example sends a single user message, the simplest case, but the same array grows into a full dialogue.
The OpenAI REST Client
In openai-client/OpenAIClient.scala, we target the /v1/chat/completions endpoint. The API key is read from the OPENAI_API_KEY environment variable and passed in the HTTP request as a Bearer token in the Authorization header:
1 //> using scala 3.6.4
2 //> using dep com.lihaoyi::requests:0.9.3
3 //> using dep com.lihaoyi::ujson:4.4.3
4
5 package openai_client
6
7 import java.io.IOException
8
9 object OpenAIClient:
10 private val DEFAULT_MODEL = "gpt-4o-mini"
11 private val API_URL = "https://api.openai.com/v1/chat/completions"
12
13 def getCompletion(prompt: String, model: String = DEFAULT_MODEL): String =
14 val apiKey = getApiKey()
15
16 val payload = ujson.Obj(
17 "model" -> model,
18 "messages" -> ujson.Arr(
19 ujson.Obj(
20 "role" -> "user",
21 "content" -> prompt
22 )
23 )
24 ).render()
25
26 val response = requests.post(
27 url = API_URL,
28 headers = Map(
29 "Content-Type" -> "application/json",
30 "Authorization" -> s"Bearer $apiKey"
31 ),
32 data = payload
33 )
34
35 if response.statusCode != 200 then
36 throw IOException(s"OpenAI request failed: ${response.text()}")
37
38 parseResponse(response.text())
39
40 private def getApiKey(): String =
41 val key = System.getenv("OPENAI_API_KEY")
42 if key == null || key.trim.isEmpty then
43 throw IOException("OPENAI_API_KEY environment variable is not set.")
44 key.trim
45
46 private def parseResponse(jsonStr: String): String =
47 val data = ujson.read(jsonStr)
48 data("choices")(0)("message")("content").str
The payload carries the model name and the messages array, here a single message with role of user. Authentication differs from Gemini in a small but standard way: instead of a key in the URL, OpenAI expects an Authorization: Bearer <key> header, the same scheme used across most modern web APIs. Passing the key in a header rather than the URL is also safer, since URLs often end up in server logs. The response comes back as a list of choices, and we read the first choice’s message.content field, which parallels Gemini’s candidates list from the last chapter. The two APIs differ only in their field names; the underlying request-response pattern is identical, which is why the same requests and ujson stack serves both.
APIs like this bill per token processed, counting both the tokens you send and the tokens the model generates. That gives a practical reason to keep prompts and histories tight: a long conversation resent on every turn costs more each time, and it also consumes the model’s finite context window.
Beyond Chat: Embeddings
The same OpenAI API offers a second capability worth knowing even though our client does not call it: embeddings. An embedding endpoint turns a piece of text into a vector of numbers, a point in a high-dimensional space, positioned so that texts with similar meaning land near each other. This is the engine of semantic search: to find documents relevant to a question, you embed the question and the documents and compare vectors by distance, matching on meaning rather than on shared keywords.
Embeddings are the missing half of a Retrieval-Augmented Generation system. The Gemini chapter grounded answers with a live web search; a production RAG pipeline more often embeds a private document collection, retrieves the passages nearest the user’s question, and pastes them into the prompt so the model answers from your data. The chat endpoint we build here generates the answer, and an embeddings endpoint would supply the retrieval, the two together forming the standard architecture for question answering over private knowledge.
Running the OpenAI Demo
The demo program in openai-client/Main.scala uses the OpenAI client to translate a sentence from English to French:
1 package openai_client
2
3 @main def openAIDemo(): Unit =
4 println("=" * 50)
5 println("OpenAI LLM Client Demo")
6 println("=" * 50)
7
8 val prompt = "Translate the following English text to French: 'Hello, how are you?'"
9 println(s"\nPrompt: $prompt\n")
10 try
11 val completion = OpenAIClient.getCompletion(prompt)
12 println(s"Response:\n$completion")
13 catch
14 case e: Exception => println(s"Error: ${e.getMessage}")
Export your API key and run the example:
1 export OPENAI_API_KEY="your-openai-api-key"
2 scala-cli run .
This compiles and runs the project, printing the response from OpenAI:
1 ==================================================
2 OpenAI LLM Client Demo
3 ==================================================
4
5 Prompt: Translate the following English text to French: 'Hello, how are you?'
6
7 Response:
8 Bonjour, comment ça va ?
The translation demonstrates a general truth about instruction-tuned models: we never wrote translation code, defined no grammar, and loaded no dictionary. We described the task in plain English and the model carried it out, because translation is one of countless behaviors it absorbed during training. This is the shift the LLM chapters mark. Where the earlier chapters built each capability from algorithms we wrote, here we describe what we want and let a trained model supply the how.