Using the OpenAI Large Language Model APIs in Java
Note: May 15, 2024: this chapter is about 90% complete
Large Language Models (LLMs) signify a significant leap forward in the progression of artificial intelligence, with a pronounced impact on the field of natural language processing (NLP), data transformation, translation, and serve as a source of real world knowledge in AI applications. They are trained on vast corpora of text data (literally most published books and most of the web), learning to predict subsequent words in a sequence, which imbues them with the ability to generate human-like text, comprehend the semantics of language, and perform a variety of language-related tasks. The architecture of these models, typically based on deep learning paradigms such as Transformer, empowers them to encapsulate intricate patterns and relationships within language. These models are trained utilizing substantial computational resources.
The utility of LLMs extends across a broad spectrum of applications including but not limited to text generation, translation, summarization, question answering, and sentiment analysis. Their ability to understand and process natural language makes them indispensable tools in modern AI-driven solutions. However, with great power comes great responsibility. The deployment of LLMs raises imperative considerations regarding ethics, bias, and the potential for misuse. Moreover, the black-box nature of these models presents challenges in interpretability and control, which are active areas of research in the quest to make LLMs more understandable and safe. The advent of LLMs has undeniably propelled the field of NLP to new heights, yet the journey towards fully responsible and transparent utilization of these powerful models is an ongoing endeavor.
In the development of practical AI systems, LLMs like those provided by OpenAI, Anthropic, and Hugging Face have emerged as pivotal tools for numerous applications including natural language processing, generation, and understanding. These models, powered by deep learning architectures, encapsulate a wealth of knowledge and computational capabilities. Here we look at the basics for getting you, dear reader, started using the OpenAI APIs for text completion tasks in Java code. In the next chapter we do the same ex pet we will run local LLMs on our laptops using the Ollama platform.
Java Library to Use OpenAI’s APIs
The library code defined in the directory Java-AI-Book-Code/openai-llm-client is designed to interact with the OpenAI API to accept a prompt string and get a text completion. Here’s a breakdown of what each part of the code does:
The getCompletion method performs the following steps:
- Initialization: This method takes a prompt as input and retrieves the OpenAI API key from the environment variables.
- JSON Object Creation: Constructs a JSON object to define the user’s role and the content (the prompt). This is added to a JSON array, which is then included in another JSON object along with the model name (here we are using gpt-3.5-turbo but you can also try gpt-4 that is more expensive but more capable.).
- API Request Setup: Constructs a URI for the OpenAI API endpoint and sets up a URL connection. It configures the connection to send data (output) and sets the request headers for content type (JSON) and authorization (using the retrieved API key).
- Sending the Request: Converts the JSON object to bytes and sends it through the connection’s output stream.
- Response Handling: Reads the response from the API using a BufferedReader. The response is built into a string using a StringBuilder.
- Parsing the Response: Converts the response string back into a JSON object, extracts the relevant part of the JSON that contains the API’s completion result (the translated text), and returns this result.
- Error Handling and Cleanup: The connection is explicitly disconnected after the response is read.
1 package com.markwatson.openai;
2
3 import java.io.BufferedReader;
4 import java.io.InputStreamReader;
5 import java.io.OutputStream;
6 import java.net.HttpURLConnection;
7 import java.net.URI;
8 import java.net.URL;
9 import java.net.URLConnection;
10
11 import org.json.JSONArray;
12 import org.json.JSONObject;
13
14 public class OpenAICompletions {
15
16 public static void main(String[] args) throws Exception {
17 String prompt = "Translate the following English text to French: 'Hello, how are you?'";
18 String completion = getCompletion(prompt);
19 System.out.println("completion: " + completion);
20 }
21
22 public static String getCompletion(String prompt) throws Exception {
23 System.out.println("prompt: " + prompt);
24 String apiKey = System.getenv("OPENAI_API_KEY");
25 String model = "gpt-3.5-turbo"; // Replace with the desired model
26
27 // New JSON message format
28 JSONObject message = new JSONObject();
29 message.put("role", "user");
30 message.put("content", prompt);
31
32 JSONArray messages = new JSONArray();
33 messages.put(message);
34 //System.out.println("messages: " + messages.toString());
35 JSONObject jsonBody = new JSONObject();
36 jsonBody.put("messages", messages);
37 jsonBody.put("model", model);
38 URI uri = new URI("https://api.openai.com/v1/chat/completions");
39 URL url = uri.toURL();
40 //System.out.println("jsonBody: " + jsonBody);
41 URLConnection connection = url.openConnection();
42 connection.setDoOutput(true);
43 connection.setRequestProperty("Content-Type", "application/json");
44 connection.setRequestProperty("Authorization", "Bearer " + apiKey);
45 // Send the JSON payload
46 try (OutputStream os = connection.getOutputStream()) {
47 byte[] input = jsonBody.toString().getBytes("utf-8");
48 os.write(input, 0, input.length);
49 }
50
51 StringBuilder response;
52 // Read the response from the server
53 try (BufferedReader br = new BufferedReader(
54 new InputStreamReader(connection.getInputStream(), "utf-8"))) {
55 response = new StringBuilder();
56 String responseLine;
57 while ((responseLine = br.readLine()) != null) {
58 response.append(responseLine.trim());
59 }
60 System.out.println(response.toString());
61 }
62
63 ((HttpURLConnection) connection).disconnect();
64 JSONObject jsonObject = new JSONObject(response.toString());
65 JSONArray choices = jsonObject.getJSONArray("choices");
66 JSONObject messageObject = choices.getJSONObject(0).getJSONObject("message");
67 String content = messageObject.getString("content");
68 //System.out.println("content: " + content);
69 return content;
70 }
71
72 /***
73 * Utilities for using the OpenAI API
74 */
75
76 // read the contents of a file path into a Java string
77 public static String readFileToString(String filePath) throws IOException {
78 Path path = Paths.get(filePath);
79 return new String(Files.readAllBytes(path));
80 }
81
82 public static String replaceSubstring(String originalString, String substringToReplace, String replacementString) {
83 return originalString.replace(substringToReplace, replacementString);
84 }
85 public static String promptVar(String prompt0, String varName, String varValue) {
86 String prompt = replaceSubstring(prompt0, varName, varValue);
87 return replaceSubstring(prompt, varName, varValue);
88 }
89 }
In the next section we write a unit test for this Java class to demonstrate text completion.
Example Applications
There is a unit test provided with this library that shows how to call the completion API:
1 String r =
2 OpenAICompletions.getCompletion("Translate the following English text to French: 'Hello, how are you?'");
3 System.out.println("completion: " + r);
Sample output is:
1 $ make
2 mvn test -q # run test in quiet mode
3 prompt: Translate the following English text to French: 'Hello, how are you?'
4 completion: Bonjour, comment vas-tu ?
For reference, the JSON response object returned from the OpenAI completion API looks like this:
1 {"id": "chatcmpl-7LbgN6PJxHAfycOuHmGkw8nbpQMm1","object": "chat.completion","created": 1714936767,"model": "gpt-3.5-turbo-0125","choices": [{"index": 0,"message": {"role": "assistant","content": "Bonjour, comment vas-tu ?"},"logprobs": null,"finish_reason": "stop"}],"usage": {"prompt_tokens": 22,"completion_tokens": 7,"total_tokens": 29},"system_fingerprint": "fp_b410720239"}
Extraction of Facts and Relationships from Text Data
Traditional methods for extracting email addresses, names, addresses, etc. from text included the use of hand-crafted regular expressions and custom software. LLMs are text processing engines with knowledge of grammar, sentence structure, and some real world embedded knowledge. Using LLMs can reduce the development time of information extraction systems.
The template we will use if in the file Java-AI-Book-Code /prompts/two-shot-2-var.txt:
1 Given the two examples below, extract the names, addresses, and email addresses of individuals mentioned later as Process Text. Format the extracted information in JSON, with keys for "name", "address", and "email". If any information is missing, use "null" for that field. Be very concise in your output by providing only the output JSON.
2
3 Example 1:
4 Text: "John Doe lives at 1234 Maple Street, Springfield. His email is johndoe@example.com."
5 Output:
6 {
7 "name": "John Doe",
8 "address": "1234 Maple Street, Springfield",
9 "email": "johndoe@example.com"
10 }
11
12 Example 2:
13 Text: "Jane Smith has recently moved to 5678 Oak Avenue, Anytown. She hasn't updated her email yet."
14 Output:
15 {
16 "name": "Jane Smith",
17 "address": "5678 Oak Avenue, Anytown",
18 "email": null
19 }
20
21 Process Text: "{input_text}"
22 Output:
The example code for this section is in a Java Unit Test method testTwoShotTemplate():
1 String input_text = "Mark Johnson enjoys living in Berkeley California at 102 Dunston Street and use mjess@foobar.com for contacting him.";
2
3 String prompt0 = OpenAICompletions.readFileToString("../prompts/two-shot-2-var.txt");
4 System.out.println("prompt0: " + prompt0);
5 String prompt = OpenAICompletions.promptVar(prompt0, "{input_text}", input_text);
6 System.out.println("prompt: " + prompt);
7 String r =
8 OpenAICompletions.getCompletion(prompt);
9 System.out.println("two shot extraction completion:\n" + r);
The output looks like:
1 two shot extraction completion:
2 {
3 "name": "Mark Johnson",
4 "address": "102 Dunston Street, Berkeley California",
5 "email": "mjess@foobar.com"
6 }
Using LLMs to Summarize Text
LLMs bring a new level of ability to text summarization tasks. With their ability to process massive amounts of information and “understand” natural language, they’re able to capture the essence of lengthy documents and distill them into concise summaries. Two main types of summarization dominate with LLMs: extractive and abstractive. Extractive summarization pinpoints the most important sentences within the original text, while abstractive summarization requires the LLM to paraphrase or generate new text to represent the core ideas. If you are interested in extractive summarization there is a chapter on this topic in my Common Lisp AI book (link to read online).
Here we use the prompt template file Java-AI-Book-Code/prompts/summarization_prompt.txt:
1 Summarize the following text: "{input_text}"
2 Output:
The example code is in the Java Unit Test testSummarization():
1 String input_text = "Jupiter is the fifth planet from the Sun and the largest in the Solar System. It is a gas giant with a mass one-thousandth that of the Sun, but two-and-a-half times that of all the other planets in the Solar System combined. Jupiter is one of the brightest objects visible to the naked eye in the night sky, and has been known to ancient civilizations since before recorded history. It is named after the Roman god Jupiter.[19] When viewed from Earth, Jupiter can be bright enough for its reflected light to cast visible shadows,[ and is on average the third-brightest natural object in the night sky after the Moon and Venus.";
2 String prompt0 = OpenAICompletions.readFileToString("../prompts/summarization_prompt.txt");
3 System.out.println("prompt0: " + prompt0);
4 String prompt = OpenAICompletions.promptVar(prompt0, "{input_text}", input_text);
5 System.out.println("prompt: " + prompt);
6 String r =
7 OpenAICompletions.getCompletion(prompt);
8 System.out.println("summarization completion:\n\n" + r);
The output is:
1 summarization completion:
2
3 Jupiter is the largest planet in the Solar System and the fifth from the Sun. It is a gas giant with a mass one-thousandth that of the Sun. It is visible to the naked eye and has been known since ancient times. Jupiter is named after the Roman god Jupiter and is the third-brightest natural object in the night sky.