More Hosted Models: Anthropic, Mistral, Groq, Moonshot, Perplexity, Hugging Face
The book so far covers Gemini, OpenAI, and local Ollama. Real work spreads wider: Claude for long docs, Mistral for cheap EU hosting, Groq for fast open weights, Moonshot Kimi for long Chinese context, Perplexity for search backed answers, Hugging Face for niche classifiers. This chapter adds all six as thin REST clients in one file, in the same shape as the book’s OpenAI client.
All code is in source-code/multi-client.
One Record for Six Vendors
Each client is one ClientConfig: a name, a URL, the env var holding its key, a header builder, a payload builder, and a reply parser. Behavior varies per vendor, so each slot is a function:
1 final case class ClientConfig(
2 name: String,
3 url: String,
4 apiKeyEnv: String,
5 headers: String => Map[String, String],
6 buildPayload: (String, String, Int) => ujson.Value,
7 parseReply: String => String
8 )
Four of the six speak the OpenAI chat shape: model, messages, max tokens. One builder serves them all, so any future code fix lands in just one place:
1 def openAiShapePayload(model: String, prompt: String, maxTokens: Int): ujson.Value =
2 ujson.Obj(
3 "model" -> model,
4 "max_tokens" -> maxTokens,
5 "messages" -> ujson.Arr(ujson.Obj("role" -> "user", "content" -> prompt))
6 )
7
8 def parseOpenAiChat(json: String): String =
9 ujson.read(json)("choices")(0)("message")("content").str
The Odd Two: Anthropic and Hugging Face
Anthropic differs in headers (x-api-key plus a version stamp) and in reply shape (content[0].text). Hugging Face differs more: POST {inputs} to a per model URL and read generated_text, which arrives bare or wrapped in a one item array:
1 def anthropicPayload(model: String, prompt: String, maxTokens: Int): ujson.Value =
2 ujson.Obj(
3 "model" -> model,
4 "max_tokens" -> maxTokens,
5 "messages" -> ujson.Arr(ujson.Obj("role" -> "user", "content" -> prompt))
6 )
7
8 def parseAnthropic(json: String): String =
9 ujson.read(json)("content")(0)("text").str
10
11 def hfPayload(prompt: String): ujson.Value =
12 ujson.Obj("inputs" -> prompt)
13
14 def parseHf(json: String): String =
15 val data = ujson.read(json)
16 if data.isInstanceOf[ujson.Arr] then data(0)("generated_text").str
17 else data("generated_text").str
The six configs then read as a table. Four share the Bearer header helper and the OpenAI shape; Anthropic and Hugging Face plug their own slots:
1 val anthropic: ClientConfig = ClientConfig(
2 "anthropic", "https://api.anthropic.com/v1/messages", "ANTHROPIC_API_KEY",
3 key => Map("Content-Type" -> "application/json", "x-api-key" -> key, "anthropic-version" -> "2023-06-01"),
4 anthropicPayload, parseAnthropic
5 )
6 val mistral: ClientConfig = ClientConfig(
7 "mistral", "https://api.mistral.ai/v1/chat/completions", "MISTRAL_API_KEY",
8 bearer, (m, p, t) => openAiShapePayload(m, p, t), parseOpenAiChat
9 )
10 val groq: ClientConfig = ClientConfig(
11 "groq", "https://api.groq.com/openai/v1/chat/completions", "GROQ_API_KEY",
12 bearer, (m, p, t) => openAiShapePayload(m, p, t), parseOpenAiChat
13 )
14 val moonshot: ClientConfig = ClientConfig(
15 "moonshot", "https://api.moonshot.ai/v1/chat/completions", "MOONSHOT_API_KEY",
16 bearer, (m, p, t) => openAiShapePayload(m, p, t), parseOpenAiChat
17 )
18 val perplexity: ClientConfig = ClientConfig(
19 "perplexity", "https://api.perplexity.ai/chat/completions", "PERPLEXITY_API_KEY",
20 bearer, (m, p, t) => openAiShapePayload(m, p, t), parseOpenAiChat
21 )
22
23 def huggingface(model: String): ClientConfig = ClientConfig(
24 "huggingface", s"https://api-inference.huggingface.co/models/$model", "HF_API_KEY",
25 bearer, (m, p, t) => hfPayload(p), parseHf
26 )
One Send for All
send reads the key from env, posts the built payload, checks status, and parses. A missing key raises before any socket opens, and the message names the var to set:
1 def send(client: ClientConfig, model: String, prompt: String, maxTokens: Int = 256): String =
2 val key = sys.env.getOrElse(client.apiKeyEnv, throw new IllegalStateException(s"Set ${client.apiKeyEnv} first."))
3 val response = requests.post(client.url, headers = client.headers(key),
4 data = client.buildPayload(model, prompt, maxTokens).render(), readTimeout = 120000)
5 if response.statusCode != 200 then
6 throw new java.io.IOException(s"${client.name} failed (HTTP ${response.statusCode}): ${response.text().take(200)}")
7 client.parseReply(response.text())
Demo and Its Output
The demo in multi-client/Main.scala prints payload shapes first, which needs no keys, then tries each live client and skips the ones with no key:
1 @main def clientsDemo(): Unit =
2 val prompt = "Mary is 30 and Bob is 25. Who is older? Answer in one short sentence."
3 println("Payload shapes (no keys needed):")
4 println("Anthropic: " + ModelClients.anthropic.buildPayload("claude-haiku-4-5", prompt, 64).render().take(160))
5 println("Mistral: " + ModelClients.mistral.buildPayload("mistral-small-latest", prompt, 64).render().take(160))
6 println("HF: " + ModelClients.huggingface("gpt2").buildPayload("gpt2", prompt, 64).render().take(160))
7 println("\nLive calls (need API keys, skipped when missing):")
8 ...
9 ModelClients.all.foreach { client =>
10 try println(s"$label: " + ModelClients.send(client, models(label), prompt, 64))
11 catch case e: Exception => println(s"$label: [skip] ${e.getMessage}")
12 }
With no keys set it prints:
1 Payload shapes (no keys needed):
2 Anthropic: {"model":"claude-haiku-4-5","max_tokens":64,"messages":[{"role":"user","content":"Mary is 30 and Bob is 25. Who is older? Answer in one short sentence."}]}
3 Mistral: {"model":"mistral-small-latest","max_tokens":64,"messages":[{"role":"user","content":"Mary is 30 and Bob is 25. Who is older? Answer in one short sentence."}]}
4 HF: {"inputs":"Mary is 30 and Bob is 25. Who is older? Answer in one short sentence."}
5
6 Live calls (need API keys, skipped when missing):
7 anthropic: [skip] Set ANTHROPIC_API_KEY first.
8 mistral: [skip] Set MISTRAL_API_KEY first.
9 groq: [skip] Set GROQ_API_KEY first.
10 moonshot: [skip] Set MOONSHOT_API_KEY first.
11 perplexity: [skip] Set PERPLEXITY_API_KEY first.
Set one or more of ANTHROPIC_API_KEY, MISTRAL_API_KEY, GROQ_API_KEY, MOONSHOT_API_KEY, PERPLEXITY_API_KEY, HF_API_KEY and the matching rows turn into live one sentence answers. The demo keeps max tokens at 64, so each call costs pocket change.
Parsers You Can Test Offline
Reply parsers are pure functions on strings, which makes them the easiest code in this book to test. The suite feeds canned server JSON for the chat shape, the Anthropic shape, and both Hugging Face shapes. It also checks the header maps and the missing key error with a bogus var name, so ambient keys in the shell cannot mask the check. If a vendor drifts its format next year, paste a fresh reply into the test and fix the parser with proof in hand:
1 val chatJson = """{"choices":[{"message":{"content":"Mary is older."}}]}"""
2 assert(ModelClients.parseOpenAiChat(chatJson) == "Mary is older.", "chat parse")
3 val anthJson = """{"content":[{"text":"Mary is older."}]}"""
4 assert(ModelClients.parseAnthropic(anthJson) == "Mary is older.", "anthropic parse")
Run the demo and the checks:
1 cd source-code/multi-client
2 scala-cli run . --main-class multiclient.clientsDemo
3 scala-cli run . --main-class multiclient.clientsTest
The checks need no keys and end with:
1 All client tests passed.