LLMs with Public APIs

The fastest way to use large language models is through cloud APIs. Google, OpenAI, and Anthropic, all offer APIs that give you access to their most capable proprietary models with just a few lines of Python code. Fireworks.ai is an inference provider in the USA that offers fast inferencing for many open weight models. You don’t need a GPU, you don’t need to download model weights, and you can start building applications in minutes.

In this chapter we work through practical examples using the Google Gemini API, the OpenAI API, and the Fireworks.ai API. Each provides Python client libraries (or compatibility layers) that handle authentication, request formatting, and response parsing. The patterns you learn here apply to other API providers as well — the core concepts of sending prompts, receiving completions, and managing conversations are the same across providers.

The examples for this chapter are in the directory source-code/llm_public_apis.

Google’s Gemini models are accessed through the Google AI API using the google-genai Python SDK. You need a free API key from Google AI Studio.

Install the SDK:

Store your API key in an environment variable:

Here is the simplest possible example — send a prompt to Gemini and print the response:

The output will be a concise explanation of transformer models. Each call to generate_content sends a request to Google’s servers, which run the model and return the generated text.

OpenAI’s GPT models are accessed through the openai Python SDK. You need an API key from OpenAI’s platform.

Install the SDK:

Store your API key:

Here is the equivalent example using OpenAI:

Both APIs follow the same pattern: create a client, send a prompt, and extract the generated text from the response.

Fireworks.ai provides fast, cost-effective access to open-weight models through an OpenAI-compatible API. This means you can use the openai Python SDK you already installed — just point it at Fireworks’ endpoint. DeepSeek V4 Flash, the default model we use here, delivers strong performance at a fraction of the cost of proprietary models.

Get a free API key from fireworks.ai and set it as an environment variable:

The simplest Fireworks example looks nearly identical to OpenAI’s chat completions pattern:

The output is a concise explanation of transformer models. The key difference from the standard OpenAI setup is the base_url parameter, which redirects the SDK to Fireworks’ servers. The messages format uses the familiar Chat Completions API structure with role-based message objects.

Text generation is the most fundamental LLM capability. You provide a prompt and the model generates a continuation or response.

The temperature parameter controls how creative or deterministic the output is. A temperature of 0 produces the most predictable output (the model always picks the highest-probability next token). Higher temperatures (up to 1.0 or 2.0) produce more varied and creative output.

For most practical applications — code generation, data extraction, question answering — use a low temperature (0.0 to 0.3). For creative writing and brainstorming, higher temperatures (0.7 to 1.5) produce more interesting results.

The Fireworks API works the same way. Since Fireworks uses the OpenAI Chat Completions format, temperature is passed as a top-level parameter:

You’ll see the same pattern as Gemini: temperature 0.0 produces a safe, predictable tagline, while 1.5 yields something more surprising and original.

Some models can engage in extended internal reasoning before producing a response. Google’s Gemini 2.5 Flash supports a thinking budget that controls how much computation the model devotes to reasoning through the problem before answering.

The thinking budget is specified in tokens. A budget of 0 disables thinking entirely (useful for simple tasks where speed matters). Higher budgets allow the model to reason through more complex problems but increase latency and cost.

Fireworks’ DeepSeek models also support thinking mode. When enabled, the model performs internal chain-of-thought reasoning before producing its final answer, and the reasoning tokens are returned separately:

The extra_body parameter passes the thinking configuration directly to the Fireworks API. DeepSeek’s approach differs from Gemini’s thinking budget — instead of controlling how many tokens to spend on reasoning, you simply enable or disable thinking mode. The reasoning trace is available via message.thinking, which is useful for debugging and understanding the model’s logic.

Real applications often involve multi-turn conversations where the model needs to remember previous exchanges. Both APIs support this by passing conversation history with each request.

Notice that the second and third messages use pronouns (“its”, “there”) that only make sense given the conversation history. The model resolves these references correctly because it sees the full conversation with each request.

The same pattern works with Fireworks using the Chat Completions message list format. Instead of building Gemini Content/Part objects, you append plain dicts to the messages array:

The Fireworks implementation is notably simpler than the Gemini version — the Chat Completions format uses standard dicts for messages rather than typed objects, which makes message history management straightforward.

Modern LLMs can process images alongside text. This enables applications like image description, document analysis, chart reading, and visual question answering.

The key detail is that contents accepts a list containing both text and image objects. The model processes them together, understanding the image in the context of the text prompt.

Some API providers allow the model to search the web as part of generating a response, which gives it access to current information beyond its training data.

Here is an example using OpenAI’s web search tool:

The tools parameter tells the model it can use web search to answer the question. The model decides whether to search based on the query — factual questions about recent events will trigger a search, while questions about well-known topics may not.

Google’s Gemini also supports grounding with Google Search through a similar mechanism. Refer to the Google AI documentation for the current syntax, as this feature is actively evolving.

For many applications you need the model to return data in a specific format — JSON, CSV, or a particular schema. LLMs can be instructed to produce structured output through careful prompting.

Using temperature 0.0 is important for structured output — you want the model to be deterministic and precise rather than creative. Some APIs also support specifying a JSON schema directly in the request, which guarantees the output conforms to a specific structure.

The Fireworks version uses the same prompting strategy. The code differs only in how the client is configured and how the response text is accessed:

The output is identical to the Gemini version — a clean JSON object with the extracted fields. The JSON cleanup logic (stripping markdown code fences) is the same because all LLMs tend to wrap code blocks in backticks.

API calls are billed per token. Input tokens (your prompt) and output tokens (the model’s response) are priced separately, with output tokens typically costing 2-4x more. Prices vary significantly between providers and models:

• Smaller, faster models (Gemini 2.5 Flash, GPT-5.4-nano) are very inexpensive — often under $0.10 per million input tokens
• Frontier models (Gemini 2.5 Pro, GPT-5.4, Claude Opus) cost 10-50x more but offer superior reasoning

For most applications, start with a fast, inexpensive model and only upgrade to a frontier model for tasks that require it.

All API providers enforce rate limits — maximum requests per minute, tokens per minute, and tokens per day. Free tiers have lower limits. If you’re building a production application, you’ll need to implement retry logic with exponential backoff and consider batching requests where possible.

API calls involve network round-trips and model inference time. Simple completions with small models return in under a second. Complex reasoning with frontier models can take 10-30 seconds or more. For interactive applications, consider streaming responses (both Gemini and OpenAI support this) so users see output as it’s generated rather than waiting for the complete response.

Any data you send to an API is transmitted to the provider’s servers. For sensitive data — medical records, financial information, proprietary code — review the provider’s data usage policies carefully. Some providers offer data residency guarantees and opt-out options for training. For maximum privacy, consider using local models instead, as covered in the next chapter.

API calls can fail for many reasons: network errors, rate limiting, content filtering, malformed requests, or service outages. Production code should handle these gracefully:

Using LLMs through public APIs is the fastest path from idea to working application. The core pattern is simple across all providers: create a client, send a prompt, process the response. The richness comes from features like multi-turn conversations, multimodal input, web search, structured output, and thinking modes.

The main tradeoffs of the API approach are cost (per-token pricing), privacy (data leaves your machine), and dependence on the provider’s availability. For applications where these tradeoffs are acceptable, public APIs give you access to the most capable models available.

In the next chapter we cover the alternative approach: running open-weights models locally on your own hardware, which offers privacy, no per-token cost, and offline operation at the expense of model capability and the need for suitable hardware.

To help solidify the concepts covered in this chapter, try implementing the following exercises. You can create these scripts in your local workspace to extend the existing code examples.

Modify the gemini_temperature.py example to create a command-line script that:

• Prompts the user to enter a business type (e.g., “coffee shop”, “dog walking service”, “indie game studio”).
• Prompts the user to enter a target audience (e.g., “college students”, “busy professionals”, “hardcore gamers”).
• Generates three different taglines using three distinct temperature values (e.g., 0.0 for deterministic/professional, 0.7 for balanced, and 1.5 for highly creative/unconventional).
• Displays the temperature alongside the generated tagline so you can compare the direct effects of the temperature parameter on creativity.

Using the gemini_conversation.py script as a starting point, build a fully interactive command-line chatbot:

• When the script starts, prompt the user to input a “persona” or system instructions (e.g., “You are a helpful assistant who answers exclusively in pirate speak” or “You are an encouraging coding mentor”).
• Configure the client or prompt structure to enforce this persona. (Hint: In the Gemini API, you can pass system instructions via types.GenerateContentConfig(system_instruction="...")).
• Enter a loop that repeatedly prompts the user for input (input("You: ")).
• Exit the loop gracefully if the user types exit or quit.
• Print the assistant’s responses and append each turn to the conversation history to maintain context.

Combine the concepts from gemini_image.py and gemini_structured.py to extract structured information from a document image:

• Find or capture an image containing unstructured text (e.g., a photo of a restaurant receipt, a business card, or a book cover).
• Load the image using Pillow and write a script that sends the image along with a prompt requesting the model to extract key details.
• Instruct the model to return a structured JSON response (e.g., for a book cover, extract "title", "author", "publisher", and "estimated_publication_year").
• Parse the JSON response in Python and display the extracted keys and values in a formatted terminal printout.

Create a robust text processing pipeline that extracts structural sentiment analysis from product reviews:

• Write a script that takes a list of raw user reviews (e.g., "The battery life is amazing, but the screen is a bit dim.").
• Define a target schema for the output containing: sentiment (must be one of Positive, Negative, or Neutral), sentiment_score (a float between 0.0 and 1.0), and a list of pros and cons.
• Implement a function to call the Gemini API using gemini-3-flash-preview to perform this extraction, ensuring temperature is set to 0.0.
• Integrate the exponential backoff retry logic described in the Error Handling section of this chapter. If a call fails, retry up to 3 times with progressive delays.
• Add a Fallback Provider: If the Gemini API call still fails after 3 retries (due to rate limits, quota limits, or API outage), catch the exception, print a warning, and fall back to the OpenAI API using gpt-5.4-nano to process that specific review.