Serving a Workflow with FastAPI

You have built a working Workflow. You want it to serve traffic (accept requests, run the workflow, return the answer) without a lot of ceremony. This chapter walks through a small useful setup: one HTTP server process, one client. The workflow itself is a two-step Q&A workflow (deliberately trivial so the deployment plumbing is the interesting part).

A quick note on llama-deploy: if you have read about LlamaIndex deployment before, you may expect this chapter to be built on it using such things as a control plane, one or more workflow services, a Redis-backed message queue between them. That was the plan, and for most of this book’s life it was the code sitting in this directory. It broke: the current llama-deploy release (0.9.x) is incompatible with llama-index-core 0.14, the version this book uses everywhere else. Rather than pin an old llama-index-core just to keep a deployment framework working, this chapter serves the workflow directly with FastAPI. It is simpler, it has no other external dependencies (no Redis, nothing to install with Homebrew or Docker), and it is the same OSS-first, run-it-on-your-laptop approach as every other chapter in the book. If llama-deploy catches back up, the swap is confined to this one file; the workflow itself does not change at all.

Everything lives in source-code/llama_index_deploy/ with our usual setup:

1 $ cd source-code/llama_index_deploy
2 $ uv sync
3 $ ollama pull qwen3.5:4b

No Redis, no other services: that is the whole setup.

The moving parts

Two things run:

  • The server: a FastAPI app wrapping the Workflow, served by uvicorn. It exposes POST /ask (run the workflow and return the answer) and GET /health (liveness check). One process, one port.
  • The client: anything that can send an HTTP request. This chapter uses a small httpx script, but a curl command or a client in any other language works identically, since the server speaks plain JSON over HTTP.

You run the server in one terminal and the client in another.

The workflow being served

The utility _workflow.py is deliberately small:

 1 from llama_index.core.workflow import StartEvent, StopEvent, Workflow, step
 2 from llama_index.llms.ollama import Ollama
 3 
 4 
 5 class QAWorkflow(Workflow):
 6     def __init__(self, *args, **kwargs):
 7         super().__init__(*args, **kwargs)
 8         self.llm = Ollama(model="qwen3.5:4b", temperature=0, request_timeout=180.0)
 9 
10     @step
11     async def answer(self, ev: StartEvent) -> StopEvent:
12         question = ev.get("question", "")
13         reply = await self.llm.acomplete(f"Answer briefly: {question}")
14         return StopEvent(result=reply.text.strip())

One step, one LLM call. In a real deployment this would be your actual workflow: a RAG pipeline, a ReAct agent, a multi-step research workflow. Nothing about the workflow class itself changes when you serve it; that is the point of the Workflow abstraction from Chapter “The Workflows API”.

Terminal 1: the server

The top-level server script for this example is 01_serve_workflow.py:

 1 from fastapi import FastAPI
 2 from pydantic import BaseModel
 3 import uvicorn
 4 
 5 from _workflow import QAWorkflow
 6 
 7 app = FastAPI(title="QA Workflow API")
 8 workflow = QAWorkflow(timeout=180.0)
 9 
10 
11 class QuestionRequest(BaseModel):
12     question: str
13 
14 
15 class AnswerResponse(BaseModel):
16     answer: str
17 
18 
19 @app.post("/ask", response_model=AnswerResponse)
20 async def ask(request: QuestionRequest):
21     """Run the QA workflow and return the answer."""
22     result = await workflow.run(question=request.question)
23     return AnswerResponse(answer=str(result))
24 
25 
26 @app.get("/health")
27 async def health():
28     return {"status": "ok"}
29 
30 
31 if __name__ == "__main__":
32     uvicorn.run(app, host="127.0.0.1", port=8000)

Walking through it. The QAWorkflow is instantiated exactly once, at import time, and reused across every request, the same pattern you would use for a model or a database connection pool. QuestionRequest and AnswerResponse are Pydantic models; FastAPI uses them to validate the incoming JSON body and to generate an OpenAPI schema for free (visit http://127.0.0.1:8000/docs while the server is running). ask is an async def route, so it can await workflow.run(...) without blocking the event loop while the LLM call is in flight, so other requests can be served concurrently. health is the kind of endpoint a load balancer or container orchestrator polls before sending traffic to this process.

Run it and leave it running:

1 $ uv run 01_serve_workflow.py
2 INFO:     Started server process
3 INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

Terminal 2: the client

Here is a small example client script 02_client.py:

 1 import asyncio
 2 
 3 import httpx
 4 
 5 
 6 async def main():
 7     async with httpx.AsyncClient(timeout=300.0) as client:
 8         # Health check
 9         resp = await client.get("http://127.0.0.1:8000/health")
10         print(f"Health: {resp.json()}")
11 
12         # Ask a question
13         resp = await client.post(
14             "http://127.0.0.1:8000/ask",
15             json={"question": "What is the capital of Arizona?"},
16         )
17         data = resp.json()
18         print(f"AGENT: {data['answer']}")
19 
20 
21 asyncio.run(main())

Nothing LlamaIndex-specific here: this is a generic async HTTP client hitting two REST endpoints. httpx.AsyncClient is used instead of the synchronous requests because the rest of the book’s async code (Workflows, agents) already pulls in an async HTTP stack; for a one-off script, requests or curl work exactly as well.

Expected output:

1 $ uv run 02_client.py
2 Health: {'status': 'ok'}
3 AGENT: Phoenix is the capital of Arizona.

The same request from the command line, no Python client needed:

1 $ curl -s -X POST http://127.0.0.1:8000/ask \
2     -H "Content-Type: application/json" \
3     -d '{"question": "What is the capital of Arizona?"}'
4 {"answer":"Phoenix is the capital of Arizona."}

Scaling from here

Everything above runs as one process on one laptop. The path to production is incremental, not a rewrite:

  1. One process: where you started. Fine for development and for low-traffic internal tools.
  2. Multiple worker processes, one machine: uvicorn’s --workers flag runs several copies of the app behind one port and spreads connections across them; that is normally the first lever to pull for more throughput. It needs the app passed as an import string (uvicorn server:app --workers 4) rather than as a live object, which means two small changes from what is shown above: move the FastAPI app into a plain importable module (the 01_ prefix used for teaching order in this book’s scripts is not a valid Python module name to import) and read host/port from the environment instead of hard-coding them. Neither the workflow nor the routes change.
  3. More workflows, same server: add another route (/summarize, /extract, whatever the next workflow does), each backed by its own Workflow instance, to the same FastAPI app. Ordinary FastAPI, not a new concept to learn.
  4. Move to a small VPS: the server needs Python and enough RAM to hold whatever model it talks to (or a network path to a hosted model); nothing else.
  5. Clients in any language: the server speaks plain HTTP and JSON, so anything that can issue a POST (curl, a browser fetch, a mobile app, a service written in Go) can call it. There is no client library to install anywhere except in this one example script, and even that is just a convenience.

If a workflow’s steps genuinely need a shared task queue (long-running jobs, retries, work distributed across many machines), that is the point where reaching for Redis plus a worker library (arq, Celery) starts to pay for itself. It is worth adding when a specific problem shows up, not before; nothing in this chapter needs it.

What we covered

  • A LlamaIndex Workflow is served over HTTP with plain FastAPI: instantiate it once, await workflow.run(...) inside an async def route.
  • Two endpoints are enough for a real service: one that does the work, one (/health) that reports liveness.
  • The client side is unremarkable: any HTTP client, in any language, talking JSON.
  • The scaling path is uvicorn --workers, more routes, and a VPS behind a reverse proxy: infrastructure you add when you need it, not infrastructure the framework requires up front.

That closes Part II. The four appendices that follow cover cross-cutting topics that apply to both LangChain and LlamaIndex projects: choosing a model, doing evaluation without LangSmith, doing observability without LangSmith, and putting a small LLM app on a $5/month VPS.

Up next

Using LLMs To Organize Information in Our Google Drives