Multi-Index Query Pipelines
Every example so far in Part II has used a single index over a single corpus. Real projects rarely look like that. A support-desk assistant has a docs index, a runbooks index, and a changelog index. A research assistant has one index per paper set. A personal knowledge base has one index per notebook, project, or year.
LlamaIndex has two prebuilt patterns for querying across multiple indices: RouterQueryEngine and SubQuestionQueryEngine. Both wrap several per-corpus query engines and use an LLM to decide how to combine them.
Examples for this chapter live in source-code/llama_index_router/. For teaching purposes each of the four ../data/*.txt files becomes its own tiny index: chemistry, economics, health, sports. In a real project the same code would drive four indices over four folders of hundreds of documents each.
Use our usual setup for running the examples:
1 $ cd source-code/llama_index_router
2 $ uv sync
3 $ ollama pull qwen3.5:4b
We use the utility build_per_topic_engines defined in the file _indices.py:
1 """Build one VectorStoreIndex per topic file in ../data/.
2
3 Returns a dict of {topic_name: query_engine} that both scripts share.
4 """
5
6 from pathlib import Path
7
8 from llama_index.core import Document, Settings, VectorStoreIndex
9 from llama_index.embeddings.huggingface import HuggingFaceEmbedding
10 from llama_index.llms.ollama import Ollama
11
12 Settings.llm = Ollama(model="qwen3.5:4b", request_timeout=180.0)
13 Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
14
15 DATA_DIR = Path(__file__).parent.parent / "data"
16
17
18 def build_per_topic_engines() -> dict[str, object]:
19 engines = {}
20 for path in sorted(DATA_DIR.glob("*.txt")):
21 topic = path.stem
22 text = path.read_text(encoding="utf-8").strip()
23 doc = Document(text=text, metadata={"topic": topic})
24 index = VectorStoreIndex.from_documents([doc])
25 engines[topic] = index.as_query_engine()
26 return engines
RouterQueryEngine: pick one index
For queries that clearly belong to one corpus, you want the router to send the whole query to that one index and get a synthesized answer back and this idea is implemented in the script 01_router_query_engine.py in both examples for this chapter. Line 7 uses the utility function defined in the last listing:
1 from llama_index.core.query_engine import RouterQueryEngine
2 from llama_index.core.selectors import LLMSingleSelector
3 from llama_index.core.tools import QueryEngineTool
4
5 from _indices import build_per_topic_engines
6
7 engines = build_per_topic_engines()
8
9 tools = [
10 QueryEngineTool.from_defaults(
11 query_engine=engines["chemistry"],
12 description="Questions about chemistry, matter, substances, and lab work.",
13 ),
14 QueryEngineTool.from_defaults(
15 query_engine=engines["economics"],
16 description="Questions about economics, markets, and schools of economic thought.",
17 ),
18 QueryEngineTool.from_defaults(
19 query_engine=engines["health"],
20 description="Questions about human health, disease, exercise, and well-being.",
21 ),
22 QueryEngineTool.from_defaults(
23 query_engine=engines["sports"],
24 description="Questions about sports, athletic activity, and physical competition.",
25 ),
26 ]
27
28 router = RouterQueryEngine(
29 selector=LLMSingleSelector.from_defaults(),
30 query_engine_tools=tools,
31 )
32
33 for q in [
34 "What is chemistry?",
35 "What is the Austrian School of Economics?",
36 "How does exercise affect the body?",
37 ]:
38 print(f"USER: {q}")
39 print(f"AGENT: {router.query(q)}\n")
The mechanism is simple. Each per-topic engine gets wrapped in a QueryEngineTool with an English description. The LLMSingleSelector reads the query plus all the descriptions and picks one tool. The router forwards the query to that tool and returns the tool’s response.
The quality of routing depends entirely on the quality of the tool descriptions. Vague descriptions produce wrong routing; overlapping descriptions produce coin-flip routing. This is the same discipline as writing good tool docstrings for a ReAct agent.
LLMMultiSelector.from_defaults() is the sibling class that picks multiple tools and combines their responses, useful when questions might legitimately touch two or three indices.
SubQuestionQueryEngine: decompose and combine
For compound questions that no single index can answer alone (“compare A and B,” “how do X, Y, and Z relate?”), you want the engine to plan a series of subquestions, run each against the appropriate index, and synthesize a combined answer and we implement these ideas in the script 02_subquestion_query_engine.py:
1 from llama_index.core.query_engine import SubQuestionQueryEngine
2 from llama_index.core.tools import QueryEngineTool, ToolMetadata
3
4 from _indices import build_per_topic_engines
5
6 engines = build_per_topic_engines()
7
8 tools = [
9 QueryEngineTool(
10 query_engine=engines["chemistry"],
11 metadata=ToolMetadata(
12 name="chemistry_index",
13 description="Corpus about chemistry, matter, substances, and lab work.",
14 ),
15 ),
16 QueryEngineTool(
17 query_engine=engines["economics"],
18 metadata=ToolMetadata(
19 name="economics_index",
20 description="Corpus about economics, markets, and schools of economic thought.",
21 ),
22 ),
23 QueryEngineTool(
24 query_engine=engines["health"],
25 metadata=ToolMetadata(
26 name="health_index",
27 description="Corpus about human health, disease, exercise, and well-being.",
28 ),
29 ),
30 QueryEngineTool(
31 query_engine=engines["sports"],
32 metadata=ToolMetadata(
33 name="sports_index",
34 description="Corpus about sports, athletic activity, and physical competition.",
35 ),
36 ),
37 ]
38
39 engine = SubQuestionQueryEngine.from_defaults(
40 query_engine_tools=tools,
41 use_async=False,
42 )
43
44 query = "How do sports, health, and chemistry relate to one another?"
45 print(engine.query(query))
Behind the scenes: an LLM planner reads the compound query and the tool descriptions, produces a list of subquestions (typically one per relevant tool), each subquestion runs against its target index, and a final LLM call synthesizes the subquestion answers into a coherent response.
There are costs to be aware of. With four tools and a compound query, you may end up with three or four subquestion runs plus two synthesis calls: six or seven LLM calls per user query. On a local model this adds latency; on a hosted model it adds a real bill. SubQuestionQueryEngine is powerful but not the tool to reach for on every question.
When to reach for which
Dear reader, you can experiment on your own, but my takeaway is:
RouterQueryEnginewithLLMSingleSelector: most queries clearly belong to one corpus. Cheap: one selector call plus one downstream query engine call.RouterQueryEnginewithLLMMultiSelector: most queries belong to one or two corpora. Slightly more expensive; useful when your corpora overlap.SubQuestionQueryEngine: compound “compare / relate / synthesize” queries that no single index can answer. Most expensive; reach for it when you have evidence users actually ask these questions.
You can also build routing yourself with the Workflows API from Chapters “The Workflows API” and “Building an Agent as a Workflow”: a classify step, a routing step, per-corpus engines behind separate steps. That is what you would do when your routing logic is deterministic (based on user role, request metadata, or a fixed classification) rather than LLM-driven.
What we covered
RouterQueryEnginesends each query to one (or a few) of several per-corpus engines via LLM-driven routing.SubQuestionQueryEnginedecomposes compound queries into per-corpus subquestions and synthesizes the results.- Both patterns depend heavily on the English descriptions attached to each
QueryEngineTool. Treat descriptions like production API contracts.
The next chapter “Structured Extraction” uses PydanticProgram to force LLM output into a validated data schema.