Choosing an Index Type

VectorStoreIndex is the right answer roughly 90% of the time. This chapter is about the other 10%, and about the one hybrid pattern that further increases the quality of lookup results.

LlamaIndex has always shipped several index types. In the previous edition I only mentioned them in passing because most were rarely worth the extra complexity. In 2026 that is still mostly true, but two of the alternatives (SummaryIndex and the BM25 + vector hybrid) have concrete use cases where they clearly outperform a naked VectorStoreIndex. The third one covered here (SimpleKeywordTableIndex) is worth knowing about even if you never ship it, because it makes the “why do we need embeddings at all” question concrete.

Dear reader, if you enjoy reading code, my Common Lisp book has a BM25 + vector hybrid implementation written using no third party libraries.

All three scripts for this chapter live in source-code/llama_index_indices/ and read from source-code/data/.

The setup for running the examples is:

1 $ cd source-code/llama_index_indices
2 $ uv sync
3 $ ollama pull qwen3.5:4b

SummaryIndex: every query touches every Node

VectorStoreIndex is optimized for “find the top-k Nodes most similar to this query.” That is the wrong shape for other use cases such as questions like “summarize the whole corpus,” “what themes recur across these documents,” or “give me an overview of everything in this folder.” Those queries want the LLM to see every Node, in order, and reason across all of them.

SummaryIndex (the current name for what used to be called ListIndex) does exactly this. It maintains the Nodes in insertion order and, at query time, feeds all of them plus the query to the LLM as seen in our first example 01_summary_index.py:

 1 from llama_index.core import Settings, SimpleDirectoryReader, SummaryIndex
 2 from llama_index.embeddings.huggingface import HuggingFaceEmbedding
 3 from llama_index.llms.ollama import Ollama
 4 
 5 Settings.llm = Ollama(model="qwen3.5:4b", request_timeout=180.0)
 6 Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
 7 
 8 documents = SimpleDirectoryReader("../data").load_data()
 9 index = SummaryIndex.from_documents(documents)
10 
11 query_engine = index.as_query_engine()
12 
13 response = query_engine.query(
14     "Give me a one-paragraph overview of the topics covered by these documents."
15 )
16 print(response)

Representative output:

1 The documents cover four distinct topics: chemistry (the scientific
2 study of matter and its transformations), economics (schools of thought
3 and their approaches to markets and government), health (factors
4 affecting human well-being and disease prevention), and sports
5 (the definition of athletic activity and its cultural and physical
6 dimensions). Together they span the natural sciences, social sciences,
7 health sciences, and cultural studies.

VectorStoreIndex.as_query_engine().query(...) on the same question would pick the top few Nodes by embedding similarity and miss most of the corpus, because “give me an overview” is a semantic query about the corpus itself, not about any specific concept the corpus contains.

The tradeoff is obvious: SummaryIndex is O(n) per query, where n is the number of Nodes. Fine for corpora with dozens of Nodes, painful with thousands, unusable with millions. In practice I use it either for small curated collections (a research folder, a single project’s docs, a book’s chapters) or as a downstream tool for questions where the router has already narrowed the corpus to a small subset.

SimpleKeywordTableIndex: retrieval by exact keyword match

Embeddings are not always the right retrieval mechanism. If your corpus is dominated by proper nouns, product identifiers, function names, drug names, or legal citations, semantic similarity is often worse than exact-string matching. The classic case: your query mentions “GPT-4o” and the relevant document is one of the few that also mentions “GPT-4o” verbatim. A dense retriever will happily return semantically-adjacent documents about “large language models,” “OpenAI,” or “GPT-4,” pushing your actual match down or off the list.

SimpleKeywordTableIndex builds an inverted index of the corpus using regex keyword extraction. No LLM, no embedding model, no ML dependency at all: just Python string processing. At query time, it extracts keywords from the query the same way and retrieves Nodes whose keyword sets overlap as seen in our second example 02_keyword_table.py:

 1 from llama_index.core import SimpleDirectoryReader, SimpleKeywordTableIndex
 2 
 3 documents = SimpleDirectoryReader("../data").load_data()
 4 index = SimpleKeywordTableIndex.from_documents(documents)
 5 
 6 retriever = index.as_retriever()
 7 
 8 for query in [
 9     "What is chemistry?",
10     "Tell me about the Austrian School",
11     "What is a laboratory?",
12 ]:
13     print(f"QUERY: {query}")
14     nodes = retriever.retrieve(query)
15     for i, n in enumerate(nodes, 1):
16         src = n.node.metadata.get("file_name", "?")
17         print(f"  [{i}] source={src}")
18     print()

Where this actually earns its keep in real projects:

  • Factoid corpora full of specific terms. Product catalogs, drug databases, legal statute collections, API references.
  • Environments where you cannot install ML dependencies. Air-gapped systems, minimal Docker images, edge devices.
  • As a cheap first-pass filter before a more expensive stage: pull a wide keyword-based set of candidates, then rerank them with an LLM or a cross-encoder.

Where it fails: any query where the important word does not appear literally in the relevant Node. Paraphrases are invisible. That is why the hybrid pattern in the next section usually beats either dense-only or sparse-only in isolation.

The KeywordTableIndex variant (no “Simple”) uses an LLM to extract richer keyword lists including synonyms and related concepts. More accurate for hard queries, slower to build, needs a model. In practice I have not shipped the LLM version in a while; the LLM cost at ingestion time is high enough that if I can afford it, I would rather spend it on a reranker at query time.

QueryFusionRetriever: the practical hybrid pattern

The one hybrid pattern I use in almost every real LlamaIndex project. Run both a dense (embedding) retriever and a sparse (BM25) retriever over the same corpus, then merge their ranked lists with reciprocal rank fusion as seen in ur third example 03_hybrid_fusion.py:

 1 from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex
 2 from llama_index.core.retrievers import QueryFusionRetriever
 3 from llama_index.embeddings.huggingface import HuggingFaceEmbedding
 4 from llama_index.retrievers.bm25 import BM25Retriever
 5 
 6 Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
 7 
 8 documents = SimpleDirectoryReader("../data").load_data()
 9 
10 vector_index = VectorStoreIndex.from_documents(documents)
11 dense = vector_index.as_retriever(similarity_top_k=3)
12 
13 sparse = BM25Retriever.from_defaults(
14     nodes=list(vector_index.docstore.docs.values()),
15     similarity_top_k=3,
16 )
17 
18 fusion = QueryFusionRetriever(
19     retrievers=[dense, sparse],
20     similarity_top_k=3,
21     num_queries=1,
22     mode="reciprocal_rerank",
23     use_async=False,
24     verbose=False,
25 )
26 
27 for query in [
28     "How does body chemistry affect exercise?",
29     "Austrian School",
30 ]:
31     print(f"QUERY: {query}")
32     nodes = fusion.retrieve(query)
33     for i, n in enumerate(nodes, 1):
34         src = n.node.metadata.get("file_name", "?")
35         print(f"  [{i}] score={n.score:.3f}  source={src}")
36     print()

The BM25 retriever wants a list of Nodes, not a full index, hence the slightly odd list(vector_index.docstore.docs.values()) pattern. In a production setup where you have already run an ingestion pipeline, you would pass the Node list directly.

Two QueryFusionRetriever parameters worth knowing about:

  • mode="reciprocal_rerank": reciprocal rank fusion. Documents ranked highly by either retriever get a good final score; documents ranked highly by both get an excellent final score. This is the default and almost always what you want.
  • num_queries=1: the user’s query is used as-is. If you set this to a higher number, the fusion retriever asks the LLM to generate that many query rewrites and runs each retriever on each rewrite. Similar to the “RAG Patterns with LangChain” chapter’s MultiQueryRetriever from LangChain but built into the fusion retriever itself. Costs LLM calls, buys recall.

On the two-query test in the script, the fusion retriever handles both cleanly: the “body chemistry / exercise” query gets a strong dense-retrieval boost, and the “Austrian School” query (a proper noun that appears verbatim in economics.txt) gets a strong BM25 boost. Neither retriever on its own would rank both queries as well as the fusion does.

Decision tree

A concrete decision procedure I use in my own projects:

  1. Start with VectorStoreIndex and see how retrieval quality holds up on real user queries. This is the right answer most of the time.
  2. If overview-style queries do badly, add a SummaryIndex over the same corpus and route those queries to it via a router (Chapter “Multi-Index Query Pipelines”).
  3. If exact-term queries do badly (product names, function names, proper nouns), swap the VectorStoreIndex retriever for a QueryFusionRetriever over both dense and BM25. This is close to free (no extra models, no meaningful latency increase) and it fixes a wide class of retrieval failures.
  4. If ingestion-time budget is nonexistent and you cannot install ML dependencies at all, use SimpleKeywordTableIndex. Ship it, measure, revisit.

Reach for TreeIndex (not covered here) if you have thousands of Nodes and need a hierarchical retrieval that traverses top-down. Reach for KnowledgeGraphIndex if you want the framework to extract entities and relationships for you and query them as a graph, but at that point you may find the DBpedia/Wikidata SPARQL agents from Chapter “DBpedia and Wikidata as Agent Tools” a cleaner fit.

What we covered

  • VectorStoreIndex is the default, but not the only tool.
  • SummaryIndex handles overview / cross-corpus queries by touching every Node: O(n) per query, worth it when the shape of the question demands it.
  • SimpleKeywordTableIndex gives you keyword-based retrieval with zero ML dependencies: a fallback and a first-pass filter.
  • QueryFusionRetriever fuses BM25 and dense retrieval with reciprocal rank fusion. This is the hybrid pattern most projects should use once a vanilla vector retriever starts missing obvious matches.

The next chapter “RAG with Reranking” covers reranking, the last piece of the retrieval-quality puzzle before we move on to LlamaIndex’s Workflows API in Chapter “The Workflows API”.

Up next

RAG with Reranking