Agentic search is retrieval controlled by an LLM agent: instead of running one fixed search and answering, the agent decides whether to search, what to search for, which tool or corpus to use, reads the results, judges whether they are enough, and searches again with a better query until it can answer or hits a budget. A vector database is one of the tools inside that loop, the one that finds passages by meaning, usually alongside keyword search, structured queries, and sometimes web search.
Bound the agent search loop by evidence gaps
How does an agentic search loop work?
The pattern descends from ReAct (Yao et al., 2022), which interleaves reasoning steps with tool calls. A typical loop:
- Plan. Read the question and decide whether retrieval is needed at all. Break a compound question into sub-questions (“compare the refund policy in the EU and US plans” becomes two lookups).
- Act. Call a tool with explicit arguments: a vector search over product docs, a BM25 search for an error code, a SQL or scalar query for an account record.
- Observe. Read the returned passages and their source IDs.
- Reflect. Grade the evidence: is it relevant, sufficient, consistent? Identify what is still missing.
- Repeat or stop. Rewrite the query, switch tools, or narrow filters if there is a specific gap; otherwise answer with citations, or abstain.
Milvus’s agentic RAG with LangGraph tutorial implements this as a small graph: a node decides whether to retrieve or respond, a retrieve node calls Milvus, a grading node checks relevance, and irrelevant results route to a rewrite node that loops back to try again.
Agentic search vs classic RAG
| Classic RAG | Agentic search | |
|---|---|---|
| Retrieval calls | One, always | Zero to many, decided per question |
| Query | The user’s text, maybe rewritten once | Generated, decomposed, and revised by the agent |
| Sources | Usually one index | Several tools: vector, keyword, structured, web, APIs |
| Handles multi-step questions | Poorly | Well, by chaining lookups |
| Latency | Predictable, often well under a second of retrieval | Variable; several LLM turns, often seconds to minutes |
| Cost | One LLM call | Several LLM calls plus tool calls |
| Failure modes | Missed evidence | Loops, drift, over-searching, tool misuse |
| Debuggability | Linear pipeline | Needs traces of every step |
Agentic search is not “better RAG” in general. It wins on questions that need several pieces of evidence, a choice of source, or verification; a single well-tuned retrieval still wins on simple lookups because it is faster and cheaper.
Where does a vector database fit?
Inside the loop, the vector database is the semantic-recall tool. Its job is to answer each tool call quickly and correctly so the agent spends its turns reasoning instead of waiting or recovering from bad results. That puts specific requirements on it:
- Low latency per call. An agent may issue five to twenty searches per task; tens of milliseconds each keeps the loop responsive.
- Hybrid retrieval. Agents generate a mix of conceptual queries and exact strings. Milvus can serve both from one collection: dense vectors for meaning, a BM25 full-text field (Milvus 2.5+) for exact terms, and
hybrid_searchto fuse them. - Filters the agent cannot bypass. Tenant, permission, and document-status constraints should be added by the tool wrapper from trusted context, not chosen by the model. Milvus applies them in the same query as the vector search.
- Stable IDs and metadata in results. The agent needs source IDs, versions, and URLs to cite, deduplicate across iterations, and avoid re-reading the same passage.
- Scale for many corpora. Separate collections or partition keys let one deployment serve many knowledge bases or tenants.
How the agent should choose between those tools for a given query is covered in how agents choose between vector, keyword, and structured search. For exposing Milvus to agents through the Model Context Protocol, see what a Milvus MCP server exposes.
How do you keep the loop bounded?
Unbounded loops are the main operational risk. Practical guardrails:
- Budgets. Cap tool calls (for example 5-8 searches), total tokens, and wall-clock time per task.
- A reason to continue. Allow another search only when the agent names a specific missing piece of evidence or a contradiction, not "to be thorough".
- Novelty checks. Stop if a new search returns mostly IDs already seen.
- Explicit abstention. “The sources do not answer this” must be an acceptable final state.
- Treat retrieved text as data. Passages may contain instructions; they should never change tool permissions or filters.
How do you evaluate agentic search?
Compare it with a fixed single-retrieval baseline on the same tasks. Measure evidence found (recall of labeled passages across all iterations), answer correctness and citation support, task completion, number of tool calls, latency, and total cost. Include questions with no answer and questions where the first plausible document is a trap. Keep a trace of each query, filters, returned IDs, and the stop reason; the trace shows whether a failure came from the tool, the data, or the control logic. Building such task sets is covered in how to build an evaluation set.
Real implementations to start from
- The LangGraph tutorial for a compact retrieve-grade-rewrite loop.
- DeepSearcher, an open-source deep-research agent that decomposes questions and iterates over private data in Milvus.
- Read AI, a production example of retrieval behind AI assistants.
Start with the tutorial against a corpus you know, compare it with one-shot RAG on twenty real questions, and keep the extra autonomy only where it pays for its latency. You can run the whole thing locally with the Milvus Quickstart.
Related reading
- What is retrieval-augmented generation (RAG)?
- How do LangGraph state and Milvus retrieval work together?
- What is Graph RAG, and where does Milvus fit?
- How can vector search retrieve useful agent trajectories?
- How should you structure agent memory in Milvus?
- What is hybrid search, and when should you use it?
