Haystack handles relevance ranking in document retrieval through a two-step process combining initial candidate retrieval and neural re-ranking. First, it uses fast retrieval methods like BM25 or dense vector search to fetch a broad set of candidate documents. These methods prioritize speed and recall, ensuring relevant documents are in the initial pool. For example, BM25 calculates term frequency and inverse document frequency to match keywords, while dense retrievers like DPR encode text into vectors for semantic similarity comparisons. This first step narrows thousands of documents to a manageable subset (e.g., 100-200 candidates) for deeper analysis.
In the second phase, Haystack applies neural rankers to reorder the candidates by relevance. Transformer-based models like BERT or RoBERTa are commonly used here, analyzing query-document pairs at a finer granularity. These models process the full text of documents and queries, capturing contextual relationships that keyword-based methods miss. For instance, a cross-encoder model might score how well a document’s passage answers “What causes climate change?” by evaluating semantic alignment rather than just keyword overlap. This step is computationally intensive but critical for precision, as it resolves ambiguities (e.g., distinguishing between “Java the island” and “Java the programming language”).
Developers can customize this pipeline in Haystack by choosing retrievers, rankers, and thresholds that fit their use case. The framework supports hybrid approaches, such as combining BM25 and dense retrieval results before re-ranking. Parameters like the number of top candidates to re-rank (e.g., 100 vs. 500) allow balancing speed and accuracy. For example, a legal search system might prioritize recall by using a dense retriever with a large candidate pool, while a chat app might optimize latency by re-ranking only 50 documents. Haystack’s modular design lets teams experiment with components—like swapping MiniLM-L6 for a larger model—without rebuilding the entire system.