Keyword search, semantic search, and reranking

Different searches answer different needs

Keyword search finds terms and combinations of terms. Semantic search uses numerical representations to find passages with related meaning. An embedding is such a representation. Neither method inherently establishes that a passage proves your proposition. [source]

Search for “termination” and a keyword system may miss “bring the agreement to an end.” A semantic search may find that wording, but may also retrieve a passage about an unrelated relationship ending. Similarity is a candidate-selection signal.

Work a miniature search problem

Question: “Who knew about the revised delivery date before 12 March?”

A useful search plan includes exact names, document IDs, “revised schedule,” “new date,” and language of receipt or awareness. It also considers contrary evidence: “not received,” “first saw,” and references to an original schedule. Search is iterative because the record teaches you its vocabulary.

Use date and matter filters deliberately. A filter based on document date can exclude a later email that describes earlier knowledge. Whether that is appropriate depends on the question. Do not confuse the event period with the creation period of its evidence.

Combine retrieval with a second inspection

A retrieve-and-rerank system first gathers candidates, then applies another model to order them by relevance to the query. This can improve which candidates reach the answer stage, but it still depends on the first stage retrieving useful candidates. [source]

A practical test set can include ten questions with known relevant passages. For each question, record whether a supporting passage appears in the first few results, whether contrary material appears, and whether the passage is complete enough to interpret. Do not evaluate search only by the quality of a generated answer: generation can hide a weak result list.

Watch chunk boundaries

A passage may contain the answer but omit the definition, condition, or exception immediately before it. Retrieve surrounding text where needed. Keep the source document and location attached to every passage. A chunk identifier that cannot be mapped back to the original is a poor final citation.

Improve the query before changing the system

When a known passage is missing, check extraction, filters, vocabulary, chunk boundaries, and indexing status. Only then decide whether a different embedding or reranker is the right repair. This sequence prevents an expensive model change from disguising a simple metadata mistake.