Site Search That Actually Works: Hybrid Retrieval, Vectors and Relevance Engineering
Site search is the highest-intent surface on your website and usually the worst-maintained. Here is how hybrid retrieval and relevance engineering fix it.
Visitors who use site search convert at multiples of those who do not. They have told you precisely what they want, in their own words, with no ambiguity about intent. And on most enterprise websites, that surface is powered by a default configuration nobody has revisited since launch, returning zero results for the third-most-common query on the site.
This guide covers how modern search is built: hybrid retrieval, deliberate ranking, and the unglamorous relevance engineering that separates a search box from a search experience.
Why keyword-only search fails
Inverted-index search matches tokens. It is exceptional at exact identifiers and terrible at meaning. Three failure modes dominate:
- Vocabulary mismatch. The user searches "jumper", your catalogue says "sweater". Zero results.
- Intent blindness. "How do I return an item" gets matched against product titles containing "item".
- Long-tail collapse. Descriptive, conversational queries — now the norm because people have learned to talk to machines — score poorly against short-field matching.
Why vector-only search also fails
Embedding-based retrieval solves meaning and immediately creates a new problem: it is approximate by design. Search for part number MX-4471-B and a pure vector system helpfully returns MX-4471-C, which is a different component that does not fit. It also struggles with negation, numeric filters and freshness, and it will always return something, which means it never tells you honestly that you have nothing relevant.
Hybrid retrieval: the practical architecture
Run both, fuse the results, then rank deliberately.
- Lexical retrieval using BM25 or similar for exact and near-exact matching, with fielded boosts.
- Vector retrieval using embeddings over titles, descriptions and chunked content for conceptual matching.
- Fusion using reciprocal rank fusion or weighted scores to merge candidate sets.
- Re-ranking with a cross-encoder or a business-rules layer applied to the top candidates only, because re-ranking is expensive and only the head of the list matters.
- Filtering applied as hard constraints — availability, permissions, locale, price band — never as soft signals.
| Query type | Best served by | Notes |
|---|---|---|
| Exact SKU or code | Lexical | Vector similarity is actively harmful here |
| Descriptive need | Vector | "waterproof jacket for cycling in winter" |
| Brand plus attribute | Hybrid + filters | Attributes must be structured, not in prose |
| Support question | Vector + grounded answer | Cite the source document |
| Misspelling | Lexical with fuzzy matching | Cheaper and more predictable than embeddings |
The part everyone skips: data quality
No retrieval architecture survives bad source data. Before tuning a single weight:
- Ensure every searchable item has a clean title, a real description and structured attributes in fields rather than buried in marketing prose.
- Build a synonym dictionary from your own zero-result and refinement logs — not from a generic list. Your customers' vocabulary is in your query logs already.
- Normalise units, sizes, colours and naming conventions. "XL", "Extra Large" and "x-large" must resolve to one value.
- Chunk long content sensibly for embedding — by heading section, with the heading retained in the chunk so context is not lost.
Ranking signals beyond relevance
Textual relevance decides which results are eligible. Business signals decide the order. Blend in availability, margin or strategic priority where appropriate, popularity and conversion history, freshness for editorial content, and personalisation only where you have genuine signal. Keep these as explicit, inspectable weights: when a merchandiser asks why a product ranks third, "the model decided" is not an answer that survives a meeting.
Evaluation, or you are just guessing
Relevance work without measurement becomes an argument between whoever spoke last. Build the smallest useful evaluation harness:
- Take your top 200 queries by volume plus 100 sampled long-tail queries.
- Have humans judge the correct results for each. Yes, manually. Once.
- Compute NDCG and precision at 5 and 10 on every configuration change.
- Track operational metrics continuously: zero-result rate, refinement rate, search abandonment, and search-to-conversion.
Then A/B test. Offline metrics catch regressions; only live traffic tells you about revenue.
Adding AI answers without wrecking trust
Generated answers belong on informational queries, grounded strictly in your own indexed content, with visible citations and a clear fallback to results when confidence is low. Never let a generated answer replace the result list for transactional queries — a user comparing four products wants four products, not a paragraph of opinion. And never generate from content the user is not permitted to see; retrieval must respect access control before generation, not after.
A 60-day improvement plan
Weeks 1–2: instrument search properly. Log queries, result counts, clicks, positions and abandonment. Read the top 200 queries yourself — this is always the most revealing two hours of the project.
Weeks 3–4: fix data and add synonyms from real zero-result logs. This alone usually removes most zero-result traffic.
Weeks 5–8: add vector retrieval alongside lexical, fuse, build the judged evaluation set, and A/B test. Only then discuss generated answers.
Search is not a feature you install. It is a product with its own metrics, its own owner and its own backlog — and it happens to sit on the highest-intent traffic you have.
Frequently Asked Questions
What is hybrid search?
Hybrid search combines traditional keyword (lexical) matching with vector similarity search, then merges the results. It captures exact matches such as product codes as well as conceptual matches where the user's words differ from your content.
Do we need a vector database?
Not necessarily. Several established search engines and managed platforms now support vector fields alongside inverted indexes, which is usually simpler to operate than running a separate vector store.
Will semantic search fix bad site search?
Only partly. Most poor site search is caused by bad data — missing attributes, inconsistent naming, no synonyms and no merchandising rules. Semantic retrieval on bad data returns confidently wrong results.
How do we measure search quality?
Track zero-result rate, click-through position, search-to-conversion rate, refinement rate and abandonment after search. Then build a judged relevance set for your top queries so changes can be evaluated rather than debated.
Should search answer questions with AI or just return results?
Both, separated clearly. Generated answers work for informational queries when grounded in your own content with citations. Transactional queries should return ranked results, because users want to choose, not be told.