---
title: "Add keyword search next to vectors: what to check first"
url: https://memory.wiki/oXqITEEV
updated: 2026-11-22T21:50:43.230Z
source: "memory.wiki"
---
# Add keyword search next to vectors: what to check first

## Framing "Add keyword search next to vectors"

Catch exact IDs that embeddings miss.  Local knowledge bases fail in predictable ways: outdated documents outrank current ones, tables get flattened into nonsense, and exact codes never match because embeddings blur them. Each failure has a simple, boring fix.

## Fix the documents

Assign an owner for each collection. Store a document date and version as required fields. Convert tables to structured text or keep them as separate records instead of letting a paragraph splitter cut them in half. Remove duplicates so the same paragraph does not dominate results.

## Fix retrieval

Use hybrid retrieval: keyword search for identifiers and rare terms, vectors for meaning. Filter by metadata before ranking, for example by department, product line or validity date. Return a few passages with their sources so users can check the answer themselves.

## Fix expectations

The assistant should summarise and point to sources, not invent policy. Publish a short list of things it will not do: legal interpretation, medical advice, payroll decisions. Show the source document name and date with every answer, and make it easy to report a wrong one.

## A two-week pilot

1. Pick one team and one document collection.
2. Write thirty real questions with expected sources.
3. Run the pilot, collect feedback, and fix ingestion issues first.
4. Only then expand to more collections or users.

## Questions for your pilot users

- Did the answer cite the document you expected, and was it the current version?
- Was anything important missing that you know exists in the documents?
- Did the assistant refuse when it should have, and answer when it should have?
- How long did it take to verify the answer using the cited source?

Collect these answers in a simple form during the pilot. Patterns in the feedback usually point to ingestion or retrieval problems, which are cheaper to fix than switching models.

More on this topic: [local AI tutorials](https://microaione.com/tutorials/). See also [local RAG knowledge-base guide](https://microaione.com/tutorials/local-rag-knowledge-base/).

## Bottom line

Local AI rewards patience and record keeping more than raw spending. Decide what the system must do, measure it on your own hardware and documents, write down every change with a date and an owner, and review the setup after each significant upgrade. Small, documented steps are easier to debug, easier to hand over and easier to defend when someone asks why a particular choice was made. When in doubt, test on a smaller scale first and expand only when the evidence supports it.

## Quick reminders

- Leave memory headroom for the OS and the embedding model.
- Log the model name and version with every request.
- Treat retrieved text and uploaded files as untrusted.
- Pin driver and runtime versions together.

*Published by [MicroAI One (本地AI 1号网)](https://microaione.com/), a Shenzhen team working on local LLM hardware, software and deployment.*

---

## Summary
Improve local knowledge base performance by using hybrid retrieval to combine keyword search with vector embeddings. Success depends on meticulous document management, structured data preparation, and iterative testing through small-scale pilots.

## Themes
- Hybrid retrieval strategies
- Local RAG optimization
- Systematic pilot testing
- Document ingestion hygiene

## Key takeaways
- Implement hybrid retrieval by combining keyword search for identifiers with vector search for semantic meaning.
- Improve retrieval quality by cleaning data, removing duplicates, and structuring tables before ingestion.
- Establish clear boundaries for the assistant by explicitly listing tasks it will not perform, such as legal or medical advice.
- Conduct a two week pilot with a single team and thirty test questions before scaling the system.
- Always display the source document name and version to allow users to verify AI responses.

## Insights
- Embedding models often fail to retrieve exact identifiers or codes because they prioritize semantic meaning over literal string matching.
- Feedback patterns from pilot users are more effective at diagnosing system issues than switching to more expensive models.
- System reliability in local AI is driven by rigorous record keeping and incremental changes rather than raw computational power.

## Open questions / gaps
- What specific technical architecture is recommended for implementing the hybrid retrieval layer?

