Every RAG tutorial stops at the retrieval call. In production, retrieval is the part that usually works — it returns passages related to the query — and the failure is that the passages do not answer it. An agent given five plausible paragraphs will write a confident synthesis regardless.
Two servers in the directory cover the retrieval layer: Qdrant and Pinecone. They are similar in shape and different in where the data lives.
The short answer
Treat similarity search as a candidate generator, not a source of truth. Return passages with scores, require the agent to cite one, and evaluate the result rather than the embedding model. The server is the least interesting decision in the stack.
What the servers actually do
Both Qdrant and Pinecone expose the same three operations: search a collection by vector or text, filter by metadata, and return payloads with scores. The differences are operational — self-hosted versus hosted, namespaces, and how much index management the tool exposes.
What neither does is judge relevance. A score of 0.82 means the vectors are close; it does not mean the passage answers the question. That judgement has to be a separate step, which is exactly what a search evaluation skill does.
The three failure modes
| Failure | What it looks like | Fix |
|---|---|---|
| Chunk too small | The answer spans two chunks and neither is retrieved alone | Chunk by section, keep parent reference |
| Chunk too large | The passage is relevant but diluted, and the model quotes the wrong part | Smaller chunks, or a re-rank |
| Filter missing | Results come from the wrong tenant or the wrong product | Metadata filter as part of the tool call |
The third one is not a quality problem; it is a permissions problem, and it is the one that causes incidents rather than disappointing answers.
The evaluation loop
A retrieval system needs twenty questions with known answers before it needs a better embedding model. For each, check two things: did the expected passage appear in the top five, and did the final answer cite it. Those are different failures — retrieval and synthesis — and separating them tells you where to spend effort.
The RAG Eval skill does exactly that. Without it, every change is judged by vibes on one question.
Namespace and tenancy
A vector index usually holds more than one corpus. If the agent's tool call can reach every namespace, then a metadata filter is the only thing between two customers' documents. Put the namespace in the tool configuration rather than the prompt, and verify it with a query you know should return nothing.
What an agent should be allowed to write
Reads are safe; index writes are not. An agent that can upsert vectors can pollute the retrieval corpus, and the pollution will look like an answer. Keep ingestion in a reviewed pipeline and give the agent search only.
Frequently asked questions
Do I need a vector database at all? For a small corpus, keyword search plus a re-rank is often enough. Vectors pay off at scale and for semantic questions.
Qdrant or Pinecone? Self-hosted control versus hosted convenience. The retrieval contract is the same either way.
How do I know it works? Twenty questions, two checks each — see giving an agent web search for the same pattern applied to web results.
Last checked: 12 November 2026.
Related: search skills vs search APIs, MCP servers, and RAG Eval.
