The retrieval pipeline returned chunks that mentioned vaccination schedules but didn't contain the actual schedule data. The model saw evidence that was on-topic but empty of the answer - and correctly refused. The root cause wasn't the model, the reranker, or the query. It was the chunking strategy splitting tables and structured data away from the prose that named them.
Read articleSame question, same branch, same retrieval, same 10 evidence chunks with identical rerank scores - and one deployment refused while another answered with full clinical detail. The root cause wasn't retrieval. It was the LLM making a borderline judgment call on evidence that covered half the question well and the other half barely at all.
Read articleVector search finds the right chunk but ranks it badly and waves through near-garbage that's vaguely on-topic. Reranking fixes that with a second model - a cross-encoder that reads the query and each chunk together instead of comparing two frozen vectors. The whole trick is affording an expensive judge by only running it on the cheap stage's survivors.
Read article