CH-01 · EXP-014live
0mst →
Iterating·Mistral 7B · QLoRA · pgvector
Can a 7B model ground its own citations?
- ›Question
- Can a small open-source model be trained to refuse to answer when retrieval confidence drops below a threshold — instead of hallucinating?
- ›Experiment
- Fine-tuned Mistral-7B with QLoRA on 4k synthetic (query, grounded-answer, refusal) triplets. Hybrid retriever (BM25 + dense) feeds top-8 chunks with calibrated scores. Generator forced to emit either a cited answer or a structured REFUSE token.
- ›Result
- Refusal precision climbed from 0.61 → 0.89. Hallucination rate dropped 3.4×. Cost: 18% increase in latency and ~600 MB extra VRAM at inference.
- ›What I learned
- A small model can learn citation discipline, but only if the training data includes plenty of 'should have refused' examples. The generator over-trusts context without them.