Faithfulness

TL;DR for operators A RAG system can return the right answer, attach a source that genuinely supports that answer, and still leave one critical question unresolved: did that source actually influence the model’s answer generation? A mechanistic study of Llama-3.1-8B-Instruct finds that inline citation behavior is not controlled by one dedicated citation feature. It emerges from a distributed sequence of attention heads and MLPs that includes early entity enrichment, matching between document and question entities, mid-layer processing, and late aggregation that shifts the model toward emitting a citation marker rather than ending the sentence.1 ...

Opening — Why this matters now The original business pitch for retrieval-augmented generation was wonderfully simple: connect the model to your documents, ask questions, get grounded answers. No need to retrain the model. No need to wait for the next foundation-model release. Just give the chatbot some files and let productivity bloom. ...

TL;DR for operators FaithAct is useful because it changes the unit of control. Instead of asking whether a multimodal model’s final answer is correct, it asks whether each intermediate claim is supported by the image before that claim is allowed to steer the next step.1 That is a more operational target. Accuracy tells you whether the system arrived somewhere acceptable; perceptual faithfulness tells you whether it drove through the road or hallucinated a bridge. ...