This research paper investigates the phenomenon of hallucinations in large language models (LLMs), focusing on distinguishing between two types: hallucinations caused by a lack of knowledge (HK-) and hallucinations that occur despite the LLM having the necessary knowledge (HK+). The authors introduce a novel methodology called WACK (Wrong Answers despite having Correct Knowledge), which constructs model-specific datasets to identify these different types of hallucinations. The paper demonstrates that LLMs’ internal states can be used to distinguish between these two types of hallucinations, and that model-specific datasets are more effective for detecting HK+ hallucinations compared to generic datasets. The study highlights the importance of understanding and mitigating these different types of hallucinations to improve the reliability and accuracy of LLMs.

Ver todos los episodios