Can Your Uncertainty Scores Detect Hallucinated Entity?

To mitigate the impact of hallucination nature of LLMs, many studies propose detecting hallucinated generation through uncertainty estimation. However, these approaches predominantly operate at the sentence or paragraph level, failing to pinpoint specific spans or entities responsible for hallucinated content. This lack of granularity is especially problematic for long-form outputs that mix accurate and fabricated information. To address this limitation, we explore entity-level hallucination detection. We propose a new data set that annotates hallucination at the entity level and introduce evaluation metrics to quantify the entity-level performance of hallucination detection. Our experimental result shows that current uncertainty-based approaches tend to over-predict hallucinations and fail to correctly locate hallucinated content. Through qualitative analysis, we identify the relationship between the tendency of hallucination and linguistic properties, and suggest two directions to improve the performance of uncertainty-based hallucination detection approaches.