논문리뷰

[논문리뷰] Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs (ICLR 2026)

syveany 2026. 9. 17. 22:19

오랜만에 논문리뷰. 

VLM 관련된 논문을 몇 개 읽어볼까 한다. 오랜만에 VLM 자체에 대한 논문들을 보니 재밌다.

 

ICLR 2026 논문이다.

https://proceedings.iclr.cc/paper_files/paper/2026/file/76818d8d85e05e45ce3a16a8468619d1-Paper-Conference.pdf

 

 

Abstract

 

 

 

1. Introduction

 

 

2. Dissecting Text and Visual Attention Dynamics in VLMs

 

 

2.1 RQ1: How does attention transition between text and visual tokens?

 

 

2.2 RQ2: Which image regions do different layers attend to?

 

 

2.3 RQ3: Do VLMs perceive visual evidence when giving wrong answers?

 

 

2.4 RQ4: Why does "Seeing but not believing" happens?

 

 

3. Simple Inference - Time Visual Evidence Augmentation

 

 

3.1 Visual evidence attribution layer profiling

 

 

3.2 Inference - Time visual evidence attribution

 

 

3.3 Attention-guided Visual evidence highlighting

 

 

4. Experiment results and analysis

 

 

5. Related works

 

 

6. Conclusions