SAGE: Recovering the Vulnerability Signals Hidden Inside LLMs
What if an LLM actually “sees” the vulnerability, but the signal is simply too faint to affect its final decision? We identify a fundamental bottleneck in LLM-based vulnerability detection called Signal Submersion: vulnerability-related features are activated inside the model, yet become numerically overwhelmed by dominant functional semantics as information flows through deeper layers. To recover these hidden signals, we develop SAGE, which probes intermediate representations and uses task-conditional Sparse Autoencoders to isolate and amplify vulnerability-relevant features while keeping the LLM backbone frozen. The result: up to 12.7× higher internal signal-to-noise ratio, 318% MCC gains on unseen distributions, and a 7B model outperforming 34B baselines. Sometimes, better vulnerability detection does not require a bigger model—it requires helping the model hear the signal it already has.
一段几百行的代码里，
可能只有那么一两处细微差别真正决定了
“安全”还是“有漏洞”。
问题是，正常代码实在太多了——
函数功能、变量关系、控制逻辑全在疯狂刷存在感，
真正关键的漏洞信号反而像在人群里小声喊了一句：
“这里有 Bug！”
然后……被淹没了。🙂
我们发现，这并不一定意味着 LLM
“完全没学会漏洞”。
很多时候，漏洞相关特征其实已经在模型内部出现，
只是到了深层之后，
被大量正常功能语义压得越来越弱——
我们把这个现象叫做
Signal Submersion。
所以 SAGE 干的事情不是继续无脑
“把模型做大”，
而是深入模型内部，
把这些快要沉底的漏洞信号
捞出来、分离出来，再放大。
最终内部信噪比最高提升
12.7×，
unseen distribution 上 MCC 最高提升
318%，
一个 7B 模型甚至可以超过
34B baseline，
并在 13 种编程语言上展现跨语言能力。
与其一味让模型 “长更大的脑子”， 不如先帮它把真正重要的那句话 听清楚。
