Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection

Imagine you're building a system to automatically detect vulnerabilities in code. You'd want it to be reliable, but what if the model's judgment is swayed by how the code is presented? That's a real concern because recent studies show that even advanced models can fall prey to the same cognitive biases that affect human decision-making. For instance, if a model sees a piece of code attributed to a well-known author, it might rate it as safer than it actually is — this is called the halo effect. Similarly, how a task is framed can change the model's assessment, leading to what's known as the framing effect. And if the model has seen previous analyses, it might anchor its judgment to those results, even if they were incorrect. These biases can lead to serious misclassifications, where a model might declare a piece of code safe when it’s actually vulnerable, especially if the vulnerability requires deeper understanding rather than just pattern recognition. This paper introduces a framework to investigate these biases systematically, testing eight different large language models across various programming languages. The findings are concerning: the models showed significant susceptibility to these cognitive biases, with the framing effect being the most impactful. This means that if you're relying on LLMs for security assessments, you need to be aware that their judgments can be easily influenced by context, potentially leading to overlooked vulnerabilities.

This work systematically explores cognitive heuristics in LLM-driven vulnerability detection, a previously unexamined area.

The study evaluates multiple models across different programming languages and provides quantitative analyses, supporting its claims.