Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries
Tânia Carvalho, Maxime Cordy
Read on arXiv →
Key claim
Attention mechanisms in models can leak sensitive training data.
In plain English
Imagine you're using a model trained on synthetic data to make predictions based on user inputs. You might think this is safe since the data isn't real, but it turns out that when you provide context examples during inference, the model can inadvertently reveal whether specific records were part of its training data. This is a serious privacy issue because it allows attackers to determine if they have a 'membership' in the training set, which is what's called a Membership Inference Attack (MIA).
Current defenses often rely on adding noise or retraining the model, which can degrade performance. However, this paper introduces a new attack method called AMIA that exploits the attention patterns of transformer models to reveal membership signals more effectively than traditional confidence-based methods. The authors also propose a defense inspired by $k$-anonymity principles that reduces the risk of leakage without sacrificing much predictive accuracy. In practice, this means that if you're building applications that rely on tabular models, you need to be aware of these vulnerabilities and consider implementing the proposed defense to protect sensitive information.
The paper introduces a new attack method that reveals privacy vulnerabilities in tabular foundation models.
The claims are supported by experimental results demonstrating the effectiveness of the proposed attack and defense mechanisms.
Deep reliability assessment
The methodology supports the claim that, for the evaluated tabular foundation models and datasets, attention concentration during in-context inference is a usable membership signal under a grey-box attacker who can inspect attention matrices. The broader claim that this is a general privacy risk for tabular FMs is plausible but overextended unless tested under stricter black-box access, more deployment-like context construction, and more diverse real sensitive datasets.
Reproducibility
No code repository is mentioned in the provided paper text. The experiments use six tabular datasets and several named models, including Locations, Dropout Success, TabPFN, Real-TabPFN, TabICL, and TabDPT, but the supplied excerpt does not provide enough implementation detail to fully reproduce the results.
Key figure
The key diagram likely shows a tabular FM receiving labelled context rows and a query, extracting transformer attention patterns over context records to compute an AMIA membership-risk score, then optionally applying targeted k-anonymity-style aggregation to high-risk context representations.