The Geometry of Updates: Fisher Alignment at Vocabulary Scale
John Sweeney
Read on arXiv →
Key claim
FisherSketch enables efficient source selection for LLMs.
In plain English
Imagine you're working with large language models (LLMs) that need to handle different scientific data types, like chemical structures or genetic sequences. You want to select the best training sources for your model, but the challenge is that these sources might share the same vocabulary but have different prediction targets. This can lead to a situation where the usual metrics for evaluating model performance become unhelpful, especially when the models seem similar but actually perform differently on specific tasks. This is what's called the activation-dark regime, where you can't easily tell which model is better based on their representations alone.
Currently, people might rely on complex metrics that are hard to compute at scale, or they might miss important differences between models because they focus too much on representation similarity. This can lead to poor source selection and ultimately affect the performance of the models in real-world applications.
What this paper introduces is a more efficient way to assess how well different models can transfer knowledge from one task to another, using a method called FisherSketch. Instead of needing to compute a full Fisher matrix, which can be computationally expensive, FisherSketch estimates the necessary alignment directly in a single pass. This means you can quickly determine how well different models are aligned in terms of their updates, even when their representations look similar. The practical takeaway is that this method not only helps in selecting the right training sources but also provides insights into the underlying similarities and differences between tasks, which can be crucial for building effective LLMs in scientific domains.
The paper introduces a new method for source selection in LLMs that addresses specific challenges in scientific domains.
The claims are supported by experimental results and validations, though the scope could be broader.
Deep reliability assessment
The theory strongly supports the narrow claim that activation similarity alone cannot identify shared-output head update alignment, and the reported experiments support FisherSketch as a useful training-free source-selection signal in activation-dark settings. The broader implication that compact FisherSketch signatures are generally transfer-relevant for LLM families is less settled, because the strongest quantitative evidence is concentrated in verbalizer-shift and small scientific-sequence proof-of-concept settings.
Reproducibility
No open-source code or repository is mentioned in the provided abstract, introduction, results, limitations, or conclusion excerpts. The paper describes experimental settings such as Llama-3.1-8B verbalizer shift, 100-domain source selection, and nine SMILES domains, but no dataset release or exact reproducibility link is provided in the supplied text.
Key figure
The key architectural idea is a streaming pipeline that takes frozen activations and logit-space errors, lifts them into joint activation-error random features, averages them into a compact task signature, and compares tasks by cosine similarity as a sketch of head Fisher alignment.