Stance Detection in Prediction Markets: Addressing Imbalanced Trader Commentary via Counterfactual Augmentation and Market Context
Thomas Mbrice
Read on arXiv →
Key claim
Market context greatly improves stance detection performance.
This paper explores stance detection in prediction market comments, revealing that market context significantly improves recall for opposing stances. The optimal augmentation strategy is found to be 50% synthetic samples, which enhances performance without degrading it.
In plain English
This paper explores stance detection in prediction market comments, revealing that market context significantly improves recall for opposing stances. The optimal augmentation strategy is found to be 50% synthetic samples, which enhances performance without degrading it.
This work introduces stance detection in prediction market commentary, a novel application of NLP techniques.
The study employs rigorous ablation studies and provides clear evidence for its claims.
Deep reliability assessment
The methodology supports an internal ablation claim: adding market-question context improves RoBERTa stance detection on this small Polymarket dataset, and LLM counterfactual augmentation has dose-dependent effects. Broader claims about general prediction-market stance detection, cross-platform robustness, and mechanistic explanations from attention maps are less well supported.
Reproducibility
Yes: the paper states that code, data, and trained model checkpoints are available via a Google Drive link in the footnote, but no GitHub repository is mentioned.
Discussion questions
- 1.Does prepending the market question truly teach stance, or does it mainly let the model exploit market-specific lexical shortcuts in a small dataset?
- 2.For builders, is comment-level stance detection useful enough beyond market price, volume, and trader history to justify production deployment?
- 3.What happens if the model is evaluated on new markets, future Polymarket slang, or another prediction-market platform with no overlap in topics?
Key figure
The key setup is a RoBERTa-base stance classifier whose input is optionally augmented by prepending the market question to a short trader comment, with optional LLM-generated Pro-to-Anti counterfactual samples added during training.