RSPC: A Benchmark for Modeling Stress and Psychiatric Conditions in Digitally Mediated Relationships using Psychiatrist Annotations
Parmitha Vangapandu, Sai Ganesh Mokkapati, Sathwik Narkedimilli, MSVPJ Sathvik, Timothy Liu, Simon See, Johannes C. Eichstaedt
Read on arXiv →
Key claim
Relational context significantly impacts mental health modeling.
In plain English
Imagine trying to understand mental health issues like anxiety and depression, but only looking at them in isolation, without considering the relationships that might influence them. This is a common approach, but it misses a lot of important context. For instance, someone might feel anxious not just because of their own thoughts, but also due to stress in their relationships. This is where the new research comes in. It uses Reddit posts about long-distance relationships to gather insights on mental health, focusing on how relational dynamics can trigger or exacerbate these conditions. They created a dataset called the Relational Stress and Psychiatry Corpus (RSPC), which includes 1,799 posts annotated by psychiatrists for various mental health categories and relational stressors. By benchmarking several transformer models on tasks like classifying disorders and detecting relational triggers, they found that different models have unique strengths. For example, Claude-3-Haiku performed best in classifying mood disorders, while GPT-4o excelled in detecting relational triggers. This work shifts the focus from viewing mental health as an individual issue to understanding it within the context of relationships, which could lead to more effective interventions and support systems.
The paper introduces a new dataset and benchmarks for understanding mental health in relational contexts, extending existing NLP applications.
The findings are supported by multiple model evaluations and clear performance metrics across tasks.
Deep reliability assessment
The methodology supports RSPC as a small, clinically annotated Reddit benchmark and shows that relational stressor labels can be modeled with modest Macro-F1 on this dataset. It does not yet support strong claims about psychiatric diagnosis, cross-cultural generalization, or real-world mental health screening, especially because labels are inferred from single English Reddit posts in one relationship context.
Reproducibility
No public code or dataset URL is mentioned in the provided paper text. The authors describe model choices, hyperparameters, metrics, deterministic LLM prompting, and say prompts/implementation details are in Appendix 7, but the excerpt does not show an actual release link.
Key figure
No Figure 1 or architectural diagram is included in the provided excerpt, so the key visual cannot be described from the supplied text.