LKValues: Aligning Large Language Models with Sri Lankan Societal Values

Imagine you're developing a language model that needs to understand and respect the diverse cultural values of a specific region, like Sri Lanka. Currently, many large language models (LLMs) are trained primarily on Western norms, which can lead to misunderstandings and misrepresentations of local values. This is particularly problematic in multilingual societies where cultural nuances are critical for effective communication. The existing benchmarks often fail to account for these local dynamics, resulting in models that may not perform well or align with the values of the communities they serve. This is what's called cultural bias, and it can manifest in various ways, such as inappropriate responses or a lack of understanding of local contexts. To address this issue, LKValues was created as a resource suite specifically designed for Sri Lankan value alignment. It combines insights from a trilingual survey with local constructs to identify 40 key societal values that resonate with the Sri Lankan populace. The authors also developed LKvaluesIT, a corpus of 150,000 scenario-based instances in Sinhala and English, along with LKvaluesBench, a benchmark for evaluating LLMs against these values. By fine-tuning several open-weight models with this new data, they found that while larger models still struggle with cultural alignment, the fine-tuning process significantly improved their performance in both English and Sinhala. This work not only enhances the understanding of Sri Lankan values in AI but also provides a replicable framework for other low-resource, culturally diverse contexts.

The introduction of a culturally specific value alignment resource for Sri Lanka is a meaningful extension in the field.

The evaluation of multiple LLMs with a new benchmark demonstrates solid methodology and results.