United States

Large language models (LLMs) perpetuate cultural stereotypes not only through biased associations but through systematic omission and orthographic erasure of underrepresented languages. We present empirical evidence of two compounding failure modes affecting Northeast Indian languages: (1) entity-level invisibility, where state-of-the-art NER systems score F1=0.000 on culturally critical named entities such as Khasi surnames, Garo festivals, and tribal names; and (2) orthographic corruption, where LLM tokenizers corrupt semantically meaningful diacritics (ï, ñ) and the Garo morpheme boundary marker (U+00B7) at rates of 18.8–50% across four of five evaluated models. Drawing on NortheastNER (F1=0.964, six entity categories, XLM-RoBERTa-base) and a systematic tokenization study across Khasi and Garo, we argue that stereotype-by-omission constitutes a distinct and measurable harm to indigenous language communities. We further show that a custom multilingual tokenizer achieves 26–50% token reduction over five baseline LLMs, demonstrating that culturally grounded infrastructure can partially remediate these failures. Our findings call for cultural representation audits as a standard component of multilingual NLP evaluation.

ACL 2026

Stereotyped by Silence: How LLMs Erase Northeast Indian Languages Through Omission and Orthographic Corruption

indigenous nlp

northeast india

orthographic erasure

stereotype-by-omission

cultural representation

low-resource languages

tokenization

named entity recognition

## Welcome to 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics!
This year, the conference is in Mexico City. NAACL was actually already planned for Mexico City in 2021, but due to the pandemic the entire conference was moved online. This year, finally, we get to go! So it is my sincere pleasure to welcome you to Mexico City, whether in person or virtually. Having the conference in Mexico City is a good opportunity to emphasize that NAACL is our flagship conference for ACL members not only in North America but also in Central and South America, even though NAACL has been bearing “North” in its name. At this year’s conference, we have a theme to match, with a theme track on the Languages of Latin America to showcase the linguistic diversity of the region.
 
The opportunity to present at NAACL should not depend on a researcher’s travel budget, or their family status. This is why it is so important to make virtual participation at NAACL as good an experience as possible – but we want to also provide a good experience for in-person participants. As a community, we are still working out the best way to do that. This year at NAACL, we are trying out a big virtual poster session ahead of the conference, with the hope that this will make make for a lively and interactive experience. At the same time, we are reducing virtual oral presentations, which seem to be particularly tricky to make to work well. A big thanks to the NAACL program chairs and to Luciana Benotti for all their ideas and work to improve the virtual experience. And participants, virtual as well as in-person: Please let us know what worked for you and what didn’t, so we can continue to improve hybrid conferences. 
 
I have been lucky to work with many amazing people. Without their insight, dedication and patience, and without the many hours of work they put in, NAACL would not have been possible. A huge thank you to the program chairs Helena Gomez, Kevin Duh, and Steve Bethard – you are the best! 
 
Finally, I would like to thank all authors, invited speakers and panelists, area chairs and reviewers, the volunteers organizing and chairing sessions, and all attendees, in-person and virtual. Thank you for helping us make NAACL 2024 come to life. 
 
Welcome and hope you all enjoy the conference! 
Katrin Erk 
The University of Texas at Austin 
NAACL-2024 General Chair 
*You can read the full Welcome message in the [Conference Handbook (downloadable)](https://drive.google.com/file/d/1H1NvW0VASQjkSCYgw3mr-3l4yYDSxz6B/view?usp=sharing)*

To access the event page you need to register [**here.**](http://acl.swoogo.com/naacl2024) 
Your access to the event page is limited based on your registration type. If you registered for workshops only, you will gain access to full workshops content on the day of the workshops program.

Please register!

NAACL 2024

2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics

workshop paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Stereotypes associate a group (G) with an attribute (A), often through broad generalizations like "Gs are A". In the age of large language models, many existing datasets and methods measure bias by considering changes in $P(A|G)$ between different social groups ($G_{alt}$). Motivated by literature on generics, we argue that constructing minimal pairs along the group dimension is not enough: explicitly considering the alternate attribute ($A_{alt}$) is necessary for a robust score. We first show how to adapt existing log-probability metrics to this setup, but we find that aggregating them across different stereotypes is inherently flawed. We propose a new metric representing the mutual information (MI) between the social groups and attributes. We argue that it is better suited for aggregation across individual points, allowing for robust comparison of stereotype strength across different languages and models. We demonstrate this on a set of paired English and Russian stereotypes from the BiasShades dataset.

The Missing Minimal Pair: Mutual Information for Stereotype Evaluation in LLMs

Real-world social judgments often depend on local norms, values, and situational expectations. We study whether multilingual language models use such cultural context robustly, or whether their judgments are overly sensitive to surface cues such as prompt language, country labels, and externally supplied contextual descriptions. Building on NormAd, a benchmark for cultural norm adherence, we keep the underlying scenario fixed while perturbing language, context granularity, and contextual consistency. We evaluate 12 models from four model families: Cohere Command, Cohere Tiny-Aya, Alibaba Qwen, and Google Gemma. Across three countries, five language conditions, and 55,000 raw generations, models benefit most from explicit norm descriptions. Adding these descriptions improves accuracy by 17.8 percentage points over country-only prompts. In contrast, adding externally supplied rules that are not applicable to the scenario is the largest harmful perturbation, reducing accuracy by 24.5 points. Language changes produce 9–14% prediction flips, but these flips are directionally balanced. These results show why flip rate alone is insufficient: it does not distinguish between correct-to-wrong (C→W) and wrong-to-correct (W→C) transitions. We therefore report cultural robustness using signed accuracy change, C→W/W→C decompositions, and perturbation-specific vulnerability alongside clean accuracy.

Robustness of Cultural Norm Reasoning Under Language and Context Perturbations

As language models scaled to internet data, exhaustive curation became infeasible and stereotype learning became an expected side effect of training. A common response is post-training local editing: identify an internal direction linked to harmful behavior and try to remove it. However, the causal effectiveness of this method is unclear. In this paper, we present a causal audit of local stereotype editing across seven open-weight LLMs using matched inject-versus-remove interventions, component-level diagnostics, and cross-source transfer stress tests. The main result is asymmetric controllability: stereotype preference is easier to increase than to reliably decrease. In matched tests, score-level inject-minus-remove asymmetry is significant in 5/7 models, and margin-level asymmetry is significant in 7/7. Supporting diagnostics show model-dependent sign reliability, suppressor contamination in ranked intervention sets, and no reliable large-effect cross-source transfer under current settings. Overall, localization provides control leverage but not a debiasing guarantee; deployment-facing claims should be validated with direct causal intervention and transfer stress tests.

Easy to Add, Hard to Erase: Causal Evidence on Stereotype Editing in LLMs

MLLM-as-a-Judge is conventionally validated by agreement with human annotations, but agreement against a culturally heterogeneous pool is pool-dependent: it silently conditions on whose judgments the aggregate represents. We introduce VOIR DIRE, a multimodal benchmark of 626 culturally paired image–prompt items spanning U.S. and mainland Chinese contexts across food, fashion, and architecture, with annotator pools that are within-pool reliable (α = 0.86/0.74) but cross-pool divergent on evaluation (Q1 r = −0.12). Across six MLLMs, the bias decomposes into two failures: a positivity-floor calibration failure (compressed scale use) and an orientation failure (default to one cultural norm). On this corpus, where contested items are sampled to split the two pools, the floor mechanically validates the more permissive Chinese reading; persona prompting partially recovers calibration, but the orientation residual survives, evidence the tilt is not reducible to scale compression. Reference pool in-context demonstrations deepen the orientation residual and inflate the high end rather than restoring use of the low end. Model origin adds a small additive tilt (~ 0.10 MAE) that is approximately invariant under demonstration. We recommend reporting alignment against each reference pool separately and treating cross-pool divergence as a judge property.

Jury Duty: Uncovering Latent Cultural Biases in MLLM-as-a-Judge Under Ambiguity

We present an open-source measurement protocol for stereotype interpretation that quantifies how users translate or interprets provocative discourse without assuming a normative direction. Building on Deleuze and Guattari's rhizomatic framework, we operationalize three modes of semantic movement —Reaffirm, De-signify, and Escape (RDE)— through an abstract-machine operator detector that combines transparent linguistic patterns (526 patterns across 8 languages) with optional contextual embeddings. The protocol is direction-agnostic: it measures equally well a user who reproduces their own semantic territory and one who departs from it, capturing diasporic, assimilationist, and escape trajectories that English-centric, Chomskyan-hierarchical taxonomies obscure. We demonstrate the protocol on five extreme user profiles (Russian conservative, Russian diaspora, trans Russian exile, Mexican malinchista, Mapuche speaker), each producing coherent and distinct RDE signatures. Deployed in a free-tier web service, the protocol enables both individual reflective use and corporate calibration of tolerable territoriality ranges for personnel engaged in intercultural translation and interpretation tasks.

Measuring Semantic Flow Without Direction: A Rhizomatic Protocol for Stereotype Translation in Cross-Cultural Language Technology

Socio-cultural stereotypical bias is an important consideration in the development and deployment of NLP systems. It is however often considered only at the national level, despite rich subnational socio-cultural structures. We present AmchiBias, the first benchmark for enmeasuring socio-cultural stereotypical bias for the Indian state of Goa with its unique historically multicultural setting. It covers various Goan identity groups and comprises 313 minimal pairs across eight sociodemographic dimensions in both English and Devanagari Konkani. 
We then evaluate stereotypical bias in five multilingual encoder models on this benchmark. We find near-chance scores in Konkani, reflecting language incompetence for general multilingual models and a lack of Goan cultural competence for Indian language models. 
Queried in English, models with a stronger Indian language coverage show higher bias for pan-Indian groups than hyperlocal Goan groups. This suggests the English signal reflects pan-Indian pretraining associations rather than genuine Goan cultural knowledge. Our findings highlight a critical gap in low-resource multilingual NLP evaluation for hyperlocal community identities.

AmchiBias: Measuring Stereotypical Bias in Goan Identity Groups with a Minimal Pair Dataset in English and Konkani

While the large language models (LLMs) are considered to be largely proficient in translation, especially among relatively high-resource languages, we posit that they are inadequate at translating colloquial and informal cultural usage. Translation of cultural involves localization at various levels of granularity, identifying equivalents for comparable values and concepts, and elaborating on fundamental conceptual differences. In this work, we show that even for a narrow instantiation of this task that focuses only on proverb translation, LLMs exhibit significant asymmetry in translating cultural terms used by English speakers into other languages compared to the translation of proverbs in the other (even high-resource) languages to English. We compare several different families of language models and observe this asymmetry even extends to model-based translation evaluation metrics like COMET. Further analysis reveals that LLMs typically tend to be literal in their translations and often fail to find distinct but equivalent concepts across languages as often as human translators, thus raising deployment concerns around differential service across language users worldwide.

Unveiling Asymmetry in Large Language Models' Ability to Translate Culture

Foundation models pretrained on internet-scale data benefit substantially from post-training alignment with human preferences. However, existing alignment techniques typically assume a homogeneous notion of human preference, overlooking the cross-regional diversity of human values. As LLMs are increasingly used for everyday and professional decision-making, ideation, and opinion formation, homogeneous preference modeling may drive convergence toward dominant values, suppressing diverse perspectives and reinforcing stereotypes.
Pluralistic reward modeling is therefore essential for building AI systems that better reflect the pluralistic values of human society. In this work, we propose **PAL**, a general reward modeling framework for pluralistic alignment. **PAL** captures *cross-user commonalities* with a shared reward backbone, while modeling *individual variation* through personalized weights over learnable group-level preference prototypes. By adapting only these weights, **PAL** enables sample-efficient preference localization for new users.
Extensive empirical evaluation demonstrates that **PAL** ***matches or outperforms state-of-the-art methods on both text-to-text and text-to-image tasks***. Finally, we provide a theoretical analysis for **PAL**, showing that its shared structure ***reduces the per-user sample complexity of personalized reward learning***.

PAL: Sample-Efficient Personalized Reward Modeling for Pluralistic Alignment

Existing retrieval-augmented generation (RAG) systems typically assume that each query has a single correct answer, overlooking common information-seeking scenarios with multiple plausible answers, where diversity is crucial to prevent responses from collapsing into a single dominant perspective and to support creativity, fairness, and inclusive information access.
Our analysis reveals a key limitation of standard RAG systems: they underuse diverse retrieved contexts, so increasing retrieval diversity alone does not necessarily produce diverse generations.
To address this limitation, we propose Diverge
, a plug-and-play agentic RAG framework that promotes diverse viewpoints through reflection-guided generation and memory-augmented iterative refinement while preserving answer quality.
We introduce novel human-aligned metrics for evaluating the diversity–quality trade-off in open-ended questions. Experiments across multiple real-world open-ended QA datasets and backbone LLMs show that Diverge achieves the best trade-off, outperforming competitive baselines by substantially improving diversity while preserving quality. Our results reveal a systematic limitation of current LLM-based systems and show that explicit diversity modeling can mitigate it.

DIVERGE: Diversity-Enhanced Retrieval-Augmented Generation for Open-Ended Information Seeking

Cross-lingual bias benchmarks such as JBBQ and KoBBQ translate English bias probes and compare scores across languages, assuming the translated probe measures the same construct. We test this assumption at the representation and behavioral levels using 13B-parameter models matched on architecture but differing in language-training regime. A multi-anchor logit lens shows that an English-centric model (Llama 2) processes Japanese and Korean inputs predominantly through English-script predictions in its middle layers, even where Centered Kernel Alignment (CKA) between languages is high: geometric convergence masks English-hub routing. Matched continual-adaptation comparisons show that target-language adaptation reduces this English-script mass: from 0.77 to 0.56 after Japanese adaptation (Swallow), and from 0.78 to 0.71 after Korean adaptation (koen), while balanced bilingual pretraining (LLM-jp) lowers it further to 0.19. Behaviorally, every model is more stereotype-biased in English than in Japanese, with gaps from 0.13 to 0.14, but this asymmetry is language-specific: in Korean it is weak and disappears after Korean adaptation, with Korean nearly as stereotype-leaning as English. Yet patching English hub states into target-language processing does not transplant this bias. Cross-lingual bias scores thus reflect genuine language-specific behavior, not an English-pivot artifact, even though the underlying representations are not comparable. We distill this dissociation between representation and behavior into a four-step audit protocol for translated bias benchmarks.

Downloads

Next from ACL 2026

The Missing Minimal Pair: Mutual Information for Stereotype Evaluation in LLMs

Similar lecture

Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

The Missing Minimal Pair: Mutual Information for Stereotype Evaluation in LLMs

Similar lecture

Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

Downloads