United States

This paper presents our findings for SemEval-2026 Task 9. We submit to all three subtasks using an LLM-as-an-annotator strategy, simulating the data annotation process with large language models. We created 30 LLM annotators using persona injection (also known as sociodemographic prompting) and experimented with various annotation aggregation methods, including Dawid-Skene and MACE. To further increase the variability in annotator responses, we used the hatefulness detection task as proxy for identifying polarisation. Our findings indicate that this reframing of the problem is effective for the binary classification of polarisation, but is less effective for finer-grained polarisation detection. For subtasks 2 and 3, majority voting yielded the best overall performance. While our unsupervised approach does not rank as highly as supervised ones, this work provides insight into the utility of persona-based prompting and the issue of LLM annotators exhibiting high intra-model agreement.

ACL 2026

ShefFriday at SemEval-2026 Task 9: LLM-Based Annotation Methods for Detecting Multilingual, Multicultural and Multievent Online Polarisation

large language model

agreement

prompting

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year!<br><br>

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).


CogSci 2021

The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

workshop paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. <br>
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) <br><br><br>
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>
<br><br><br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Conspiracy believers use significantly more psycholinguistic markers per post than nonbelievers (Cohen's d = 0.56, p < 10⁻⁸⁰), a pattern we term narrative density, suggesting that belief manifests as structurally denser conspiratorial frames distributed across the full text rather than concentrated in specific lexical cues.

We present Truth Gradient's system for SemEval-2026 Task 10 Subtask 2 (Samory et al., 2026): a DeBERTaV3-large model with mean pooling and a 5-seed probability-averaging ensemble achieving macro F1 = 0.829 on the 77-sample development set and 0.75 on the official test set. The 5-fold CV estimate (0.734 ± 0.007) proves the more reliable predictor of test performance, and we recommend it as standard practice for low-resource shared tasks.

Two convergent tests support the narrative density account: masking annotated marker spans drops F1 by 5.3 pp, and direct marker-count fusion recovers +0.9 pp, though we note these are not conclusive given the small dev set. Cross-validated ablation identifies encoder fine-tuning as the dominant design factor (−7.2 pts), and layer-wise probing confirms belief information peaks at mid-stack layers (layer 16/24).

Truth Gradient at SemEval-2026 Task 10:Conspiracy Belief Detection via Narrative Density and Mean Pooling

Conspiracy theories widely spread on social media and can harm society 
by increasing mistrust, vaccine hesitancy, and political radicalization. 
However, most automated detection systems have traditionally relied on 
topic-specific classifiers, which often struggle to generalize across 
domains and provide little explanation for why a text is considered 
conspiratorial. To address these limitations, this paper explores various 
LLMs on the SemEval-2026 Task 10: psycholinguistic conspiracy marker 
extraction and binary conspiracy detection from Reddit submission 
statements. Specifically, we adopt a training-free few-shot prompting 
approach using different instruction-tuned large language models under a 
variety of few-shot settings (k in {0,1,5,10,15, 20}). Within this 
framework, the proposed prompting strategy incorporates psychology-informed 
instructions to guide the models in identifying conspiracy-related signals. 
As a result, the presented system achieves an F1 score of 0.21 for marker 
extraction and 0.81 for conspiracy detection, ranking 16th out of 30 teams 
in Subtask~1 and 36th out of 52 in Subtask~2 without any task-specific 
fine-tuning. These results suggest that psycholinguistically grounded 
prompting can support interpretable conspiracy analysis; however, challenges 
remain in identifying implicit markers.

CUET_320 at SemEval-2026 Task 10: Few-Shot Large Language Models for Psycholinguistic Marker Extraction and Conspiracy Detection

Despite the need to address the proliferation of conspiracy theories in online discussions, there is a lack of benchmarks for effectively detecting conspiracy-related content in everyday conversational settings. We introduce a novel dataset of comments from Reddit, ranging from politics to TV series, as well as two synergetic tasks: (1) extracting five psycholinguistic markers, grounded in evolutionary psychology, and (2) detecting conspiracy content. The data enable multi-task approaches, allowing testing of whether marker extraction improves detection performance.

SemEval-2026 Task 10: PsyCoMark -- Psycholinguistic Conspiracy Marker Extraction and Detection

We present our shared task on evaluating the adaptability of LLMs and NLP systems across multiple languages and cultures. The task data consist of an extended version of our manually constructed BLEnD benchmark (Myung et al., 2024), covering more than 30 language--culture pairs, predominantly representing low-resource languages spoken across multiple continents. As the task is designed strictly for evaluation, participants were not permitted to use the data for training, fine-tuning, few-shot learning, or any other form of model modification.

Our task includes two tracks: (a) Short-Answer Questions (SAQ) and (b) Multiple-Choice Questions (MCQ). Participants were required to predict labels and were allowed to submit any NLP system and adopt diverse modelling strategies, provided that the benchmark was used solely for evaluation. The task attracted more than 140 registered participants, and we received final submissions from 62 teams, along with 19 system description papers.

We report the results and present an analysis of the best-performing systems and the most commonly adopted approaches. Furthermore, we discuss shared insights into open questions and challenges related to evaluation, misalignment, and methodological perspectives on model behaviour in low-resource languages and for under-represented cultures. Our data and resources are available at https://github.com/BLEnD-SemEval2026/SemEval-2026-Task-7.

SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures

Medical query rewriting transforms verbose
consumer health questions into concise clin-
ical queries, a critical step in health infor-
mation retrieval. Large language models
(LLMs) perform well on this task by stan-
dard metrics, yet high ROUGE or BERTScore
does not guarantee preservation of clinical
content. To address this issue, we intro-
duce MedFaith-F1, a category-level faithful-
ness metric over four clinically salient cate-
gories: clinical problems, medications, proce-
dures, and follow-up intent. We further propose
a hybrid Evidence and Knowledge-Grounded
Retrieval-Augmented Generation (EKG-RAG),
an evidence and knowledge-grounded frame-
work combining hybrid retrieval over PubMed
and MedlinePlus resources with UMLS (Uni-
fied Medical Language System)-aligned ontol-
ogy grounding. Evaluating LLMs LLaMA-
3 and Qwen2.5 across zero-shot, few-shot,
and QLoRA settings on MeQSum and med-
ical question-pair (MQP) datasets revealed
that base models exhibit category-level faith-
fulness failure rates (CHR) exceeding 40%,
invisible to standard metrics, while EKG-
RAG with QLoRA reduces CHR to 26.75%,
achieving MedFaith-F1 of 0.73. Our find-
ings call for faithfulness-aware evaluation
in clinical query rewriting, and MedFaith-
F1 provides a reproducible step in that di-
rection.

Beyond Lexical Similarity: Evaluating Faithfulness in LLM-Based Medical Question Reformulation

This paper presents a hybrid solution for a binary classification of medical PubMed articles created for identifying reports that associate clinical metadata with SARS-CoV-2 genomic sequences. The system is designed to catch the subtle distinction between reports of sequence-associated patient metadata and sentences where such metadata is either unrelated, irrelevant, or linked to previous studies. The biggest challenge is the fact that the medical dataset is highly imbalanced, consisting of only 13.3 % of medical reports labeled positive. Our system proposes a hybrid solution combining four approaches that include dual-evidence tagging, negation-aware suppression, semantic frame extraction, and adversarial training. All these approaches were tested on multiple models: BiomedBERT-baseabstract, BioLinkBERT-large, PubMedBERTbase-fulltext, followed by a best-subset ensemble search to obtain an F1 score of 0.786, setting a new benchmark and positioning the solution in 1st place in the competition.

MetaMiners at SMM4H-HeaRD 2026: A Semantic-StructuralKnowledge-Enriched Ensemble for SARS-CoV-2 Metadata Identification

This paper describes the BIT.UA sys-
tem for the MultiClinNER shared task at
#SMM4H–HeaRD 2026, targeting multi-
lingual clinical named entity recognition
across seven languages for three entity types
(Disease, Procedure, Symptom). We extend
the Multi-Head CRF architecture, originally
developed for multi-class NER on Spanish
clinical text, to the multilingual setting. To
enable joint multi-entity training despite per-
entity text variations in the dataset, we develop
an adaptive text consolidation pipeline that
preserves over 94% of annotations. Our central
finding is that a single xlm-roberta-large model,
trained jointly on all seven languages and
three entity types, achieves competition rank 2
for five of seven languages, outperforming
dedicated monolingual models by up to +6.94
F1 points, while requiring only a single set
of weights. Ensembling multiple seeds of
this model achieves rank 1 for those five
languages, and combining it with monolingual
models yields rank 1 for the remaining
two.

BIT.UA at #SMM4H–HeaRD 2026: Towards Multi-Class Multilingual
Clinical Entity Recognition with Multi-Head CRF Ensembles

Word-level cross-lingual alignment is central to
annotation projection, translation auditing, and
cross-lingual faithfulness estimation, yet exist-
ing neural aligners are rarely adapted to special-
ized domains. In this paper, we introduce Clin-
icalAligner26AM, a large-context multilingual
aligner model for biomedical and clinical text
initialized from ClinicalEncoder26AM (Remy,
2026b). Our training recipe is inspired by
AWESoME Align (Dou and Neubig, 2021). We
build our soft alignment target by sharpening
with Sinkhorn–Knop optimal transport a cost
matrix established for parallel clinical texts and
conversations through the fusion of sentence-
level, phrase-level, and token-level signals. We
distill this sharpened alignment matrix directly
into our student aligner, by encouraging its
naive cosine-based token similarity scores to
match this target. At inference time, we project
source-span scores through the learned token
alignment matrix and decode the longest valid
high-scoring span in the target text, optionally
supported by MultiClinNER predictions sum-
marized in Appendix B. We evaluate CA26AM
on the MultiClinCorpus shared task, which
projects Spanish clinical entity annotations into
six target languages (Gallego-Donoso et al.,
2026; Lopez-Garcia et al., 2026). Our two sub-
mitted systems ranked respectively first and
second across all languages and entity types,
with character-weighted F1 scores above 0.95
in nearly all settings.

Parallia at #SMM4H-HeaRD 2026:
ClinicalAligner26AM: A Cross-Lingual Aligner for Dataset Translation;
Evidences from the MultiClinCorpus Shared Task

We describe two system submissions to Task 4
of the SMM4H-HeaRD 2026 Shared Task
on automated SOAP note generation from
doctor–patient dialogues. Our first submis-
sion is a standalone entity-conditioned gen-
eration model: Mistral-7B-Instruct-v0.1 fine-
tuned with QLoRA on 8,529 MedSynth train-
ing dialogues, where both training and infer-
ence prompts include clinical entities extracted
and grouped by SOAP section. Our second
submission is a four-stage modular pipeline
that additionally incorporates a hybrid retrieval
stage and a rule-based verification stage. The
key finding of this work is that incorporating
structured clinical domain knowledge, in the
form of NER entities grouped by SOAP sec-
tion, directly into the generation prompt pro-
duces consistent and reliable improvements
over dialogue-only generation. Our four-stage
pipeline submission achieved an average score
of 0.54 on the official test set, ranking first on
the shared task leaderboard.

NU_DeepHealthNLP at #SMM4H-HeaRD 2026: Entity-Conditioned
Generation and a Four-Stage Pipeline for Automated SOAP Note
Generation

TNM cancer staging is a critical process for
characterizing tumor burden and guiding clini-
cal decisions. Nevertheless, its automated ex-
traction remains challenging due to the unstruc-
tured and heterogeneous nature of free-text
pathology reports. This paper describes the par-
ticipation of the URJC-Team in Task 6 of the
Social Media Mining for Health/Health Real-
World Data (#SMM4H-HeaRD) 2026 Shared
Tasks. It focuses on predicting TNM stag-
ing from pathology reports. The proposed
workflow combines hand-crafted regular ex-
pressions with a Large Language Model (LLM).
First, explicit TNM mentions are extracted us-
ing rule-based patterns. Then, any stage not
recovered by these rules is inferred by an LLM.
Overall, the proposal provides competitive re-
sults across all official shared-task phases.

Downloads

Next from ACL 2026

Truth Gradient at SemEval-2026 Task 10:Conspiracy Belief Detection via Narrative Density and Mean Pooling

Similar lecture

What interventions can decrease or increase belief polarisation in a population of rational agents?

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

Truth Gradient at SemEval-2026 Task 10:Conspiracy Belief Detection via Narrative Density and Mean Pooling

Similar lecture

What interventions can decrease or increase belief polarisation in a population of rational agents?

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

Downloads