Morocco

Evaluating spoken assistants requires judging \emph{content} (C), \emph{voice quality} (VQ), and \emph{paralinguistics} (EQ). Most benchmarks collapse these dimensions into a single overall preference. We show this creates artifacts (forced winners and untyped ties) that overweight textual content and make overall accuracy &quot;hackable&quot; by transcript-only judges. To address this, we re-annotate two public datasets using a Human Chain-of-Thought (HCoT) protocol that elicits \emph{dimension-first} judgments with \emph{typed ties} (\emph{both-good} vs. \emph{both-bad}). This yields reliable, diagnostic labels validated under a protocol aligned with ITU-T guidance on separate dimension ratings. Building on HCoT, we propose \emph{JSON-Judge}, a training-free, two-stage evaluator: (i) assemble a compact JSON blueprint of inexpensive, off-the-shelf signals for C/VQ/EQ; (ii) prompt an LLM for per-dimension decisions, which a deterministic rule maps to the overall label. Across SpeakBench and S2S-Arena, JSON-Judge attains higher overall agreement with HCoT than transcript-only and audio-LLM judges and delivers stronger EQ fidelity, while being \emph{significantly cheaper than audio-LLM judges}. We will release the HCoT re-annotations and the JSON blueprints/prompts, providing a scalable, interpretable, and human-centric framework for evaluating voice assistants.

EACL 2026

Hearing Between the Lines: Unlocking the Reasoning Power of LLMs for Speech Evaluation

speech-to-speech evaluation

dialogue and interactive systems

spoken language understanding

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

The purpose of the AAAI conference series is to promote research in Artificial Intelligence (AI) and foster scientific exchange between researchers, practitioners, scientists, students, and engineers across the entirety of AI and its affiliated disciplines. AAAI-26 will feature technical paper presentations, special tracks, invited speakers, workshops, tutorials, poster sessions, senior member presentations, competitions, and exhibit programs, and a range of other activities to be announced.<br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to AAAI 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

Please log in

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://aaai.org/conference/aaai/aaai-26/registration/) first.

**Online Registration Form**: https://aaai.getregistered.net/conference-2026 

Registration Required

AAAI 2026

We are pleased to announce the Fortieth AAAI Conference on Artificial Intelligence (AAAI-26), which will be held in Singapore EXPO from January 20 to January 27, 2026.

Evaluating spoken assistants requires judging \emph{content} (C), \emph{voice quality} (VQ), and \emph{paralinguistics} (EQ). Most benchmarks collapse these dimensions into a single overall preference. We show this creates artifacts (forced winners and untyped ties) that overweight textual content and make overall accuracy "hackable" by transcript-only judges. To address this, we re-annotate two public datasets using a Human Chain-of-Thought (HCoT) protocol that elicits \emph{dimension-first} judgments with \emph{typed ties} (\emph{both-good} vs. \emph{both-bad}). This yields reliable, diagnostic labels validated under a protocol aligned with ITU-T guidance on separate dimension ratings. Building on HCoT, we propose \emph{JSON-Judge}, a training-free, two-stage evaluator: (i) assemble a compact JSON blueprint of inexpensive, off-the-shelf signals for C/VQ/EQ; (ii) prompt an LLM for per-dimension decisions, which a deterministic rule maps to the overall label. Across SpeakBench and S2S-Arena, JSON-Judge attains higher overall agreement with HCoT than transcript-only and audio-LLM judges and delivers stronger EQ fidelity, while being \emph{significantly cheaper than audio-LLM judges}. We will release the HCoT re-annotations and the JSON blueprints/prompts, providing a scalable, interpretable, and human-centric framework for evaluating voice assistants.

poster

#### *Message from the General Chair, Aline Villavicencio*
I’m delighted and honoured to welcome you to the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026), taking place in the beautiful city of Rabat, in Morocco, in March 24-29, 2026. EACL is the flagship European conference of the Association and EACL 2026 proudly continues our field’s tradition of excellence in scholarship, innovation, and inclusivity. I am deeply grateful to the many volunteers whose dedication, generosity, and tireless efforts have made this conference possible.
For the first time EACL is being hosted in the African continent. This is an important milestone for our community, and we are grateful to our Moroccan hosts for enabling this historic moment by bringing this edition of EACL to Rabat. We are also delighted that the Second Arabic NLP School is co-located with EACL. We hope attendees enjoy this wonderful opportunity to strengthen ties with the Computational Linguistics communities across the African continent. *[Read full message](https://drive.google.com/file/d/14NlmHvwM6fPJuMmOvVh7K0vtQbEyv3SZ/view?usp=sharing)*<br><br>

<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://underline.io/events/522/reception','_blank')">Go to Workshops and Tutorials Program</button></html>
<br><br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to EACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.eacl.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/eacl2026

Welcome to the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL). EACL 2026 will be held in Rabat, Morocco, from March 24–29, 2026. 

Recent advances in audio generation led to an increasing number of deepfakes, making the general public more vulnerable to financial scams, identity theft, and misinformation. Audio deepfake detectors promise to alleviate this issue, with many recent studies reporting accuracy rates close to 99%. However, these methods are typically tested in an in-domain setup, where the deepfake samples from the training and test sets are produced by the same generative models. To this end, we introduce XMAD-Bench, a large-scale cross-domain multilingual audio deepfake benchmark comprising 668.8 hours of real and deepfake speech. In our novel dataset, the speakers, the generative methods, and the real audio sources are distinct across training and test splits. This leads to a challenging cross-domain evaluation setup, where audio deepfake detectors can be tested "in the wild". Our in-domain and cross-domain experiments indicate a clear disparity between the in-domain performance of deepfake detectors, which is usually as high as 100%, and the cross-domain performance of the same models, which is sometimes similar to random chance. Our benchmark highlights the need for the development of robust audio deepfake detectors, which maintain their generalization capacity across different languages, speakers, generative methods, and data sources. Our benchmark is publicly released at https://anonymous.4open.science/r/XMAD-Bench-detection/.

XMAD-Bench: Cross-Domain Multilingual Audio Deepfake Benchmark

KV caching significantly improves the efficiency of Large Language Model (LLM) inference by storing attention states from previously processed tokens, enabling faster generation of subsequent tokens. However, as sequence length increases, the KV cache quickly becomes a major memory bottleneck. To address this, we propose PagedEviction, a novel fine-grained, structured KV cache pruning strategy that enhances the memory efficiency of vLLM's PagedAttention. Unlike existing approaches that rely on attention-based token importance or evict tokens across different vLLM pages, PagedEviction introduces an efficient block-wise eviction algorithm tailored for paged memory layouts. Our method integrates seamlessly with PagedAttention without requiring any modifications to its CUDA attention kernels. We evaluate PagedEviction across Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, and Llama-3.2-3B-Instruct models on the LongBench benchmark suite, demonstrating improved memory usage with better accuracy than baselines on long context tasks.

PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference

Diffusion language models (DLMs) have recently emerged as a compelling alternative to autoregressive generation, offering parallel generation and improved global coherence. During inference, DLMs generate text by iteratively denoising masked sequences in parallel; however, determining which positions to unmask and which tokens to commit forms a large combinatorial search problem. Existing inference methods approximate this search using heuristics, which often yield suboptimal decoding paths; other approaches instead rely on additional training to guide token selection. To introduce a principled search mechanism for DLMs inference, we introduce MEDAL, a framework that integrates Monte Carlo Tree SEarch initialization for Diffusion LAnguage Model inference. We employ Monte Carlo Tree Search at the initialization stage to explore promising unmasking trajectories, providing a robust starting point for subsequent refinement. This integration is enabled by restricting the search space to high-confidence actions and prioritizing token choices that improve model confidence over remaining masked positions. Across multiple benchmarks, MEDAL achieves up to 22.0\% improvement over existing inference strategies, establishing a new paradigm for search-based inference in diffusion language models.

Diffusion Language Model Inference with Monte Carlo Tree Search

The Text-to-SQL task translates natural language questions into SQL queries, enabling intuitive database interaction for non-experts. While recent methods leveraging Large Language Models (LLMs) achieve strong performance, their reliance on proprietary models raise concerns about deployment feasibility and data privacy. In this work, we introduce LitE-SQL, a Lightweight and Efficient framework with two components: (i) a Schema Retriever that performs efficient schema linking using a vector database of pre-computed schema embeddings, and (ii) a SQL Generator fine-tuned in two stagesвЂ”supervised fine-tuning followed by execution-guided reinforcementвЂ”enabling self-correction without costly multi-candidate generation. On BIRD, LitE-SQL achieves 72.10% execution accuracy, and on Spider 1.0 it reaches 88.45%, demonstrating comparable or superior performance to LLM-based methods despite using 2Г— to 30Г— fewer parameters. Our findings demonstrate that high-quality Text-to-SQL generation is feasible with lightweight models, offering a practical solution for privacy-sensitive and resource-constrained settings.

LitE-SQL: A Lightweight and Efficient Text-to-SQL Framework with Vector-based Schema Linking and Execution-Guided Self-Correction

We investigate self-training of large language models (LLMs), in which models are recursively trained on their own generated text. Prior studies have reported conflicting outcomes in this setting: some find evidence of performance gains (i.e., self-improvement), while others observe performance degradation (i.e., model collapse). To clarify this discrepancy, we use the OLMo-2 models as non-toy LLMs and perform multiple rounds of continual pre-training using self-generated text with varying data synthesis and filtering strategies. Our experiments show that naive self-training does not improve either perplexity on the original pre-training corpus nor downstream task performance, regardless of model size. These results suggest that model collapse observed in naive self-training is inherent to the training procedure, while self-improvement likely owes its success not to the model’s autonomous refinement but to human designed, strategic synthetic pipelines that inject external intelligence.

Demystifying Mixed Outcomes of Self-Training: Pre-training Analyses on Non-Toy LLMs

Explaining why content is hateful using natural language is crucial for fostering transparency in automated content moderation systems. However, evaluating the quality of such explanations remains an open challenge. General-purpose reward models (RMs), commonly used for scoring natural language outputs, are typically optimized for broad notions of safety. We argue that this optimization penalizes situations where references to stereotypes or offensive content are essential for faithful hate speech explanations. To address this gap, we introduce SBIC-Explain, a human-validated dataset of 370,788 LLM generated NLEs for offensive content, spanning three levels of human-annotated contextual richness: Tier 1: text-only, Tier 2: + classification-aware, and Tier 3: + semantics-informed. We hypothesize that as human-annotated context increases, explanations should better reflect human preferences. Yet, we find that existing RMs systematically assign lower scores to more contextually rich (and often more offensive) explanations, revealing a misalignment between model preferences and explanatory fidelity for this context. We propose HARM (Hate-Aware Reward Model), a RM that integrates interpretable signals to better align reward scores with the needs of hate speech explanation. HARM outperforms general-purpose baselines, improving NLE pair-wise preference.

HARM: Learning Hate-Aware Reward Model for Evaluating Natural Language Explanations of Offensive Content

Length generalization is the ability of language models to maintain performance on inputs longer than those seen during pretraining. In this work, we introduce a simple yet powerful position encoding (PE) strategy, Random Float Sampling (RFS), that generalizes well to lengths unseen during pretraining or fine-tuning. In particular, instead of selecting position indices from a predefined discrete set, RFS uses randomly sampled continuous values, thereby avoiding out-of-distribution (OOD) issues on unseen lengths by exposing the model to diverse indices during training. Since assigning indices to tokens is a common and fundamental procedure in widely used PEs, the advantage of RFS can easily be incorporated into, for instance, the absolute sinusoidal encoding, RoPE, and ALiBi. Experiments corroborate its effectiveness by showing that RFS results in superior performance in length generalization tasks as well as zero-shot commonsense reasoning benchmarks.

Position Encoding with Random Float Sampling Enhances Length Generalization of Transformers

Argument Mining (AM) aims to identify and interpret argumentative structures in unstructured text, with Argument Component Classification (ACC) as a core task. Despite significant advances, most ACC approaches rely on manually pre-segmented inputs, an assumption that rarely holds in practice due to the high cost and effort of expert human annotation, creating a major bottleneck for scalable AM systems. In this work, we focus on the foundation Argument Component Segmentation (ACS) task by proposing a fine-grained, paired-tag annotation schema that explicitly distinguishes between relevant and surrounding content, thus overcoming the limitations of previous single-separator approaches. Leveraging small and open Large Language Models (LLMs) fine-tuned on our paired-tag annotation schema, we can perform ACS with quality comparable to human expert annotators across multiple benchmark datasets. We further validate our approach on the downstream ACC task, showing that automated segmentation with fine-tuned LLMs yields ACC performances comparable to pipelines relying on human annotations. These findings suggest that reliable automated ACS via LLMs is both feasible and effective, paving the way for more scalable AM pipelines without human intervention.

Argument Component Segmentation with Fine-Tuned Large Language Models

Deep neural network classifiers for dysarthria impairment severity face limitations regarding interpretability and treatment guidance. To overcome these, we introduce CLINIC-GENIE, an explainable two-stage framework consisting of: (1) CLINIC, a dysarthria severity classification model combining acoustic and speech embeddings with Clinically Explainable Acoustic Features (CEAFs); and (2) GENIE, a module translating CEAFs and their Shapley values into intuitive natural language explanations via a large language model. CLINIC achieved a balanced accuracy of 0.952 (17.3\% improvement over using CEAFs alone), and certified speech-language pathologists rated explanations from CLINIC-GENIE with an average fidelity score of 4.94, confirming enhanced clinical utility.

Diagnosis of Dysarthria Severity and Explanation Generation Using XAI-Enhanced CLINIC-GENIE on Diadochokinetic Tasks

The paper extends the Data Movement Distance (DMD) -- a metric defined to measure the locality in computer memory -- to text by defining a normalized version called nDMD. A key feature of nDMD is a new term designed to better characterize low-frequency tokens. By evaluating nDMD on English subset of the M4 dataset and GenAI detection shared task, the paper shows three key findings. First, nDMD is systematically higher in human-written text than in machine-generated text. Second, nDMD-based features not only outperform frequency baselines but also improve overall performance when combined. Finally, the proposed DMD normalization is more effective in distinguishing human and machine text than alternative approaches.

Downloads

Next from EACL 2026

XMAD-Bench: Cross-Domain Multilingual Audio Deepfake Benchmark

Similar lecture

HierarNet: Independent Interactive Hierarchical Disease Outbreak Forecasting

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES