India

Automatic Sign Language Recognition (ASLR) has emerged as a vital field for bridging the gap between deaf and hearing communities. However, the problem of sign-to-sign retrieval or detecting a specific sign within a sequence of continuous signs remains largely unexplored. We define this novel task as Sign Language Spotting. In this paper, we present a first step toward sign language retrieval by addressing the challenge of detecting the presence or absence of a query sign video within a sentence-level gloss or sign video. Unlike conventional approaches that rely on intermediate gloss recognition or text-based matching, we propose an end-to-end model that directly operates on pose keypoints extracted from sign videos. Our architecture employs an encoder-only backbone with a binary classification head to determine whether the query sign appears within the target sequence. By focusing on pose representations instead of raw RGB frames, our method significantly reduces computational cost and mitigates visual noise. We evaluate our approach on the Word Presence Prediction dataset from the WSLP 2025 shared task, achieving 61.88\% accuracy and 60.00\% F1-score. These results demonstrate the effectiveness of our pose-based framework for Sign Language Spotting, establishing a strong foundation for future research in automatic sign language retrieval and verification.

IJCNLP-AACL 2025

Pose-Based Sign Language Spotting via an End-to-End Encoder Architecture

sign spotting

sign language

encoder

### Welcome to IJCNLP-AACL 2025! 
 It is a great honor to host this joint conference in Mumbai, India, from December 20 to 24, 2025. The joint conferences of IJCNLP and AACL are organized with alternating leadership in the Asia-Pacific region. The event is run by the Asian Federation of Natural Language Processing (AFNLP) in odd years, and by AACL in even years, while it is organized solely by ACL when the annual ACL meeting is held in the region. This year, the conference is primarily organized by AFNLP. 
*Kentaro Inui
MBZUAI, UAE
General Chair, IJCNLP-AACL 2025* 
Read full message and download the Conference Handbook [**here**](https://drive.google.com/file/d/1UTwxkAqSqI-GAoJC3wE1zZt5VP1Y8GX0/view?usp=sharing).

The 14th IJCNLP & 4th AACL will be held in Mumbai, India from December 20th to December 24th, 2025.

workshop paper

As large language models (LLMs) become increasingly embedded in our daily lives, evaluating their quality and reliability across diverse contexts has become essential. While comprehensive benchmarks exist for assessing LLM performance in English, there remains a significant gap in evaluation resources for other languages. Moreover, because most LLMs are trained primarily on data rooted in European and American cultures, they often lack familiarity with non-Western cultural contexts. To address this limitation, our study focuses on the Persian language and Iranian culture. We introduce 19 new evaluation datasets specifically designed to assess LLMs on topics such as Iranian law, Persian grammar, Persian idioms, and university entrance exams. Using these datasets, we benchmarked 41 prominent LLMs, aiming to bridge the existing cultural and linguistic evaluation gap in the field. The evaluation results are publicly available on our live leaderboard: https://huggingface.co/spaces/opll-org/Open-Persian-LLM-Leaderboard

MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Language

We assess the capabilities of large language models on tasks involving Buddhist texts written in Sanskrit and Classical Tibetan—two typologically distinct, low-resource historical languages. To this end, we introduce DharmaBench, a benchmark suite comprising 13 classification and detection tasks grounded in Buddhist textual traditions: six in Sanskrit and seven in Tibetan, with four shared across both. The tasks are curated from scratch, tailored to the linguistic and cultural characteristics of each language. We evaluate a range of models, from proprietary systems like GPT-4o to smaller, domain-specific open-weight models, analyzing their performance across tasks and languages. All datasets and code are publicly released, under the CC-BY-4 License and the Apache-2.0 License respectively, to support research on historical language processing and the development of culturally inclusive NLP systems.

DharmaBench: Evaluating Language Models on Buddhist Texts in Sanskrit and Tibetan

Existing benchmarks for frontier models often test specialized, "PhD-level" knowledge that is difficult for non-experts to grasp. In contrast, we present a benchmark with 613 problems based on the NPR Sunday Puzzle Challenge that requires only general knowledge. Our benchmark is challenging for both humans and models; however correct solutions are easy to verify, and models' mistakes are easy to spot. As LLMs are more widely deployed in society, we believe it is useful to develop benchmarks for frontier models that humans can understand without the need for deep domain expertise.

Our work reveals capability gaps that are not evident in existing benchmarks: OpenAI o1 significantly outperforms other reasoning models on our benchmark, despite being on par with other models when tested on benchmarks that test specialized knowledge. Furthermore, our analysis of reasoning outputs uncovers new kinds of failures. DeepSeek R1, for instance, often concedes with "I give up" before providing an answer that it knows is wrong. R1 can also be remarkably "uncertain" in its output and in rare cases, it does not "finish thinking," which suggests the need for techniques to ``wrap up'' before the context window limit is reached. We also quantify the effectiveness of reasoning longer to identify the point beyond which more reasoning is unlikely to improve accuracy on our benchmark.

ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models

Sign Language Translation (SLT) is a crucial technology for fostering communication accessibility for the Deaf and Hard-of-Hearing (DHH) community. A dominant approach in SLT involves a two-stage pipeline: first, transcribing video to sign language glosses, and then translating these glosses into natural text. This second stage, gloss-to-text translation, is a challenging, low-resource machine translation task due to data scarcity and significant syntactic divergence. While prior work has often relied on training translation models from scratch, we show that fine-tuning large, pre-trained language models (PLMs) offers a more effective and data-efficient paradigm. In this work, we conduct a comprehensive bidirectional evaluation of several PLMs (T5, Flan-T5, mBART, and Llama) on this task. We use a collection of popular SLT datasets (RWTH-PHOENIX-14T, SIGNUM, and ASLG-PC12) and evaluate performance using standard machine translation metrics. Our results show that fine-tuned PLMs consistently and significantly outperform Transformer models trained from scratch, establishing new state-of-the-art results. Crucially, our bidirectional analysis reveals a significant performance gap, with Text-to-Gloss translation posing a greater challenge than Gloss-to-Text. We conclude that leveraging the linguistic knowledge of pre-trained models is a superior strategy for gloss translation and provides a more practical foundation for building robust, real-world SLT systems.

Finetuning Pre-trained Language Models for Bidirectional Sign Language Gloss to Text Translation

Quantum Natural Language Processing (QNLP) has emerged as a novel paradigm that leverages the principles of quantum mechanics to address fundamental challenges in language modeling, particularly in capturing compositional meaning. This survey charts the evolution of QNLP, from its theoretical foundations in the Distributional Compositional Categorical (DisCoCat) framework to its modern implementation on Noisy Intermediate-Scale Quantum (NISQ) hardware. We review the primary architectural approaches, including variational quantum circuits and tensor networks, and summarize the growing body of empirical work in tasks such as text classification, sentence similarity, and question answering. A recurring finding is the potential for QNLP models to achieve competitive performance with significantly fewer parameters than their classical counterparts. However, the field is critically constrained by the limitations of NISQ-era hardware. We conclude by discussing these challenges and outlining the future trajectory towards achieving a demonstrable quantum advantage and building more interpretable, efficient language models.

A Survey of Quantum Natural Language Processing: From Compositional Models to NISQ-Era Empiricism

Legal document analysis is pivotal in modern judicial systems, particularly for case retrieval, classification, and recommendation tasks. Graph neural networks (GNNs) have revolutionized legal use cases by enabling the efficient analysis of complex relationships. Although existing legal citation network datasets have significantly advanced research in this domain, the lack of large-scale open-source datasets tailored to the Indian judicial system has limited progress. To address this gap, we present the Indian Legal Citation Network (LeCNet) - the first open-source benchmark dataset for the link prediction task (missing citation recommendation) in the Indian judicial context. The dataset has been created by extracting information from the original judgments. LeCNet comprises 26,308 nodes representing case judgments and 67,108 edges representing citation relationships between the case nodes. Each node is described with rich features of document embeddings that incorporate contextual information from the case documents. Baseline experiments using various machine learning models were conducted for dataset validation. The Mean Reciprocal Rank (MRR) metric is used for model evaluation. The results obtained demonstrate the utility of the LeCNet dataset, highlighting the advantages of graph-based representations over purely textual models.

LeCNet: A Legal Citation Network Benchmark Dataset

The increasing volume and complexity of Indian High Court judgments require high-quality automated summarization systems. Our agentic workflow framework for the summarization of Indian High Court judgments achieves competitive results without model fine-tuning. Experiments on CivilSum and IN-Abs test sets report ROUGE-1 F1 up to 0.547 and BERTScore F1 up to 0.866, comparable to state-of-the-art supervised models, with advantages in transparency and efficiency. We introduce two zero-shot modular agentic workflows: Lexical Modular Summarizer (LexA), a three-stage modular architecture optimized for lexical overlap (ROUGE), and Semantic Agentic Summarizer (SemA), a five-stage integrated architecture optimized for semantic similarity (BERTScore). Both workflows operate without supervised model fine-tuning, instead relying on strategic data processing, modular agent orchestration, and carefully engineered prompts. Our framework achieves ROUGE-1 F1 of 0.6326 and BERTScore F1 of 0.8902 on CivilSum test set, and ROUGE-1 F1 of 0.1951 and 
BERTScore F1 of 0.8299 on IN-Abs test set, substantially outperforming zero-shot baselines, rivaling leading fine-tuned transformer models while requiring no supervised training. This work demonstrates that modular, zero-shot agentic approaches can deliver production-grade results for legal summarization, offering a new direction for resource-limited judicial settings.

Legal Document Summarization: A Zero-shot Modular Agentic Workflow Approach

Despite comprehensive food safety regulations worldwide, violations continue to pose significant public health challenges. This paper presents an LLM-driven pipeline for analyzing legal texts to identify structural and procedural gaps in food safety enforcement. We develop an end-to-end system that leverages Large Language Models to extract structured entities from legal judgments, construct statute-and-provision-level knowledge graphs, and perform semantic clustering of cases. Applying our approach to $782$ Indian food safety violation cases filed between 2022-2024, we uncover critical insights: $96\%$ of cases were filed by individuals and organizations against state authorities, with $60\%$ resulting in decisions favoring appellants. Through automated clustering and analysis, we identify major procedural lapses including unclear jurisdictional boundaries between enforcement agencies, insufficient evidence collection, and ambiguous penalty guidelines. Our findings reveal concrete weaknesses in current enforcement practices and demonstrate the practical value of LLMs for legal analysis at scale.

LLM Driven Legal Text Analytics: A Case Study For Food Safety Violation Cases

The shared task of Legal Summarization (L-Summ) focuses on generating abstractive summaries for the Indian court judgments in English. This task presents unique challenges in producing fluent, relevant, and legally appropriate summaries given voluminous judgment texts. We experiment with different sequence-to-sequence models and present a comprehensive comparative study of their performance. We also evaluate various Large Language Models (LLM) with zero-shot settings for testing their summarization capabilities. Our best performing model is fine-tuned on a pre-trained legal summarization model where relevant passages are identified using the maximum marginal relevance(MMR) technique. Our findings highlight that retrieval-augmented fine-tuning is an effective approach for generating precise and concise legal summaries. We obtained a rank of 5th overall.

NIT-Surat@L-Sum: A Semantic Retrieval-Based Framework for Summarizing Indian Judicial Documents

In recent years, the research and development of large language models (LLMs) has been dominated by a small number of private organizations, largely conducted in a closed manner, and been predominantly English centric. Although some leading companies, such as Meta and DeepSeek, have adopted more open-source strategies than their competition, their training data and development processes remain closed to the wider scientific community and society as a whole. This closedness and at best limited transparency leads to an inability of the great majority of us to investigate and explore key scientific and societal challenges associated with this emerging technology, such as adapting and evaluating it for our own languages and exploring and addressing known issues such as biases and hallucinations. To address this, the LLM-jp project was launched in May 2023 under the leadership of the National Institute of Informatics as a sovereign, open, and collaborative initiative. Our aim is to develop Japanese-competent LLMs, elucidate the mechanisms behind their capabilities, and openly release all models, datasets, tools, and even share our discussions and failures with the public. Anyone who shares this vision is invited to participate, making LLM-jp an example of truly open and team science with more than 2,400 members to date.

The project is structured as a Big Science effort, bringing together multiple working groups covering corpus construction, model training, tuning and evaluation, safety, multimodal processing, and real-world applications. More than a dozen leading Japanese researchers from both universities and research institutes collaborate closely, supported by large-scale, national computational infrastructure, in alignment with emerging global AI safety initiatives, and with governmental support from the Ministry of Education, Culture, Sports, Science, and Technology.

In this keynote, I will present the motivations, organization, and progress of LLM-jp, and discuss how building a sovereign LLM ecosystem can not only advance Japan’s academic and industrial prowess, but also contribute to a global movement toward open and transparent AI research.

Downloads

Next from IJCNLP-AACL 2025

MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Language

Similar lecture

Improving Sign Language Understanding with a Multi-Stream Masked Autoencoder Trained on ASL Videos

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from IJCNLP-AACL 2025

MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Language

Similar lecture

Improving Sign Language Understanding with a Multi-Stream Masked Autoencoder Trained on ASL Videos

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

Downloads