United States

Large language models show strong capabilities in natural language generation and have been applied to translate complex structured data into human-readable insights. While these models excel at surface-level fluency, they remain unreliable as they produce factually inaccurate outputs and struggle with consistent logical inference beyond surface-level patterns. Moreover, they often lack a clear sense of relevance and produce shallow or uninformative insights. This thesis proposal argues that a key source of these limitations is the fact that tasks are often underspecified, requiring models to make implicit assumptions about missing context. We will investigate how such underspecification leads to unintentional assumptions and how these affect faithfulness and evaluation. We will examine how models can identify missing premises and surface multiple plausible interpretations to improve factuality of their inferences and make their evaluation more rigorous. We will also study how underlying assumptions influence the perceived interestingness of insights. Finally, we will explore how to improve reasoning to enable deeper inferences, focusing on code generation and qualitative reasoning. By shifting focus from surface-level generation to assumption-aware deeper inferences, this work aims to improve reliability, interpretability, and user controlability in NLG.

ACL 2026

Thesis Proposal: Intentional Inference for Insight Generation

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

ACL 2023

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

poster

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. 
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) 
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>

To access this event page, you need to log in with the **email address you registered with**. Access credentials will be sent to your email from Underline - subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Large language models and LLM-based agents are widely deployed as personal chat assistants, enterprise copilots, and autonomous workflow agents. Across all these applications, memory (the ability to retain, access, and reason over information accumulated over long contexts and multiple interactions) plays a crucial role in determining the reliability of any agent. We introduce RECON (Reasoning over Extended Contexts with Obfuscated Narratives), a benchmark for evaluating compositional temporal reasoning over long contexts. RECON uses investigation case file (100k tokens) and tests agents on \textbf{six memory-intensive tasks}: reconstructing multi-hop evidence chains, propagating cascading invalidations, resolving source conflicts, counterfactual reasoning, satisfying temporal constraints, and temporal fact retrieval. Recent memory benchmarks evaluate whether agents can retrieve scattered facts or detect if a fact has changed; RECON evaluates what happens after the change, whether agents can trace which downstream conclusions are affected, which survive through independent support, and how alternative timelines would have unfolded. We evaluate systems across three architectural families comprising long-context LLMs, retrieval-augmented generation (RAG) agents, and commercial memory agents and find that existing memory mechanisms are not yet sufficient for reliable reasoning over long, evolving narratives. Our benchmark and code are available at \url{https://anonymous.4open.science/r/recon-benchmark/README.md}.

RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts

To accelerate scientific knowledge acquisition, LLMs are increasingly used to synthesize multiple papers into structured tables by inferring schemas and values. While value generation within a fixed schema can often be reduced to extractive question answering, the schema generation problem, determining which dimensions to compare a set of documents, lacks a formal mapping to standard NLP tasks. In this work, we formulate schema generation as a reinforcement learning problem and investigate whether these dimensions can be induced without access to gold-standard schemas. We design a multi-faceted reward framework capturing schema coverage, non-redundancy, relevance, and format, and train a small language model on a literature review dataset. Our approach yields consistent improvements over the untuned base model across intrinsic, reference-based, and LLM-judge metrics, and remains competitive with supervised fine-tuned models at 5$\times$ the parameter count on structural and diversity dimensions.\footnote{Code will be released upon publication.}

Reference-Free Schema Generation for Literature Review Tables via Multi-Faceted Rewards

The widespread integration of large language models (LLMs) across diverse domains, from entertainment to healthcare and banking, has underscored the pressing need to address the risks associated with artificial intelligence (AI). Indeed, as LLMs have grown in scale, new challenges have emerged, including privacy breaches, dissemination of misinformation, issues related to robustness and alignment, and concerns regarding the societal biases ingrained in these models. In response, there has been a surge in research focused on responsible AI and AI safety. To address these challenges, recent literature has introduced novel techniques for mitigating the risks associated with LLMs and testing the efficacy of safety protocols through red-teaming. Despite these efforts, concerns persist regarding the perpetuation of biases in next-generation LLMs and the lack of robustness in existing safety measures. In this thesis proposal, we argue that some failures in LLM safety arise from flawed evaluation practices and explore ways to more effectively assess and characterize them.

Thesis Proposal: Rethinking Safety Evaluation in Large Language Models

When large language models encounter conflicts between parametric knowledge and contextual evidence, does the parametric circuit's contribution change, or is the conflict resolved at the output distribution? We apply a three-level analysis framework, decomposing conflict resolution into circuit-level (parametric-site logit contribution, $\Phi_P$), pathway-level (MLP vs.\ attention), and output-level (token rank) effects across four transformer architectures spanning three attention types and two scale points. We find that $\Phi_P$ is near-invariant under conflict ($\leq$1.8\% variation), while output token ranks shift substantially, establishing that conflicts resolve through additive logit-space competition. Activation patching confirms parametric sites are causally sufficient and necessary, with contextual heads exerting 30--40$\times$ weaker influence. Layer-by-layer analysis reveals three architecturally determined resolution mechanisms, and parametric--contextual coupling decreases substantially with scale, exceeding random-subspace predictions. Domain generalization shows parametric sites are stratified by knowledge complexity with no overlap, yet the resolution mechanism generalizes across domains.

Dissociating Circuit-Level and Distribution-Level Effects of Knowledge Conflicts in LLMs

Before a tax authority can issue a ruling, it must receive a complete description of the taxpayer's situation---yet no benchmark measures whether language models can systematically elicit all relevant facts through dialogue. We introduce the \textbf{Factual State Discovery Benchmark}, in which a discovery agent questions a simulated taxpayer grounded in a real tax document. The dataset comprises 500 narratives from official Polish tax interpretations, decomposed into 32\,874 atomic facts with validated supported precision (97.6\%), atomicity (93.8\%), and sentence coverage (96.0\%). Experiments with four models show that even the best system recovers only 77\% of facts on easy samples and under 49\% on hard samples after 50 turns. These findings establish conversational fact elicitation as a challenging open problem requiring retrieval-augmented and adaptive questioning strategies.

Factual State Discovery Benchmark: Evaluating Fact Elicitation in Polish Tax Law

Uncertainty quantification (UQ) methods for large language models are predominantly designed by hand based on domain knowledge and heuristics, limiting their scalability and generality. We apply LLM-powered evolutionary search to automatically discover unsupervised UQ methods represented as Python programs. On the task of atomic claim verification, our evolved methods outperform strong manually-designed baselines, achieving up to 6.7\% relative ROC-AUC improvement across 9 datasets while generalizing robustly out-of-distribution. Qualitative analysis reveals that different LLMs employ qualitatively distinct evolutionary strategies: Claude models consistently design high-feature-count linear estimators, while Gpt-oss-120B gravitates toward simpler and more interpretable positional weighting schemes. Surprisingly, only Sonnet 4.5 and Opus 4.5 reliably leverage increased method complexity to improve performance~-- Opus 4.6 shows an unexpected regression relative to its predecessor. Overall, our results hint that LLM-powered evolutionary search is a promising paradigm for automated, interpretable hallucination detector design.

Evolutionary Search for Automated Design of Uncertainty Quantification Methods

Recursive models that iteratively refine latent representations have demonstrated strong performance on variety of reasoning tasks. However, these models only control if and when to stop early, not how computation is distributed. In this work, we introduce shortcut reasoning, a framework for distilling recursive latent reasoning into a multiscale jump model that enables flexible test-time compute. We reinterpret recursive reasoning as a latent-time dynamical process and train a student model to predict the effect of multiple reasoning steps at once. To ensure robustness, we augment shortcut transitions with a repair mechanism, where a denoising base-model update projects approximate latent states back to a valid reasoning manifold. We further introduce stepwise improvement supervision, encouraging each shortcut step to increase the likelihood of the correct answer. Experiments on ARC-AGI show that our approach achieves competitive accuracy compared to recursive baselines while requiring fewer sequential updates.

Learning Shortcut Models for Efficient Recursive Reasoning

Multi-agent debate systems are usually evaluated only by whether the final answer is correct. However, this overlooks the quality of the intermediate reasoning that debate is meant to produce. This PhD research studies the relationship between three signals in multi-agent debate: token-level log-probability distributions over reasoning tokens, LLM-as-judge rubric scores assigned to those tokens, and final task accuracy. Instead of focusing only on final outputs, we examine whether internal confidence signals predict externally evaluated reasoning quality, and whether either signal is aligned with task correctness. We apply this framework to three domains: rubric-based scoring, mathematical reasoning, and factual question answering. Our setup uses a two-agent debate architecture in which a Constructor and an Auditor generate opposing reasoning traces while exposing token-level log-probabilities during decoding. A separate LLM-as-judge then scores each agent's reasoning using rubric-based criteria. Preliminary experiments show that early-token log-probability statistics correlate meaningfully with judge scores and are especially useful for identifying major reasoning failures. These findings motivate the broader cross-domain investigation proposed in this dissertation.

Thesis Proposal: When Does Confidence Signal Quality? Log-Probabilities and LLM-as-Judge in Multi-Agent Debate

Recent work has shown that LLMs develop internally coherent utility functions that emerge with scale, yet whether these value systems encode systematic demographic hierarchies remains unexplored. We elicit pairwise preferences across 15 intersectional demographic groups (defined by race, gender, and their combinations) and 8 policy domains on three 7--8B instruction-tuned LLMs, fitting Thurstonian utility models to the resulting preference matrices. All three models converge on a compensatory hierarchy that inverts real-world structural advantage, consistently ranking marginalized groups, the highest and dominant groups are lowest. Intersectional utilities do not combine additively: single-axis audits that measure gender and race gaps independently overestimate the most extreme intersectional gap by 26- 40\% in our experiments. Geometrically, we identify a linear direction in the representation space that predicts the full utility hierarchy from neutral sentences alone, and show that this direction is substantially aligned with gender encoding but not with race encoding. Orthogonalization reveals that gender separation in representations is not fully explained by utility encoding. The hierarchy is already present in base (pre-alignment) models and is amplified several-fold by instruction tuning, suggesting it originates in pre-training data rather than alignment procedures.

Convergent Demographic Utility Hierarchies: Geometry of Intersectional Values in LLMs

There are now multiple proposals for systems based on Large Language Models(LLMs) to conduct automated qualitative interviews. This approach scales up qualitative interview techniques that have traditionally been constrained by the high costs of data collection. However, most of the current solutions rely on proprietary LLMs, which compromise reproducibility and data security. They also rely on LLMs for all interview tasks, which limits standardisation of question wording as well as control over question order. To address these issues, we introduce the AInterviewer platform, based on a multi-agent framework that combines controlled question administration of survey software with the flexibility of LLMs. AInterviewer can run with locally hosted models to ensure security and transparency. Our platform provides a web-based GUI supporting each phase of data collection: from interview guide design and pilot testing to interview distribution and data collection monitoring.

Downloads

Next from ACL 2026

RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts

Similar lecture

Semantic Accuracy in Natural Language Generation: A Thesis Proposal

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts

Similar lecture

Semantic Accuracy in Natural Language Generation: A Thesis Proposal

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES

Downloads