Research Track Paper
4–8 page papers in COLM format on theory, methodology, or systems related to the three core challenges of AI measurement science.
NLP/CORE Unranked
AIMS at COLM 2026
First Workshop in AI Measurement Science: Toward Rigorous AI Evaluation - Competition Track
The AIMS Workshop at COLM 2026 is an inaugural one-day in-person event focused on establishing AI Measurement Science as a rigorous field, addressing fundamental limitations in current AI evaluation paradigms. It brings together researchers from machine learning, statistics, psychometrics, economics, and policy to tackle challenges in measuring interactive, strategically optimized, and non-stationary AI systems.
Key deadlines
Verified September 9, 2026Full paper
August 15, 2026
AoE
Workshop timeline
Submission and decisions
Full paperKey deadline
August 15, 2026 · AoE
Paper fit
Contribution paths
A strong submission should clearly identify its contribution and evaluate it appropriately.
Competition Track Paper
4–8 page papers in COLM format describing team approaches to the Predictive AI Evaluation Challenge, with methods, results, and lessons learned.
Research areas in scope
01
Research Track
Interactive Measurement: Evaluation of policies, long-horizon behavior, tool-using agents, and statistical validity under non-i.i.d. and path-dependent observationsMeasurement under Strategic Optimization: Evaluation as a game between learner and evaluator; robustness to gaming and contamination; incentive-compatible, private, randomized, or adaptive leaderboardsMeasurement under Non-Stationarity: Covariate, label, and construct drift; adaptive test design, recalibration, and longitudinal monitoring of deployed systems
02
Competition Track
Write-ups from participants of the Predictive AI Evaluation Challenge: methods, results, and lessons learned in predicting model responses from sparse observations
Policies worth checking twice
- Submissions must be 4–8 pages in COLM format with unlimited references
- Review is double-blind with at least three reviews per paper
- Accepted papers are non-archival
- Submissions must be made via OpenReview
- Four contributed talks will be selected from accepted submissions
Official sources
Compiled from the official call for papers. The organizers’ pages remain authoritative.
Last verified September 9, 2026