4-page short papers
Submissions on new agent evaluation methods, RL environment design, agentic benchmarks, and real-world case studies; work-in-progress submissions are encouraged.
Machine Learning/CORE Unranked
ACM CAIS 2026
RLEval: Methods and Reinforcement Learning Environments for Evaluating AI Agents
The RLEval workshop at ACM CAIS 2026 is the first research venue dedicated to evaluating AI agents, focusing on reinforcement learning environments, evaluation methods, benchmarks, and real-world deployment case studies. It aims to address critical open questions in measuring and improving agentic capabilities through interventional, causal, and counterfactual techniques.
Paper fit
Contribution paths
A strong submission should clearly identify its contribution and evaluate it appropriately.
Research areas in scope
01
Topics of Interest
Agent evaluation methods, in particular interventional and causal/counterfactual techniquesRL environments: design principles, software frameworks, synthetic data, tool designAutomated graders: LLM-as-a-judge, verifiers, rubrics, reward hacking, human feedbackBenchmarks: new benchmarks, analyses of existing benchmarksEnterprise agent case studies: production evaluation and deployment lessonsConsiderations in the above topics for Agents with particular capabilities: code execution, computer use, multimodal I/O, NL2SQL, skills, memory, web search
Policies worth checking twice
- Single-blind review (no anonymization required)
- Paper length: 4 pages main text; additional pages allowed for references and appendix
- Formatting must use ACM acmart/sigconf template
- Under-review papers elsewhere are allowed
- Already-published papers are not allowed
- At least one author of each accepted paper must register and attend
- Accepted papers are not archival
Official sources
Compiled from the official call for papers. The organizers’ pages remain authoritative.
Last verified September 9, 2026