UniAgent at CIKM 2026
Synopsis/Call
Agentic AI systems combine large language models, information retrieval, planning, and tool use to act on behalf of users in complex digital environments. Yet their evaluation remains difficult for lack of reproducible, bounded laboratories. A university administration is a closed-world, socio-technical environment in which many tasks follow standardized, well-documented processes. Such an environment is therefore particularly suited to a Cranfield-style evaluation of AI agents. The controlled, reproducible evaluation of agentic AI in such enterprise-like settings is an underexplored yet highly relevant problem.
This shared task offers the first reusable laboratory for the controlled evaluation of agentic AI in a university environment, namely the University of Kassel. Participants build agents that solve standardized administrative tasks (e.g., travel permits and reimbursements) by retrieving the relevant regulations and planning the necessary steps, while complying with the institution's codes of conduct. The document collection is compiled together with the OpenWebSearch.eu project.
We invite you to participate in the first UniAgent shared task, part of the CIKM 2026 AnalytiCup in Rome, 8 to 11 November 2026, featuring two sub-tasks.
Register for participation Join the UniAgent forum on TIRAShared Tasks
- Agentic AI in University Administration — four administrative task areas and two sub-tasks (retrieval, solving)
Important Dates
| Event | Date |
|---|---|
| Registration Opens; spot-check datasets available on TIRA | now |
| Registration Closes | |
| Submission of the Systems | |
| Notification of Results | 3 November 2026 |
| Notebook Paper Submission | 8 November 2026 |
| Presentation at the CIKM 2026 AnalytiCup | 8 November 2026 |
All times are in Anywhere on Earth (AoE) timezone. The CIKM 2026 main conference runs from 9 to 11 November 2026.
Participation
Participation is open to research groups and individuals from IR, NLP, AI, and adjacent communities, as well as industry teams.
- Register on TIRA. Sign up on TIRA, then register your team to the UniAgent 2026 task. The form asks for a team name (up to 30 characters), your email, affiliation, country, and the other team members. You may enter either of the two sub-tasks and revise that choice later.
- Develop your agent against the public part of the data and the demo dataset (see Data & Tools). You build retrieval on the mounted crawl with your own pipeline, and you may ship your own tools inside your image.
- Submit your systems as code submission (a Docker image) via TIRA, in the prescribed formats together with the agent behavior log. You may submit and run as often as you like during development. For the official evaluation, each team submits up to five runs, which you select yourself before any results are known.
- Run your systems in TIRA After a code submission was made, the system can run in TIRA on all datasets. We will run all submissions against several LLM endpoints.
- Results and findings are presented at the CIKM 2026 AnalytiCup session. You describe your approach in a notebook paper.
Paper Submission
Results and findings are presented at the CIKM 2026 AnalytiCup session. You describe your approach in a notebook paper.
- As the paper title, use the format
<Your Team Name> at UniAgent: <Brief Description of your Approach>. - As the paper subtitle, use
UniAgent at CIKM 2026. - Write around 6 pages, references included. Use the one-column CEURART template on Overleaf and keep its section “Declaration on Generative AI”.
- We will likely publish papers in the CEUR Workshop Proceedings, or alternatively on Zenodo or arXiv.
Program
The shared task results and findings will be presented during the CIKM 2026 AnalytiCup session on 8 November 2026 in Rome. Please find the program of CIKM 2026 conference here.
The session consists of an introduction to the task by the organizers and presentations by the participants. The maximum length of the participant presentations will be announced.
Presenting: teams can present their system in person in Rome or online. Please tell us which you prefer via a message on TIRA.
Awards
Winning teams are recognized with certificates, a presentation slot at the CIKM 2026 AnalytiCup session, and a mention in the overview paper. We foresee no monetary prizes at this stage, though awards may be extended if sponsorship becomes available.
Results
Results will be announced after the evaluation period; see Important Dates.
Resources
- Document collection (OpenWebSearch.eu): to be announced
- Baselines and code repository: to be announced
- Reproducibility platform (TIRA): to be announced
Organizing Committee
Contact
To participate, register on TIRA as described under Participation. Questions and discussion take place in the UniAgent forum on TIRA. You can also reach the organizers at uniagent@lists.its.uni-kassel.de. Releases are announced on this page.
