1st CASTLE Challenge at ACM Multimedia 2025
The first CASTLE multimodal analytics challenge will be held at ACM Multimedia 2025 in Dublin, Ireland.
To express your interest in participating to the challenge, please fill this form.
Timeline
- 09 March 2025: Challenge Announcement & Website Launch
- 09 March 2025: Registration Opens
- 24 March 2025: Dataset & Query Release
- 30 June 2025: Fully-Automated Submission Deadline
- 24 July 2025: Notification to Authors
- 26 August 2025: Camera-Ready Deadline
Guidelines for Participants
Participants will be required to register and agree to the dataset usage policy. Details regarding dataset licensing and submission guidelines will be provided upon release.
Tasks
The inaugural edition of the CASTLE Challenge features a diverse set of tasks, including event detection, retrieval, and question answering. Future editions will expand the scope, but for this edition, the tasks include:
🔍 Event Instance Search
Given a textual description (in English), participants must identify all timeframes where a specific event occurs. Events should be reported with both a time range and a video ID.
📦 Object Instance Search
Given a textual (in English) or visual (i.e., using an image) example of a physical object, participants must find all occurrences of that object across any of the video streams.
💬 Question Answering
Given a question in natural language (in English), participants must provide an answer. The response should be formulated in natural language and include references to relevant sensor streams and time intervals as supporting evidence.
Evaluation
The challenge will operate across two tracks: fully-automatic and interactive.
⚙️ Fully-Automatic Track
Participants receive queries in advance and generate results using any method they choose. These results are then submitted to the challenge organizers for evaluation. Please see the list of queries below.
🎮 Interactive Track
This track will be evaluated live during the conference. Participants must solve tasks synchronously and interactively within a limited timeframe. This format follows established competitions such as the Video Browser Showdown and the Lifelog Search Challenge.
Queries
🔍 Event Instance Search
- MM25-EIS01: Find instances of somebody using a portable electric kitchen gadget.
- MM25-EIS02: Find instances of somebody unwrapping a sweet snack and eating it.
- MM25-EIS03: Find instances of somebody telling a joke or making a pun.
- MM25-EIS04: Find winning hands of poker.
- MM25-EIS05: Find instances pouring the last amount of liquid from a beverage container.
- MM25-EIS06: Find instances of somebody complimenting the food.
- MM25-EIS07: Find instances of people mispronouncing the German name of a food item.
- MM25-EIS08: Find instances of people singing.
- MM25-EIS09: Find instances of starting the dishwasher.
- MM25-EIS10: Find instances of someone using the pepper shaker that actually contains salt.
📦 Object Instance Search
- MM25-OIS01: Find the bird-shaped cookie cutter.
- MM25-OIS02: Find the thermal image camera.
- MM25-OIS03: Find the ace of spades.
- MM25-OIS04: Find 'A Christmas Carol' (the book).
- MM25-OIS05: Find the set of colored pens.
- MM25-OIS06: Find a paper airplane made from a white sheet of paper.
- MM25-OIS07: Find a partially eaten apple.
- MM25-OIS08: Find the yellow octopus toy.
- MM25-OIS09: Find the squirrel-shaped tree onrnament.
- MM25-OIS10: Find a kitchen scale.
💬 Question Answering
- MM25-QA01: How many ‘Zimtstern’ cookies were made in total?
- MM25-QA02: Who drew a picture of a ‘Connect Four’ board?
- MM25-QA03: What was the category in the first round of ‘Happy Quiz’?
- MM25-QA04: Who won at Mikado?
- MM25-QA05: What was the ingredient not available (i.e. also not replaced) for the pumpkin risotto?
- MM25-QA06: What open-source movie did Klaus want to watch?
- MM25-QA07: What K-pop band is Cathal into?
- MM25-QA08: Whom did Werner give a small bottle of hand sanitizer?
- MM25-QA09: Which was the name or brand of the wine, of which the empty bottle was used as a rolling pin?
- MM25-QA10: What book did Florian spend the longest time reading?