NTCIR19-Lifelog Task

Lifelog Subtasks

LSAT - Lifelog Semantic Access subTask

The Lifelog Semantic Access Task (LSAT) is a known-item search task that can be undertaken in an interactive or automatic manner. A text, visual, or multimodal query is given, and participants must retrieve specific moments based on the query in a lifelogger's life. Moments are defined as semantic events or activities.

Task Characteristics:

• Query Types: Textual, visual, or multimodal queries
• Retrieval Target: Specific moments (semantic events or activities) in a lifelogger's life
• Modes: Interactive or automatic
• Topic Types: Known-item search (KIS) and Ad-hoc queries (AD)
• Dataset: LSC'22-24 dataset (same as NTCIR18-Lifelog6)

Evaluation: LSAT uses post-evaluation methodology with precision, recall, and other standard information retrieval metrics. Relevance judgments are created through a pooled approach with binary relevance assessments.

Dry Run Topics: Available for testing. See the Dry Run Topics page for details.

CASTLE Subtasks

CSAT - CASTLE Semantic Access subTask

The CASTLE Semantic Access Task (CSAT) is a search-based task where participants retrieve key interactions or events from multi-modal data (e.g., audio, video, chat transcripts) based on a semantic query. This task focuses on understanding and retrieving meaningful events from collaborative session data.

Task Characteristics:

• Data Modalities: Audio, video, chat transcripts, and other multimodal collaborative session data
• Retrieval Target: Key interactions or events from collaborative sessions
• Query Type: Semantic queries describing activities, interactions, or events
• Dataset: CASTLE dataset (available on Hugging Face)

Evaluation: CSAT evaluation focuses on the accuracy of retrieving relevant interactions or events based on semantic understanding of the query and multimodal data.

Dry Run Topics: Available for testing. See the Dry Run Topics page for details.

CAST-Seg - CASTLE Conversation Segmentation subTask

The CASTLE Conversation Segmentation subTask (CAST-Seg) requires participants to segment collaborative sessions into meaningful conversational units, such as tasks, discussions, or decisions, to enable better analysis and retrieval.

Task Characteristics:

• Objective: Identify meaningful boundaries in collaborative sessions
• Segmentation Units: Tasks, discussions, decisions, or other meaningful conversational units
• Data: Multimodal collaborative session data (audio, video, chat transcripts)
• Dataset: CASTLE dataset

Evaluation: CAST-Seg evaluation assesses the accuracy of segmentation boundaries and the meaningfulness of identified conversational units.

Note: CAST-Seg does not have dry run topics. Participants should refer to task guidelines and dataset documentation.

Recipe Generation

The Recipe Generation subtask challenges participants to generate personal recipes or cooking procedures using multimodal collaborative session data from the CASTLE dataset. This task focuses on understanding cooking activities, meal planning, and recipe creation from collaborative cooking sessions.

Task Characteristics:

• Input Data: Multimodal CASTLE dataset including audio, video, chat transcripts, and collaborative session data from cooking activities
• Data Sources: The CASTLE dataset includes menu schedules showing meal assignments and a Castle Cookbook containing recipes for various dishes (e.g., Irish Breakfast, Pineapple Fried Rice, Bò Kho, Lẩu Gà, Italian Spinach Lasagne, Pumpkin Risotto, Tagliatelle, Chicken Curry, Zürchergeschnetzeltes & Rösti, Cookies, Shepherds Pie)
• Output: Personal recipes or cooking procedures generated from the collaborative session data
• Approach: Creative modelling of lifestyle-driven meal preparation based on multimodal collaborative cooking sessions
• Dataset: CASTLE dataset (multimodal collaborative session data including cooking activities, menu schedules, and recipe information)

Evaluation: Recipe Generation evaluation assesses the quality, relevance, and creativity of generated recipes based on the provided multimodal collaborative session data from the CASTLE dataset.

Note: Recipe Generation does not have dry run topics. Participants should refer to task guidelines and dataset documentation. The CASTLE dataset includes menu schedules and cookbook information that can be used as reference for recipe generation.