NTCIR19-Lifelog Task

Lifelog Task Submissions

LSAT Submissions

A submitted run for the LSAT task is in the form of a single CSV file per run. Each group can submit up to 10 runs, each as an individual file.

Submission Method: Submission files should be sent (one email per group) to ntcir.lifelog.dcu@gmail.com by the due date with the subject line 'NTCIR-Lifelog LSAT Submission'.

File Naming Convention: The submission file should be named as follows: GroupID-RunID-[Interactive or Automatic].txt, where:

• GroupID: The registration ID of your group at NTCIR
• RunID: The number of the run (e.g., DCULSAT01, DCULSAT02, etc.)
• Type: Label as either "Automatic" or "Interactive"

Submission Format: For every topic, every image considered relevant should have one line in the CSV file. For some topics there will be only one relevant item (one line in the submission), for others there will be many relevant items (many lines in the submission), up to 100. It is also possible that no relevant items are found for a topic, so then there should be no entry in the file for that topic.

CSV Format for Automatic Run:

GROUP-ID, RUN-ID, TOPIC-ID, IMAGE-ID, SCORE (in decreasing order of relevance)
DCU, DCULSAT01, 16001, u1_2016-08-15_112559, 1.0
DCU, DCULSAT01, 16001, u1_2016-08-15_120354, 1.0
...

Run Types:

• Automatic Runs: Runs in which the query is presented to the system and the results generated with no further user input.
• Interactive Runs: Runs in which a user is involved as an active participant in the result generation process.

CASTLE Task Submissions

CSAT Submissions

A submitted run for the CSAT (CASTLE Semantic Access Task) is a single CSV file per run, mirroring the LSAT format but adapted for the CASTLE 2024 collection. Unlike the discrete lifelog images used in LSAT, CASTLE 2024 consists of 15 time-aligned UHD 50fps video streams with audio (10 egocentric cameras worn by the participants and 5 static cameras) recorded over four days by 10 participants, together with 6DoF IMU, GPS, and biometric data and automatically generated speech transcripts. Each group can submit up to 10 runs, each as an individual file.

Submission Method: Submission files should be sent (one email per group) to ntcir.lifelog.dcu@gmail.com with the subject line 'NTCIR-Lifelog CSAT Submission'.

File Naming Convention: The submission file should be named GroupID-RunID-[Interactive or Automatic].txt, following the same convention as LSAT. The extension is .txt even though the content is comma-separated (for example, DCU-DCUCSAT01-Automatic.txt), where:

• GroupID: The registration ID of your group at NTCIR
• RunID: The identifier of the run (e.g., DCUCSAT01, DCUCSAT02, etc.)
• Type: Label as either "Automatic" or "Interactive"

Retrieval unit: CSAT topics ask you to "find instances of…", so each result is a single occurrence of the queried event: a time range on one recording day, seen from one stream. An instance is reported by its time range and source (video ID), with no predefined shot segmentation. Report each distinct occurrence of the event as its own line.

Column Layout: Each line has the following comma-separated columns, combining the instance fields (day, start, end, source) with the NTCIR run identifiers:

GROUP-ID, RUN-ID, TOPIC-ID, DAY, START-TIME, END-TIME, SOURCE, SCORE

• GROUP-ID: Your NTCIR group registration ID (e.g., DCU).
• RUN-ID: The run identifier (e.g., DCUCSAT01).
• TOPIC-ID: The CSAT topic being answered (e.g., CSAT26001), the equivalent of the CASTLE "task" id.
• DAY: The recording day as an integer 1–4 (matching the day1…day4 folders, i.e. the four consecutive recording days 4–7 December 2024).
• START-TIME / END-TIME: The start and end of the event in 24-hour HH:MM:SS wall-clock time (recordings span roughly 08:00–21:00). For a near-instantaneous event, START-TIME and END-TIME may be equal or a few seconds apart.
• SOURCE: The stream/perspective the event is observed from, using the dataset's stream names: a participant's egocentric camera (Allie, Bjorn, Cathal, Florian, Klaus, Luca, Onanong, Stevan, Tien, Werner) or a static camera (e.g., Kitchen, Living1, Living2).
• SCORE: A confidence score used to rank results; lines for each topic must be listed in decreasing score order (highest first).

For every topic, each retrieved instance has one line, ranked by decreasing score, up to a maximum of 100 instances per topic. If no relevant instances are found for a topic, include no lines for that topic. Because the 15 streams are time-aligned, the same event may be visible from several cameras; report the perspective in which it is clearest (you may submit the same event from more than one source only if you consider each a distinct useful result).

Header Row: Do not include a header row; the first line of the file should be the first result. The column names shown in the example below are for reference only.

CSV Format for CSAT (example for one topic, CSAT26001 "playing guitar"):

GROUP-ID, RUN-ID, TOPIC-ID, DAY, START-TIME, END-TIME, SOURCE, SCORE (in decreasing order of relevance)
DCU, DCUCSAT01, CSAT26001, 1, 14:32:05, 14:33:10, Luca, 1.0
DCU, DCUCSAT01, CSAT26001, 3, 19:05:40, 19:06:55, Kitchen, 0.9
DCU, DCUCSAT01, CSAT26001, 2, 11:08:20, 11:08:35, Allie, 0.8
...

Example submission files: Ready-to-use sample runs for pipeline validation (illustrative times and sources only, not ground truth):

• DCU-DCUCSAT01-Automatic.minimal.txt: four topics, 11 instances
• DCU-DCUCSAT01-Automatic.txt: sample automatic run covering all 25 topics (70 instances)

Run Types:

• Automatic Runs: Runs in which the query is presented to the system and the results generated with no further user input.
• Interactive Runs: Runs in which a user is involved as an active participant in the result generation process.

CAST-Seg Submissions

A submitted run for CAST-Seg (Conversation Segmentation) is a single CSV file per run listing the conversational segments your system identifies. Each group can submit up to 10 runs. Formal-run submissions that pass assessor review will be included in a public benchmark release for future conversation-segmentation research on CASTLE 2024; we encourage all CASTLE participants to contribute at least one run.

File Naming: GroupID-RunID-CASTSeg.txt (for example, DCU-DCUCASTSeg01.txt). The extension is .txt even though the content is comma-separated.

Column Layout: GROUP-ID, RUN-ID, DAY, SOURCE, START-TIME, END-TIME, LABEL. Within a session (a given DAY and SOURCE), segments must be non-overlapping and ordered by start time. DAY is an integer 1–4, START-TIME and END-TIME are 24-hour HH:MM:SS, SOURCE is the stream name (e.g., Kitchen, Luca), and LABEL is the conversational-unit type (e.g., task, discussion, decision).

Example:

GROUP-ID, RUN-ID, DAY, SOURCE, START-TIME, END-TIME, LABEL
DCU, DCUCASTSeg01, 1, Kitchen, 09:00:00, 09:12:30, task
DCU, DCUCASTSeg01, 1, Kitchen, 09:12:30, 09:20:15, discussion
DCU, DCUCASTSeg01, 1, Kitchen, 09:20:15, 09:35:40, decision
...

Submission Method: Submission files should be sent to ntcir.lifelog.dcu@gmail.com with the subject line 'NTCIR-Lifelog CAST-Seg Submission'.

Recipe Generation Submissions

A submitted run for Recipe Generation is a single file per run containing the generated recipe for every target dish. Each group can submit up to 10 runs.

File Naming: GroupID-RunID-RecipeGen.txt (for example, DCU-DCURecipeGen01.txt).

Submission Format: For each dish, provide (i) the dish name, (ii) an ingredient list with quantities, and (iii) an ordered list of preparation and cooking steps. JSON is preferred; plain text with the same three fields per dish is also accepted.

Example (JSON):

[
  {
    "dish": "Shepherd's pie",
    "ingredients": [
      "1 kg minced beef",
      "2 large onions",
      "6 carrots",
      "1 kg potatoes",
      "85 g butter"
    ],
    "steps": [
      "Preheat the oven to 200C.",
      "Brown the minced beef with the chopped onions and diced carrots.",
      "Boil and mash the potatoes with the butter.",
      "Spread the meat in a dish, top with the mashed potato, and bake for 25 minutes."
    ]
  }
]

Submission Method: Submission files should be sent to ntcir.lifelog.dcu@gmail.com with the subject line 'NTCIR-Lifelog Recipe Generation Submission'.