Skip to Main content Skip to Navigation
Conference papers

The Brain-IHM Dataset: a New Resource for Studying the Brain Basis of Human-Human and Human-Machine Conversations

Abstract : This paper presents an original dataset of controlled interactions, focusing on the study of feedback items. It consists on recordings of different conversations between a doctor and a patient, played by actors. In this corpus, the patient is mainly a listener and produces different feedbacks, some of them being (voluntary) incongruent. Moreover, these conversations have been re-synthesized in a virtual reality context, in which the patient is played by an artificial agent. The final corpus is made of different movies of human-human conversations plus the same conversations replayed in a human-machine context, resulting in the first human-human/human-machine parallel corpus. The corpus is then enriched with different multimodal annotations at the verbal and non-verbal levels. Moreover, and this is the first dataset of this type, we have designed an experiment during which different participants had to watch the movies and give an evaluation of the interaction. During this task, we recorded participant's brain signal. The Brain-IHM dataset is then conceived with a triple purpose: 1/ studying feedbacks by comparing congruent vs. incongruent feedbacks 2/ comparing human-human and human-machine production of feedbacks 3/ studying the brain basis of feedback perception.
Complete list of metadatas

Cited literature [34 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02893898
Contributor : Roxane Bertrand <>
Submitted on : Wednesday, July 8, 2020 - 4:01:23 PM
Last modification on : Tuesday, October 6, 2020 - 3:08:57 AM
Long-term archiving on: : Monday, November 30, 2020 - 3:28:22 PM

File

PEPS-LREC final 2020.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02893898, version 1

Collections

Citation

Magalie Ochs, Roxane Bertrand, Aurélie Goujon, Deirdre Bolger, Anne-Sophie Dubarry, et al.. The Brain-IHM Dataset: a New Resource for Studying the Brain Basis of Human-Human and Human-Machine Conversations. Language Resources and Evaluation Conference (LREC), May 2020, Marseille, France. ⟨hal-02893898⟩

Share

Metrics

Record views

52

Files downloads

27