Multimodality in VR: A survey
Virtual reality (VR) is rapidly growing, with the potential to change the way we create and
consume content. In VR, users integrate multimodal sensory information they receive to …
consume content. In VR, users integrate multimodal sensory information they receive to …
The role of iconic gestures in speech comprehension: An overview of various methodologies
KG Kandana Arachchige, I Simoes Loureiro… - Frontiers in …, 2021 - frontiersin.org
Iconic gesture-speech integration is a relatively recent field of investigation with numerous
researchers studying its various aspects. The results obtained are just as diverse. The …
researchers studying its various aspects. The results obtained are just as diverse. The …
Hand gestures as visual prosody: BOLD responses to audio–visual alignment are modulated by the communicative nature of the stimuli
During public addresses, speakers accompany their discourse with spontaneous hand
gestures (beats) that are tightly synchronized with the prosodic contour of the discourse. It …
gestures (beats) that are tightly synchronized with the prosodic contour of the discourse. It …
The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech
One of the key features of human interpersonal communication is our ability to integrate
information communicated by speech and accompanying gestures. However, it is still not …
information communicated by speech and accompanying gestures. However, it is still not …
TMS reveals dynamic interaction between inferior frontal gyrus and posterior middle temporal gyrus in gesture-speech semantic integration
W Zhao, Y Li, Y Du - Journal of Neuroscience, 2021 - Soc Neuroscience
Semantic processing is an amodal process with modality-specific information integrated in
supramodal “convergence zones” or “semantic hub” with executive mechanisms that tailor …
supramodal “convergence zones” or “semantic hub” with executive mechanisms that tailor …
Speaker's hand gestures modulate speech perception through phase resetting of ongoing neural oscillations
Speakers often accompany speech with spontaneous beat gestures in natural spoken
communication. These gestures are usually aligned with lexical stress and can modulate the …
communication. These gestures are usually aligned with lexical stress and can modulate the …
Left motor δ oscillations reflect asynchrony detection in multisensory speech perception
During multisensory speech perception, slow δ oscillations (∼ 1–3 Hz) in the listener9s
brain synchronize with the speech signal, likely engaging in speech signal decomposition …
brain synchronize with the speech signal, likely engaging in speech signal decomposition …
Multimodal language processing: How preceding discourse constrains gesture interpretation and affects gesture integration when gestures do not synchronise with …
Previous studies have suggested that a co-speech gesture needs to be synchronous with
semantically related speech (semantic affiliates) for its successful semantic integration into a …
semantically related speech (semantic affiliates) for its successful semantic integration into a …
A speaker's gesture style can affect language comprehension: ERP evidence from gesture-speech integration
In face-to-face communication, speech is typically enriched by gestures. Clearly, not all
people gesture in the same way, and the present study explores whether such individual …
people gesture in the same way, and the present study explores whether such individual …
Motion capture-based animated characters for the study of speech–gesture integration
Digitally animated characters are promising tools in research studying how we integrate
information from speech and visual sources such as gestures because they allow specific …
information from speech and visual sources such as gestures because they allow specific …