Multimodal language processing in human communication
J Holler, SC Levinson - Trends in Cognitive Sciences, 2019 - cell.com
The natural ecology of human language is face-to-face interaction comprising the exchange
of a plethora of multimodal signals. Trying to understand the psycholinguistic processing of …
of a plethora of multimodal signals. Trying to understand the psycholinguistic processing of …
Depicting as a method of communication.
HH Clark - Psychological review, 2016 - psycnet.apa.org
In everyday discourse, people describe and point at things, but they also depict things with
their hands, arms, head, face, eyes, voice, and body, with and without props. Examples are …
their hands, arms, head, face, eyes, voice, and body, with and without props. Examples are …
Speech gesture generation from the trimodal context of text, audio, and speaker identity
For human-like agents, including virtual avatars and social robots, making proper gestures
while speaking is crucial in human-agent interaction. Co-speech gestures enhance …
while speaking is crucial in human-agent interaction. Co-speech gestures enhance …
Virtual reality: A game-changing method for the language sciences
D Peeters - Psychonomic bulletin & review, 2019 - Springer
This paper introduces virtual reality as an experimental method for the language sciences
and provides a review of recent studies using the method to answer fundamental …
and provides a review of recent studies using the method to answer fundamental …
[HTML][HTML] Origins of vocal-entangled gesture
Gestures during speaking are typically understood in a representational framework: they
represent absent or distal states of affairs by means of pointing, resemblance, or symbolic …
represent absent or distal states of affairs by means of pointing, resemblance, or symbolic …
Gesture–speech physics: The biomechanical basis for the emergence of gesture–speech synchrony.
The phenomenon of gesture–speech synchrony involves tight coupling of prosodic contrasts
in gesture movement (eg, peak velocity) and speech (eg, peaks in fundamental frequency; …
in gesture movement (eg, peak velocity) and speech (eg, peaks in fundamental frequency; …
Acoustic information about upper limb movement in voicing
We show that the human voice has complex acoustic qualities that are directly coupled to
peripheral musculoskeletal tensioning of the body, such as subtle wrist movements. In this …
peripheral musculoskeletal tensioning of the body, such as subtle wrist movements. In this …
The quantification of gesture–speech synchrony: A tutorial and validation of multimodal data acquisition using device-based and video-based motion tracking
There is increasing evidence that hand gestures and speech synchronize their activity on
multiple dimensions and timescales. For example, gesture's kinematic peaks (eg, maximum …
multiple dimensions and timescales. For example, gesture's kinematic peaks (eg, maximum …
Energy flows in gesture-speech physics: The respiratory-vocal system and its coupling with hand gestures
Expressive moments in communicative hand gestures often align with emphatic stress in
speech. It has recently been found that acoustic markers of emphatic stress arise naturally …
speech. It has recently been found that acoustic markers of emphatic stress arise naturally …
Co-thought and co-speech gestures are generated by the same action generation process.
People spontaneously gesture when they speak (co-speech gestures) and when they solve
problems silently (co-thought gestures). In this study, we first explored the relationship …
problems silently (co-thought gestures). In this study, we first explored the relationship …