Action recognition of point-light displays presented with semantically (in)congruent auditory stimuli: behavioral correlates

Dissertação de mestrado em Interuniversitário em Neuropsicologia Clínica e Experimental
Humans are experts in identifying and understanding other's movements. The visual information is often enough for a very accurate action identification. However, actions usually have an associated sound and
thus the integration between visual and auditory modalities can benefit perception. Considering the
improved identification driven by the integration of multiple sensory information, researchers began to
investigate how such inputs are integrated in a unified percept. Following such line of investigation, we
aimed to evaluate if the presence of congruent/incongruent action sounds would improve/impair
recognition of the visual Point-Light Display (PLD) of human actions and if non-biological PLDs
accompanied by action sounds would bias participants into a false perception of visual human action.
Therefore, participants were presented with several masked human and scrambled PLD videos
accompanied by sounds that were either biological or a white noise and asked to judge if the video
depicted a human figure or not and in the affirmative case, they should name the action. Results showed
a significant enhancement for audiovisual biological congruent when compared to the visual biological
paired to auditory noise and unimodal visual stimuli. Similarly, a significantly better performance on action
recognition occurred for the audiovisual biological congruent condition when compared to the unimodal
visual stimuli. Lastly, considering the scramble stimuli we found a significant bias towards the
identification of a human figure for the visual stimuli paired to auditory noise when compared to the
unimodal visual scramble condition.These findings suggest that adding coincident sounds to a human
action visual display impacts human figure identification and action perception in biological PLDs and
that insignificant sounds might lead to confound perception of non-biological PLDs.