The influence of virtual reality on voice perception and production
Date
2023
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
Objectives: The purpose was to examine the influence of auditory vs. visual vs. combined audiovisual input on perception and production of one’s own voice, using immersive virtual reality technology. ☐ Methods: Thirty-one vocally healthy men and women were investigated under 18 sensory input conditions, using immersive virtual reality technology. The conditions included two auditory rooms with varying reverberation times, two visual rooms with varying volumes, and the combination of audiovisual conditions, with and without background noise. Speech tasks included counting, sustained vowel phonation, a sentence from the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V; “We were away a year ago”), and the first sentence from the Rainbow Passage. Participants performed the speaking tasks as prompted in each of the 18 VR conditions, randomly ordered. Perception outcome measures were derived from participants’ responses to questions about their self-reported perceptions of their vocal loudness, vocal effort, and vocal comfort in speech in each condition. Production outcome measures were Sound Pressure Level (SPL), and Spectral moments (spectral mean and standard deviation in Hz, skewness and kurtosis) in the same conditions. ☐ Results: Linear Mixed Effects (LME) models were conducted for data analysis, using self-reported vocal loudness, vocal effort, and vocal comfort in percent (0 = ‘not at all,’ 100 = extremely), SPL in dB, and spectral moments in Hz. The reference level used in the models was a baseline audiovisual deprivation condition (no visual input and bone-conduction audio input). Results suggested (i) the perception of increased vocal loudness and effort, and decreased comfort, under conditions with increased room volume, speaker-to-listener distance, audiovisual input, and background noise conditions, and (ii) increased SPL and changes in spectral moments under the same conditions. ☐ Conclusions: Not only auditory, but also visual and audiovisual input influenced voice perception and production in ways that have not been previously documented. The findings contribute to the basic science understanding the role of visual and audiovisual as well as auditory input in voice perception and production, and also to models of voice training and therapy. The findings also set the foundation for the use of virtual reality environments in voice and speech therapy, which we suggest as a remedy for the generalization problem in therapy.
Description
Keywords
Audiovisual, Multisensory, Virtual reality, Voice, Voice production, Voice therapy