Perceptual Illusions · Sound Illusion
McGurk Effect
Can you hear with your eyes?
The experiment
Before you start
- This experiment uses sound. If you can, wear headphones and start with the volume low.
What happened?
The sound of the video is always “ba”. Only the mouth says “va”: each “ba” was recorded separately and laid over a video of “va”, at the exact moment the vowel begins. If you heard “va”, your eyes swapped the consonant.
Tap “Listen without the picture”: with the mouth covered, the “ba” comes through. Then watch again while looking at the mouth. For many people the “va” comes back, even when they know the trick.
Why does your brain do this?
Understanding speech isn’t only about your ears. Your brain combines the sound with the movement of the mouth to decide which syllable was said, and it does so before you notice. In “ba”, the lips close and open. In “va”, the upper teeth touch the lower lip. That difference is easy to see, but in a short sound it is much subtler to hear.
When the two cues disagree, the picture can decide where the consonant was made. It isn’t a conscious choice: you simply hear “va”.
When does this shortcut work well?
In a real conversation, the mouth and the sound match. Seeing a speaker’s face helps you follow what they say in noisy places, like a party or a bar, and people with hearing loss rely heavily on this cue. Because the blending is automatic, the help comes for free: you don’t have to decide to watch the mouth.
Where does it show up in daily life?
In a dubbed movie, the actor’s mouth says one thing and the voice says another, and many people feel an odd mismatch that is hard to put into words. On a lagging video call, the voice arrives out of step with the lips. And understanding someone wearing a mask in a noisy room is harder precisely because the mouth cue is missing.
Deep Dive
Open Nerd Mode
Harry McGurk and John MacDonald described the effect in 1976, with a video in which the sound “ba” went along with a mouth saying “ga”. Many participants reported hearing “da”, a syllable that was in neither the sound nor the picture. That is the classic fusion version.
This experiment’s version, with the sound “ba” and the mouth “va”, follows the work of Lawrence Rosenblum and Helena Saldaña (1992): a mouth that articulates with lip and teeth pulls perception toward that consonant. With a video chosen for being compelling, Saldaña and Rosenblum (1994) obtained a visually influenced percept 99% of the time.
Why the evidence is mixed
The effect is real, but its strength varies a lot. Debshila Basu Mallick, John Magnotti and Michael Beauchamp (2015) tested 165 English-speaking adults with 12 different videos. The illusion rate ranged from 0% to 100% across people and from 17% to 58% across videos. For each video, 77% of participants almost never or almost always fell for it (10% of the time or less, or 90% or more). Those retested a year later answered very similarly (correlation of 0.91).
In other words, not hearing “va” isn’t a defect, and hearing it isn’t an overactive imagination: people differ in a stable way. Recent reviews (Alsius, Paré and Munhall, 2018; Van Engen and colleagues, 2022) question whether the illusion is a good measure of how sight and hearing combine in everyday speech, since it relies on isolated syllables and conflicting sound and picture.
Where it happens in the brain
Michael Beauchamp, Audrey Nath and Siavash Pasalar (2010) used functional MRI to locate, in each participant, the part of the superior temporal sulcus that responds to speech that is both seen and heard. A pulse of transcranial magnetic stimulation at that spot reduced the chance of the McGurk effect, but only within a narrow window, from 100 ms before to 100 ms after the sound began. The same pulse did not disturb the perception of syllables where mouth and sound matched.
Technical terms
- Hearing
- Vision
- Multisensory integration
- Speech perception
Brain regions involved
- Superior temporal sulcus
Discovery
Described by Harry McGurk e John MacDonald in 1976.
Level of evidence
Mixed evidence
The effect exists, but its size, conditions or interpretation vary across studies.
Basis: Basu Mallick, D.; Magnotti, J. F.; Beauchamp, M. S. (2015). Variability and stability in the McGurk effect: contributions of participants, stimuli, time, and response type.
References
- Original studyMcGurk, H.; MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746–748.
- Original studyRosenblum, L. D.; Saldaña, H. M. (1992). Discrimination tests of visually influenced syllables. Perception & Psychophysics, 52(4), 461–473.
- Original studySaldaña, H. M.; Rosenblum, L. D. (1994). Selective adaptation in speech perception using a compelling audiovisual adaptor. The Journal of the Acoustical Society of America, 95(6), 3658–3661.
- ReplicationBasu Mallick, D.; Magnotti, J. F.; Beauchamp, M. S. (2015). Variability and stability in the McGurk effect: contributions of participants, stimuli, time, and response type. Psychonomic Bulletin & Review, 22(5), 1299–1307.
- Original studyBeauchamp, M. S.; Nath, A. R.; Pasalar, S. (2010). fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. The Journal of Neuroscience, 30(7), 2414–2417.
- ReviewAlsius, A.; Paré, M.; Munhall, K. G. (2018). Forty years after hearing lips and seeing voices: the McGurk effect revisited. Multisensory Research, 31(1-2), 111–144.
- ReviewVan Engen, K. J.; Dey, A.; Sommers, M. S.; Peelle, J. E. (2022). Audiovisual speech perception: Moving beyond McGurk. The Journal of the Acoustical Society of America, 152(6), 3216–3225.
Review
This text has not been reviewed by a volunteer yet. Want to help review it?
Found a mistake? Write to [email protected]