Video lecture for week 5, class 2 of CMST 10100, “Introduction to Film Analysis.”
Synesthesia
Double Blind
Ian here—
I’ve written about synesthetic interfaces before: that is, interfaces that perform a sensory substitution, translating the information normally associated with one sense modality into the phenomenal forms normally associated with another. In my previous work, I’ve usually focused on forms of nonhuman perception and certain modes of perceptual expertise. The release of Perception (The Deep End Games, 2017) yesterday, however, gives me an opportunity to dip into a new topic: disability.
Perception is a horror game about a blind woman exploring a haunted house. Unlike a game such as Papa Sangre (Somethin’ Else, 2010), however—an experiment in audio-only digital game design that has sadly been taken off of the iOS App Store as of this writing—Perception doesn’t court blind and other low-vision players. Rather than featuring robust, binaural sound localization simulation, Perception re-imagines the auditory perception of its blind protagonist Cassie as a kind of sonar vision, thrown up on the player’s screen in spooky, warbly monochrome.
This isn’t the first time games nominally about blindness have been served up to sighted players. In this post, I take up a comparative investigation of Perception alongside Beyond Eyes (Tiger and Squid / Team17 Digital Ltd, 2015), which drops the horror angle in favor of child-friendly, colorful adventure.
Mapping the Synesthetic Interface
Ian here—
The following is the spoken presentation version of my talk from DiGRA’s 2014 conference in Snowbird, UT. The full paper, as drafted up for the conference’s proceedings, is available here. You can follow along with the visual presentation for this spoken version here.
Today, I’d like to address a cluster of game user interface design options that I have lumped together under the category of synesthetic interfaces. By this, I’m referring to interfaces that perform a sensory substitution, translating the information normally associated with one sense modality into the phenomenal forms normally associated with another. This is part of a larger interest of mine of examining approaches within game UI design in terms of the epistemic strategies they enact when establishing the relation of players to their avatars, and avatars to their worlds.
Lesson Plan: Film Scores and Synesthesia
Ian here—
Spending one week on sound in an Introduction to Film course can be a daunting task. So much vocabulary, and so many new issues to discuss, with only a class or two to dwell on them. What to do?
I like to take this week to introduce the concept of synesthesia—the “bleeding” of one sense into another that results in sensations in one sensory modality being interpreted as impressions in another. It’s a phenomenon that has been studied from the era of classical Greek philosophy up through modern neuroscience, and it has provided inspiration to artists for nearly as long. For understandable reasons, it was something that was on a lot of filmmaker’s minds during the transition to sound cinema. Turning to this topic allows us to rope in stalwarts of classical film theory such as Eisenstein and Vertov, and to freely intermingle experiments in feature filmmaking with more radical experiments in the avant-garde.
In this particular lesson, I focus on music. If one’s spending two days on sound, this leaves another class for sound effects, if one so desires. Continue reading
Intermittent Mechanism 