I’ve written about synesthetic interfaces before: that is, interfaces that perform a sensory substitution, translating the information normally associated with one sense modality into the phenomenal forms normally associated with another. In my previous work, I’ve usually focused on forms of nonhuman perception and certain modes of perceptual expertise. The release of Perception (The Deep End Games, 2017) yesterday, however, gives me an opportunity to dip into a new topic: disability.
Perception is a horror game about a blind woman exploring a haunted house. Unlike a game such as Papa Sangre (Somethin’ Else, 2010), however—an experiment in audio-only digital game design that has sadly been taken off of the iOS App Store as of this writing—Perception doesn’t court blind and other low-vision players. Rather than featuring robust, binaural sound localization simulation, Perception re-imagines the auditory perception of its blind protagonist Cassie as a kind of sonar vision, thrown up on the player’s screen in spooky, warbly monochrome.
This isn’t the first time games nominally about blindness have been served up to sighted players. In this post, I take up a comparative investigation of Perception alongside Beyond Eyes (Tiger and Squid / Team17 Digital Ltd, 2015), which drops the horror angle in favor of child-friendly, colorful adventure.
Today, I’d like to address a cluster of game user interface design options that I have lumped together under the category of synesthetic interfaces. By this, I’m referring to interfaces that perform a sensory substitution, translating the information normally associated with one sense modality into the phenomenal forms normally associated with another. This is part of a larger interest of mine of examining approaches within game UI design in terms of the epistemic strategies they enact when establishing the relation of players to their avatars, and avatars to their worlds.
The moving image and the graphical user interface: they are often regarded as two separate forms, with their own unique histories, and with resulting divergent aesthetics. Recent years, however, have seen this division beginning to break down, as the GUI has gradually invaded moving-image storytelling.
Representations of the graphical user interface have a long history in science-fiction, from the augmented reality elements that clutter the cyborg’s point-of-view to the fantastic potentials of human-machine interfaces. Continue reading →