In the late 1940s, radio engineer-turned-composer Pierre Schaeffer celebrated a defining property of
audio recording and radio transmission: the ability to separate sounds from their visible sources. This
affirmation cut against the grain of modern thought, for no lesser cultural critics than Martin Heidegger,
Theodor Adorno, and Max Horkheimer had assailed these technologies for dulling our auditory
sensibility. Schaeffer, however, argued that records and radio triumphantly subvert the hegemony of
vision to make possible the experience of “sound as such.” In doing so, Schaeffer continued, they
revive a neglected form of listening he termed “acousmatic,” in deference to the ancient akousmatikoi,
disciples of Pythagoras who were made to listen to their master’s voice while he was hidden behind a
curtain. (1)
Schaeffer’s position remains a significant one within the practice of sound art today. Indeed, any sound art worthy of the name affirms something of this effort to restore to sound its ontological and aesthetic value. (Such insistence on the autonomy of sound and its acousmatic experience is manifested most dramatically now in the work of Spanish composer and sound artist Francisco López, who performs behind a shroud, urges his listeners to don blindfolds, and delivers sonic abstractions that thwart recognition of the environmental sounds from which they are generated.) Yet for the most part, contemporary sound artists and their curators have been interested in negotiating the visual, rather than rejecting it wholesale. In fact, the very tension of such negotiation is what animates this uncertain art form operating between music and visual art, medium specificity and a postmedium condition.
This provocative ambiguity becomes particularly evident as one compares institutional presentations of sound art since its coming to prominence in the late 1990s. Exhibitions such as “Volume: Bed of Sound” (P.S. 1 Contemporary Art Center, New York, 2000), “Rooms for Listening” (CCA Wattis Institute for Contemporary Art, San Francisco, 2000), “BitStreams” (Whitney Museum of American Art, New York, 2001), and the 2002 Whitney Biennial, for example, implicitly adopted Schaeffer’s paradigm, providing banks of headphones or darkened rooms for the acousmatic delivery of audio. A contrary approach was taken in last year’s “Treble” exhibition at SculptureCenter in New York, which foregrounded the use of sound as a tool to link media from drawing to sculpture.
Schaeffer’s position remains a significant one within the practice of sound art today. Indeed, any sound art worthy of the name affirms something of this effort to restore to sound its ontological and aesthetic value. (Such insistence on the autonomy of sound and its acousmatic experience is manifested most dramatically now in the work of Spanish composer and sound artist Francisco López, who performs behind a shroud, urges his listeners to don blindfolds, and delivers sonic abstractions that thwart recognition of the environmental sounds from which they are generated.) Yet for the most part, contemporary sound artists and their curators have been interested in negotiating the visual, rather than rejecting it wholesale. In fact, the very tension of such negotiation is what animates this uncertain art form operating between music and visual art, medium specificity and a postmedium condition.
This provocative ambiguity becomes particularly evident as one compares institutional presentations of sound art since its coming to prominence in the late 1990s. Exhibitions such as “Volume: Bed of Sound” (P.S. 1 Contemporary Art Center, New York, 2000), “Rooms for Listening” (CCA Wattis Institute for Contemporary Art, San Francisco, 2000), “BitStreams” (Whitney Museum of American Art, New York, 2001), and the 2002 Whitney Biennial, for example, implicitly adopted Schaeffer’s paradigm, providing banks of headphones or darkened rooms for the acousmatic delivery of audio. A contrary approach was taken in last year’s “Treble” exhibition at SculptureCenter in New York, which foregrounded the use of sound as a tool to link media from drawing to sculpture.
But, a third strategy, surely the most widespread and significant, has come to prominence in a spate
of recent shows presenting sound under the banner of “synaesthesia,” an aesthetic appropriation of
the neurological condition in which stimulation of one sensory modality triggers involuntary sensation
in another. Sound artists and curators have long flirted with the “synaesthesia” idea. However, this
engagement has lately emerged—problematically, it must be said—as the dominant mode of
conceiving conjunctions between the sonic and the visual. At least five exhibitions have been
organized under this rubric within the past year: the largely overlapping blockbusters “Visual
Music” (Hirshhorn, Washington, DC/moca, Los Angeles) and “Sons & Lumières” (Centre Pompidou,
Paris), which reconsidered the history of modernism as a story of crossovers between sight and
sound; then three smaller shows assembling new work in this vein, “Synaesthesia: A Neuro-
Aesthetics Exhibition,” mounted at London’s Institute for Contemporary Arts last fall, and “What Sound
Does a Color Make?” and “Crossed Circuits,” held in New York this past summer at Eyebeam and the
Hogar Collection, respectively.
What are we to make of this new fascination with synaesthesia? And what are the stakes for the very conception of sound art? To answer, it’s worth noting that the art world’s attraction to sensory cross- wirings is in fact part of a more general cultural formation. In contemporary science, for example, freak occurrences of colored hearing or tactile smell—dismissed as fakery for much of the twentieth century —have suddenly become the subjects of a booming industry in the fields of cognitive psychology and
What are we to make of this new fascination with synaesthesia? And what are the stakes for the very conception of sound art? To answer, it’s worth noting that the art world’s attraction to sensory cross- wirings is in fact part of a more general cultural formation. In contemporary science, for example, freak occurrences of colored hearing or tactile smell—dismissed as fakery for much of the twentieth century —have suddenly become the subjects of a booming industry in the fields of cognitive psychology and
neurology. Developments in contemporary technology also promote the idea of synaesthesia. Brain-
imaging technologies used to explore the phenomenon are, paradoxically enough, themselves
synaesthetic in their psychedelic visual representation of nonvisual sensory phenomena. This quality
points to the more general fact that digital technologies offer, if not a union of the senses, then
something akin: the intertranslatability of media, the ability to render sound as image, and vice versa.
As Friedrich Kittler, who has written extensively on communication technology, puts it: “The general
digitization of channels and information erases the differences among individual media. . . . Inside the
computers themselves everything becomes a number: quantity without image, sound, or voice. And
once optical fiber networks turn formerly distinct data flows into a standardized series of digitized
numbers, any medium can be translated into any other.” (2)
The emergence of sound art as a prominent practice is aligned with this more general revaluation of
the senses and, particularly, of hearing. And the curatorial and artistic interest in synaesthesia is
surely a strategy for dealing with this emergence of auditory culture. Thus far, this strategy has been
complex and ambivalent. On the one hand, the synaesthesia paradigm recognizes and invites sound
as an aesthetic element; on the other, the paradigm still privileges the old order, conceiving sound
under the hegemony of the visual and thwarting the development of a genuine sound art.
However lamentable, this imbalance of media and sensation is true to the neurological experience of
synaesthesia. This condition (and the aesthetic analogy to it) may hold out the ideal of sensuous
plenitude and cross-mixing; yet by far its most common expression is the unidirectional visual
experience of sound. (Sound provoked by sight is extremely rare. (4) In the aesthetic domain, even
when generated to enhance aural experience—for example, the ’60s light shows featured in “Visual
Music” or, today, the iTunes Visualizer—the visual is almost never a mere supplement to the auditory.
As film theorist Christian Metz points out, our syntax and entrenched sensual hierarchy hold us in
thrall to a metaphysics according to which sight and touch signify being and presence, while sound— spatially vague, materially elusive, and temporally ephemeral—signifies absence and can only have the status of a secondary “attribute” in relation to a primary visual and tactile “substance.” (5) Cinema might in principle be a synaesthetic art, an intersensorial conjunction of sound and image. In practice, however, cinematic sound is almost invariably subservient to the image. So too is it with synaesthetic art more generally. Indeed, the dominance of the visual in synaesthetic art corresponds with the prevailing idea that sound-in-itself is unnatural or inadequate, in need of an anchor in the visible.
This situation was already evident in early experiments with the visualization of sound, most prominently those of Ernst Chladni, an important but, until recently, neglected figure. In 1787, Chladni drew a violin bow along the edge of a metal plate covered with a thin layer of sand. The vibrating plate bounced the granules into symmetrical forms: stars, waves, grids, and labyrinths. Chladni’s demonstration made visible and palpable the hitherto elusive and fleeting materiality of sound. Napoleon was so impressed that he put Chladni on his payroll. Friedrich Nietzsche was also intrigued; but he warned of a potential misapprehension of Chladni’s results: “One can imagine a man who is totally deaf and has never had a sensation of sound,” he wrote. “Perhaps such a person will gaze with astonishment at Chladni’s sound figures; perhaps he will discover their causes in the vibrations of the string and will now swear that he must know what men mean by ‘sound.’” (6) Wary of the attempt to reduce sound to sight, Nietzsche insists that the visual and the auditory constitute separate spheres and that the relationship between the two can only ever be a matter of translation or metaphor (in the etymological sense of “carrying over”) that bears the traces of an unassimilated remainder. For Nietzsche, the distinction between the metaphorical and the literal is simply that the latter no longer acknowledges the difference that constitutes it, taking itself to be what it represents. Such literalness is a chief characteristic of the aesthetic discourse of synaesthesia today.
Presented almost tangentially in “Visual Music,” an excerpt from Oskar Fischinger’s film Ornament
Sound, circa 1932, provides a very different and much richer model for sound-image translation. The
German abstract filmmaker drew bands of jagged and undulating patterns across the optical
soundtrack and extending into the visual frame. These forms, read by the projector as both sound and
image, produce corresponding bursts of multitextured and variously pitched noise. Sound ceases to
be a mere accompaniment to image or suture for visual cuts, but instead collaborates directly with
image in the production of a genuine audio-visual experience. Indeed, Fischinger’s forms appear as
stylized versions of ordinary optical sound bands and thus draw attention to the sound track both
visually and aurally.
Such procedures and results not only revive Fischinger but recapitulate the work of video pioneers
Nam June Paik, Steina Vasulka, and Gary Hill—selections by whom were included in “What Sound
Does a Color Make?” to the detriment of the more recent work on view. For example, Hill’s Full Circle
(formerly Ring Modulation), 1978, brilliantly foregrounds both correspondences and differences among
sound, text, and image. A video screen divided into three sections presents several ways of rendering
the sound of a voice that repeatedly drones an “ahhh” sound held for various durations and wavering in
pitch and volume. In the upper left portion of the screen, an oscilloscope image represents the sound
as a jittery, rotating ring; below, a close-up image shows a pair of hands bending a metal wire into a
circle; above right, the screen presents a distorted full-body view of this wire-bending exercise. True to
the theme of circularity, one is never certain which element controls which others. The voice seems to
be attempting to form a perfect circle in the oscilloscope image that represents it, but it could also be
matching the hand movements that seem to mimic its vocal fluctuations. The malleable wire and
quivering ring also reference the vibrating vocal chords; and the title of the piece puns on the verbal
connection between the vocal-visual-manual efforts to form circles and the technical process of
generating dissonance by multiplying electronic signals (ring modulation).
From a quarter-century’s distance, Hill’s piece presents an appropriate directive to sound art today and underscores the deficiencies of facile synaesthetic discourses. The best sound works neither reject the visual nor succumb to it, but instead amplify differences among media and sensory modalities, drawing attention to sound as a semiautonomous power. They are complex engagements with the visual that intensify the moment of translation and the movement of metaphor (in Nietzsche’s sense of the term). For the silence of the visual can cut two ways. It can stifle or, as John Cage taught us, powerfully disclose sound. Exemplary instances of this second sense are recent installations by Stephen Vitiello, which feature suspended speaker cones that tremble inaudibly. At once mouths and ears, these mobilized membranes draw attention to the kinetic energy of sound, the vibrations that constitute its production and reception.
Or consider Christian Marclay’s The Sound of Silence, 1988, a framed photograph of Simon & Garfunkel’s 1965 single “The Sounds of Silence”—displaying, in effect, the record as a mute visual object. The piece is a joke, but an epistemologically and ontologically profound one, the humor of
which consists in an evident confusion of categories: Photograph, object, and text are absurd because they cannot be what they claim they are. Sound is thereby shown to be of another order, one inadequately represented or even foreclosed by the imaginary domain of the visual and the symbolic domain of the written word. Sound is real, Marclay’s work seems to say, something too quickly forgotten by the fantasy of a “union of the senses,” which remains a visual fantasy. Genuine sound art today is fostered not by this consensus but by a dis-sensus that gives sound and hearing their due.
Finally, the new discourse on synaesthesia must be considered against the background of a broad
revaluation of the senses and their traditional hierarchy—particularly the modern supremacy of vision
over audition, sight over sound. Since the 1960s, theorists from Marshall McLuhan and Walter Ong to
Jacques Attali and Thomas Docherty have forecast this sort of revolution in the sensorium. And while
skeptics have treated these claims as purely speculative or merely wishful, there is growing empirical
evidence for them. Witness, for example, the recent profusion of historical and anthropological
scholarship—collected in volumes such as Hearing History: A Reader and Hearing Cultures: Essays
on Sound, Listening, and Modernity (both 2004)—that takes its evidence not primarily from sight, but
from sound. (3)
Consider the example of “Visual Music,” which was not so much a historical as a genealogical effort to
disclose the modernist lineages of current sound-art practice and synaesthesia discourse. The
exhibition was subtitled “Synaesthesia in Art and Music Since 1900”; yet, true to its main title, it
clearly favored the image. (In contrast, “Sons & Lumières” only tangentially flirted with synaesthesia,
rightly subsuming it within a more expansive examination of sound-image relationships in the
multimedia art of the twentieth century.) From the “musical” canvases of Wassily Kandinsky and
Franti?sek Kupka to the musically inspired but silent projections of Thomas Wilfred and Leo Villareal,
“Visual Music” almost solely presented one-way translations from sound into sight. One may
reasonably ask why the curators did not complement these selections with sound-centered classics
such as Walter Ruttman’s film-without-images Weekend, 1930; Luc Ferrari’s Presque rien (Almost
Nothing), 1970, a sonic portrait of dawn in a Dalmatian fishing village; Derek Jarman’s monochrome
film, Blue, 1993; or a Janet Cardiff audio walk. Or why the exhibitions “Crossed Circuits” and “What
Sound Does a Color Make?” similarly centered on the image, focusing attention on screens,
photographs, and drawings that occluded sound by standing in for it, much in the same way that the
mute but visible score came, within modernity, to circulate as the musical work itself.
thrall to a metaphysics according to which sight and touch signify being and presence, while sound— spatially vague, materially elusive, and temporally ephemeral—signifies absence and can only have the status of a secondary “attribute” in relation to a primary visual and tactile “substance.” (5) Cinema might in principle be a synaesthetic art, an intersensorial conjunction of sound and image. In practice, however, cinematic sound is almost invariably subservient to the image. So too is it with synaesthetic art more generally. Indeed, the dominance of the visual in synaesthetic art corresponds with the prevailing idea that sound-in-itself is unnatural or inadequate, in need of an anchor in the visible.
This situation was already evident in early experiments with the visualization of sound, most prominently those of Ernst Chladni, an important but, until recently, neglected figure. In 1787, Chladni drew a violin bow along the edge of a metal plate covered with a thin layer of sand. The vibrating plate bounced the granules into symmetrical forms: stars, waves, grids, and labyrinths. Chladni’s demonstration made visible and palpable the hitherto elusive and fleeting materiality of sound. Napoleon was so impressed that he put Chladni on his payroll. Friedrich Nietzsche was also intrigued; but he warned of a potential misapprehension of Chladni’s results: “One can imagine a man who is totally deaf and has never had a sensation of sound,” he wrote. “Perhaps such a person will gaze with astonishment at Chladni’s sound figures; perhaps he will discover their causes in the vibrations of the string and will now swear that he must know what men mean by ‘sound.’” (6) Wary of the attempt to reduce sound to sight, Nietzsche insists that the visual and the auditory constitute separate spheres and that the relationship between the two can only ever be a matter of translation or metaphor (in the etymological sense of “carrying over”) that bears the traces of an unassimilated remainder. For Nietzsche, the distinction between the metaphorical and the literal is simply that the latter no longer acknowledges the difference that constitutes it, taking itself to be what it represents. Such literalness is a chief characteristic of the aesthetic discourse of synaesthesia today.
As Kittler notes, the ready translatability of digital media encourages this literalness. The fact that all
digital material shares a common base—binary code—supports the illusion that sound, image, word,
and movement can be made identical and interchangeable. What is forgotten is that they can be made
so only via the intermediary of arbitrary mapping formuli decided in advance. This disparity was
evident in the sound-image juxtapositions of numerous pieces in “What Sound Does a Color Make?”
When sound and image pulse together in Granular-Synthesis’s video projection LUX, 2003, for
example, it is because they have been programmed to do so, not because of any real correspondence
between these sounds and images. The same is true of the correspondence between vocal tones and
luminosity in Jim Campbell’s Self-Portrait of Paul DeMarinis, 2003, which uses sound to draw a
pixelated figure with LEDs, and the relationship between the viewer’s movements and the alteration of
image and sound in Atau Tanaka’s Bondage, 2004, which features a mutating digital photograph
projected onto a shoji screen.
While scarcely followed in the past seven decades, Fischinger’s model has been rejuvenated today in
a number of works utilizing audio-visual feedback loops without any intermediary, from Austrian video
artist Billy Roisz’s AVVA, 2004, to installations such as Carsten Nicolai’s Telefunken, 2000, and Scott
Arford’s Static Room, 2003. Arford, for example, begins by generating a palette of video static:
granulated washes, throbbing bands, and pixel fragments. He then runs the video output through the
audio input, producing a matching gamut of noise: dense blasts, dirty pulses, and disintegrated
drones. The translation is effected not by the conversion of color and sound to a neutral digital
substratum, or by the idiosyncratic sensual associations of the artist, but is, rather, effected simply by
the routing of the electronic signal and the medium of display. Herein lies the true potential for a
sound-art discourse steeped in a multisensory approach. Where indirect and arbitrary digital translation
too often attempts to elide the differences between media and sensory modalities, this direct, analog
translation does the reverse, intensifying sensory differences and the materiality of the video medium.
From a quarter-century’s distance, Hill’s piece presents an appropriate directive to sound art today and underscores the deficiencies of facile synaesthetic discourses. The best sound works neither reject the visual nor succumb to it, but instead amplify differences among media and sensory modalities, drawing attention to sound as a semiautonomous power. They are complex engagements with the visual that intensify the moment of translation and the movement of metaphor (in Nietzsche’s sense of the term). For the silence of the visual can cut two ways. It can stifle or, as John Cage taught us, powerfully disclose sound. Exemplary instances of this second sense are recent installations by Stephen Vitiello, which feature suspended speaker cones that tremble inaudibly. At once mouths and ears, these mobilized membranes draw attention to the kinetic energy of sound, the vibrations that constitute its production and reception.
Or consider Christian Marclay’s The Sound of Silence, 1988, a framed photograph of Simon & Garfunkel’s 1965 single “The Sounds of Silence”—displaying, in effect, the record as a mute visual object. The piece is a joke, but an epistemologically and ontologically profound one, the humor of
which consists in an evident confusion of categories: Photograph, object, and text are absurd because they cannot be what they claim they are. Sound is thereby shown to be of another order, one inadequately represented or even foreclosed by the imaginary domain of the visual and the symbolic domain of the written word. Sound is real, Marclay’s work seems to say, something too quickly forgotten by the fantasy of a “union of the senses,” which remains a visual fantasy. Genuine sound art today is fostered not by this consensus but by a dis-sensus that gives sound and hearing their due.
