Understanding Underwater: the Art and Science of Interpreting Whale Sounds.

By Yolande Harris

Abstract

This paper considers the importance of underwater sound. Making this inaudible environment audible to limited human hearing capabilities demands technical, imaginative and interpretative approaches to sound. Transdisciplinary approaches that treat sound as sonic evidence, suggest a shifted role for the composer and sonic ecologist. My analysis joins three seminal works on whale sound: Payne and McVay’s ‘Songs of Humpback Whales’, André and Kamminga’s ‘Rythmic Dimension in the Echolocation of Click Trains of Sperm Whales’ and Alvin Lucier’s Quasimodo: The Great Lover. Through a critical comparison of the scientists’ use of musical ideas of song and rhythm with the composer’s interest in processes of sound transmission over long distances, the necessity of exploring the contextual nature of sound in the environment becomes apparent. To this end I propose the physiological experience of sound in order to understand the sonic contexts of remote environments, exemplified by artworks from my Scorescapes project, Pink Noise, Fishing for Sound and Swim.

Scorescapes, making the inaudible audible and contextual sound

Scorescapes is an artistic research project on sound and the environment, building on over a decade of my artistic practice.1 The project concerns sound, its image and its role in relating humans and our technologies to the environment. The work is predicated on the idea that sonic ecologists are a part of the environment we study.2 When listening, we cannot assume the passive role of a consumer, but need to recognise the influence of our actions and presence on the environment with which we engage. Acknowledging our systemic relationship among complex networks of interacting ecological parts requires an active approach in which we are an immersed participant rather than simply an observer. This realisation, in turn, demands that we see beyond the conventional limitations of the composer as musical specialist and embrace a transdisciplinary approach towards sound and sonic research.

To complicate matters further, it is especially challenging to be actively engaged in an environment that is inaccessible to human physiology, whether due to being extreme in climate, underwater, or beyond human hearing capabilities. Only a portion of environmental sound occurs within human hearing, so accessing sounds beyond these ranges demands some form of mediation. Making the otherwise inaudible audible through audification, sonfication and visualisation techniques (Harris 2010), requires imagination to think of the presence of sounds one cannot hear, and skills of interpretation in making sense of the sonic environments revealed to our hearing range through these processes.3

Fishing for Sound; Yolande Harris; 2010.

Given this attitude towards direct engagement in the environments we are immersed in, my artistic work forms a central part of a practice-based research method. The Scorescapes project includes audio-visual installations and performances, performative lectures, electronic instruments, sonic walks and collaborations with improvising musicians. In addition, the project has incorporated close interactions with composers David Dunn, Alvin Lucier and Pauline Oliveros, and bio-acoustic scientist Michel André. The audio-visual installation, The Pink Noise of Pleasure Yachts in Turquoise Sea (2010), the performance, Fishing for Sound (2010), and sound and video work Swim (2010/11) exemplify my artistic approach to absorbing the complex topic of underwater sound into a perceptually immersive (rather than didactic) experience, while emphasising the importance of making the inaudible audible. In Pink Noise I experiment with hydrophones (underwater microphones) to question our access to and influence on the underwater environment. Recorded in a National Marine Reserve in Spain, the work reveals the unearthly anthropogenic sounds of pleasure yachts set against an idyllic moving image of the surface of the environmentally protected seascape.4 Conceptually more complex, Fishing for Sound presents a plethora of spatial connections between phenomena originating underwater, in the mind, and from outer space.5 Recorded from a swimmer’s perspective out at sea, Swim captures their rhythm of breathing and the physical motion of the strokes, caught on the surface between air and water, between dreaming and wakefulness.6 (These works are discussed later in more detail. See Videos 1, 2 and 3).

The Pink Noise of Pleasure Yachts in Turquoise Sea; Yolande Harris; 2010.

Through the process of making Pink Noise, Fishing for Sound and Swim, I recognised the central importance of considering the context in which sounds occur rather than exclusively the qualities of a sound in itself. For example, Pink Noise and Fishing for Sound present a wide spectrum of sounds recorded underwater with barely any post-production editing or isolating of individual sounds. The effect is a very dense sonic texture reminiscent of how one may actually hear it in the field. This is in contrast to many field recordings recorded and edited for artistic purposes, which typically proceed by splitting up the soundscape into isolated sonic events. That standard editing approach, combined with playback in a location different from the recording, effectively isolates the sound from any suggestion of its original context. The same approach characterises most scientific research on bioacoustics. Yet in working with environmental sound, an understanding of the contextual nature of sound, in terms of spatial acoustics and interaction with other sonic ecologies, is crucial. This insight builds on theorist Brandon LaBelle’s notion of the ‘relational’ quality of sound (LaBelle 2009), where sounds can only be understood in relation to the broad environment in which they occur. One of the key questions then becomes, to what extent can such notions of a relational, contextual approach to sound in the environment extend to physically inaccessible environments such as underwater?

Swim; Yolande Harris; 2010 / 2011.

This paper offers some answers to this question. I briefly review the salient characteristics of underwater sound and consider the use of sound as ‘sonic evidence’. My analysis of whale sounds aims to reveal contrasting and complementary ways of interpreting and understanding underwater sound through the overlap of practices of music and science. Whales, one of the most studied marine species, provide examples of both scientific and musical experiments into underwater sound. The overlapping interest of scientists using musical metaphors for interpreting data, and composers exploring scientific and acoustic properties, raise important implications for composers, sound artists and sonic ecologists working today.

The discussion addresses three seminal works on whale sounds – two by scientific teams and one by a composer – that have not previously been considered in relation to each other. Particular attention is paid to their different approaches to sonic context. I compare Payne and McVay’s analysis of Humpback Whale sounds as ‘songs’ (1971), with marine scientists André and Kamminga’s work on decoding the click trains of a Sperm Whale pod using rhythmic analysis (2000), and composer Alvin Lucier’s interpretation of sending sounds over extremely long distances in Quasimodo: The Great Lover (1970). The first deals with notions of song and melodic pattern structure revealed through visualisation techniques, the second with revealing communication through rhythmic structures in the whale’s clicking sounds, and the third with processes of sound transmission through the whale’s environments with an emphasis on their sonic context.

Of course, each of these projects has different aims, methodologies and outcomes. It is clear that the scientists are influenced by music, and the composer by science, but what can be learnt from the comparison beyond this rather superficial observation? By placing these works next to each other, they resonate together and offer new perspectives on our methods and relationships to the underwater environment through sound. This is in contrast to musician David Rothenberg’s recent work on whales and his approach of “interspecies jamming” as active engagement, where the musician plays a clarinet through an underwater speaker system to the whale and listens through a hydrophone to its ‘responses’ (Rothenberg 2008a, 2008b). Rather than attempting to directly interact with whales, I propose developing a heightened sensitivity to a contextual experience of sound in specific environments. This perspective offers a more productive approach towards understanding the immersive experience of underwater sound. The works discussed in this paper lead away from a certain anthropomorphising of whales, understanding them in terms of human song and musical structures, towards an immersion in the sonic process, medium and context the whales inhabit. My own artistic projects, Pink Noise and Fishing for Sound, explore this approach to underwater sonic context through installation and performance. Such an immersion in sound may lead to more contextual approaches to understanding and exploring the underwater environment both in arts and sciences.

Understanding biological ecosystems through underwater sound

The assumption that underwater is silent has been turned on its head in the last decades as scientists begin to understand just how crucial sound is to aquatic life, in a largely dark environment where it is used to detect motion, currents and prey, and to communicate. It continues to be an extremely difficult area to study, as the sound source can often be undetectable. Moreover, little is known about how sounds are produced or used and what they might mean. In The Soundscape R. Murray Schafer devotes only two pages to “the sounds of water creatures”. He writes, “many fish have no sound producing mechanisms and no developed organs to hear sounds” citing the few exceptions to this as if characters from an alien world (Schafer 1977, p.37). It is striking how limited knowledge on underwater sound was at that time, even by specialists in environmental sound like Schafer.7

Although research has developed dramatically since Schafer’s writing, underwater sound is an extensive area about which we still know relatively little. Bio-acoustician Michael Stocker (2002) examines research since 1950, giving an extremely informative overview of the issues of underwater sound, including the characteristics of sound propagation in water, different species’ use, production and sensing capabilities of sound, and the quality and effect of human made sounds in the ocean.8 However, even with this knowledge he states, “while considerable efforts are being made to understand the auditory perception of sea animals, our understanding is miniscule compared to the vast diversity of sea animals and their adaptations to sound” (Stocker 2002, p.18).

Stocker describes the breadth of the topic, the alien nature of the environment to humans and the importance of sound. Because of the density of water, sound travels five times faster in water than in air. Because light levels are very low, sound is the primary sensory faculty for biological life. Many species of fish, crustaceans and molluscs can sense both wave motion and particle motion at such sensitive levels as to detect currents, tides and approaching prey. Sound can travel vast distances underwater, reflected through layers of the ocean, a feature used by whales for communication (and probably used by other species for navigation). Cetaceans and other marine mammals use highly developed echolocation to navigate and hunt, and some species, like the snapping shrimp, stun their prey by sound. Abiotic sounds (natural – sea, storm), biotic sounds (animal), and anthropogenic sounds (motors, seismic explosions, sonar testing) form the three most basic categories of bioacoustics. Underwater sound is in no way bound by human sense perception, extending well into larger and smaller scales, frequencies, time-frames, spatial dispersion and volumes (Stocker 2002).

Given the field’s infancy, it is not yet known how the proliferation of anthropogenic sound may affect the long-term development of marine organisms and their larger ecology (Slabbekoorn 2010; André 2011). Marine bio-acoustic scientist Michel André clearly identifies the problem and suggests that research on cetaceans offers a particularly fruitful line of inquiry into the sustainability of marine ecosystems.

Ocean noise has always existed, both in natural and biological forms. Without any doubt, due to its recent and uncontrolled character, the massive introduction of artificial sound sources at a large scale has become a threat to its balance, more importantly than any other pollution found in the marine environment. Cetaceans, as top predators of the food chain, have evolved for millions of years on their acoustic perception of the environment and can be considered as bio-indicators of the acoustic balance in the oceans. Understanding how marine mammals perceive their environment and unraveling their communication methods means investigating for the conservation of the marine ecosystems and the development of sustainable human activities in the sea (André 2010, webpage, my emphasis).

These ideas on sound in the ocean suggest that its impact is potentially of greater threat to marine ecologies than toxic waste, oil spills and other forms of pollution.9 Such concerns suggest possible roles and responsibilities of composers, sonic artists and sonic ecologists acting towards sustainable environments.

Underwater sound, sonic evidence and the composer

Underwater sound intensifies the relationship between human dependency on technological mediation and biological life. Our relationship with the underwater environment is fundamentally defined by how alien it is, an uninhabitable medium, which we can access through active imagination and immersion. In the underwater environment our physiological human limitations are challenged and our only knowledge of these environments and their sounds must be mediated by technological extensions. The implications for sound research and the role of the composer are considerably more radical and expansive than those of land-based soundscapes where human listeners can experience the sounds directly. Exploring this sonic arena requires investigating the history, science, technologies and myths of underwater sound.

Every underwater sound raises questions about its origin, function and potential meaning, for both a species embedded in that environment and a researcher listening to it. Sound is, therefore, listened to in order to find answers. In the case of the whale and other cetaceans, sound is a primary source for researching the animals’ lives, behaviours, communications, intelligence and interactions within ocean ecologies. Sound contains clues and can be used to prove or disprove hypotheses. In this sense sound is a witness, or ‘evidence’ to be unraveled within its larger context of underwater environments and ecologies. These are sounds that are entirely embedded in their context, a context in which humans are alien and have difficulty accessing. In order to make sense of sound in this environment researchers must piece together its function from highly mediated sonic evidence, by correlating the qualities of the sounds themselves with behavioural patterns and contextual knowledge of the environment.

The role of sound within this context is far from what is generally considered as the role of sound within musical practice. However, to incorporate such ideas into one’s musical practice may lead to fruitful lines of inquiry. Understanding scientific approaches to underwater sound can offer composers alternative perspectives on the role of music in human terms. It may be possible to go beyond the techniques and common presentation strategies of artistic field recording to include and further develop concepts and techniques specific to underwater sound. And by developing a deeper understanding of scientific techniques, composers will be able to contribute more effectively to research into underwater sound.

The whale as a stage

Research on underwater sound is intimately bound up with the story of the whale. These underwater mammals are known to use sound in a very sophisticated manner. Humans have only recently been able to hear whale sounds and still have very limited understanding of them. The whale’s popular appeal, our “natural human empathy for these intelligent, air-breathing creatures”, contributes to increased funding for research, often to the detriment of urgent research into other species (Stocker 2002, p.16).  Nonetheless the power of this animal in human imagination, the whale’s function as “a touchstone for our common knowledge” (Stocker 2002, p.18), leads us to further understanding of underwater environments and underwater sound. Interest in the whale goes far further than scientific research endeavours to find out about the specifics of the animal, its sounds, its context and social behaviours. The mythic resonance of the whale often overshadows its qualities as an animal species in its own right.

The whale is an imaginative springboard, a stage on which myths of environmental destruction have been played out. It has catalysed environmental movements and even suggested possibilities of interspecies communication (Lilly 1962; Bateson 1972). The whale seems to re-emerge at moments of intense environmental awareness, for example Herman Melville‘s Moby Dick (1851) coincides with Thoreau’s Walden (1854). The release of Songs of the Humpback Whale, the first publicly available recordings of ‘whale songs’ in 1970, coincides with the founding of Greenpeace in 1971, the growing environmental movement of that era, as well as with the concurrent development of soundscape studies by the Acoustic Ecology movement. Similarly, a number of recently published books on the whale, including David Rothenberg’s Thousand Mile Song (2008) and Philip Hoare’s Leviathan, or The Whale (2008), coincide with current debates about climate change, sustainability and the environment. The figure of the whale again provides a stage on which these debates and questions can be dramatised.

Human interaction with the whale and cetaceans in general crosses art, science and activism. However, the status of cetaceans as popular icons ironically makes them a difficult topic to research as they are often dismissed as merely ‘charismatic’ animals such as the cetacean star of the film Flipper (1963). A plethora of cetacean projects founded on this semi-mythical notion – and often incorrect scientific information – do a disservice to this important area of study. Alongside artistic approaches, run scientific investigations into communication through sound in underwater environments. But even these researches seem to generate myths of their own.10 However, composer David Dunn refers to the necessity of this combination of myth and science. Calling on Gregory Bateson’s notion of double description, he quotes, “The richest knowledge of the tree includes both myth and botany” (Dunn 2008).11 The various interpretations of the figure of the whale – in science, as song and music, as spatial composition, and as encompassing long distance and interspecies communication – gain richness through the interaction between the whale as a mythical and romantic figure and scientific hypotheses.

Instances of scientists calling on the help of music to analyse whale sounds are rare and controversial but also illuminating. They raise critical questions about ‘music in nature’, ‘beauty’, and the function of music as a communicative or even evolutionary force. Dunn’s opinion that, “Music is one of the most profound means we have for growing the capacity to perceive the world through sound” (Miller 2007, p.14) highlights the tensions between our definitions and understandings of the role of music and its relationship to sound in the environment.

Humpback whale songs

The seminal paper and accompanying recordings, ‘Songs of Humpback Whales’, by Roger Payne and Scott McVay, remains one of the most clear and accessible reports on whale sound and is worth examining in detail as it has been highly influential in both science and music. It is significant for a number of reasons: firstly, it offered the first publicly available recordings of whale sound; secondly, it used a primitive visualisation technique to discover and analyse the sound patterns; and thirdly, it uses simple musical analysis to describe whale sounds in terms of songs, themes, phrases and units. The authors describe a series of recordings of humpback whales over a number of years from the coast of Bermuda, “It is from these studies of the herd sojourning these waters that we have become aware of the humpbacks’ most extraordinary feature – they emit a series of surprisingly beautiful sounds,” (Payne and McVay 1971, p.585).

Drawing on aesthetic musical notions, these “surprisingly beautiful sounds” are described as “the humpbacks’ sonic repertoire”. Payne and McVay justify their use of the term ‘song’ by referring to a report by Broughton (1963) on classification of animal sounds where he defines song as “a series of notes, generally of more than one type, uttered in succession and so related as to form a recognisable sequence or pattern in time.” (Payne and McVay 1971, p.590). Rothenberg in Thousand Mile Song opines that, “too many whale scientists consider beauty to be too subjective to trust that term to describe the sounds they spend years studying … it’s a shame they forget that nature offers up beautiful music as well” (Rothenberg 2008, p.133-135). But how helpful or misleading is this discussion of whale sound in relation to human music? To judge an idea of musical beauty in natural sounds, when ideas of beauty are contentious even in human music encourages an interpretation of whale song as some kind of animal form of music making, overruling important questions about communication. Such an attitude suggests a largely anthropogenic approach towards the topic, a circular logic that reflects back on our own notion of what music is.

As with most animals, extraordinary differences in the relative hearing capabilities between whales and humans result in inevitable compromise in the analysis of these sounds, a fact that emphasises the importance of making the inaudible audible or visual (Harris 2010). The association of whale sound with human music forms the basis of Payne and McVay’s analysis, even though the whales emit frequencies beyond our hearing range and outside of our temporal perceptual ability. Recognition of this fact is apparent throughout their paper, noting that further analysis “must await recordings on equipment sensitive to ultrasound.” (Payne and McVay 1971, p.594). On first listening to these humpback whale sounds one has “the impression of an almost endless variety of sounds” because the temporal length is beyond what we can easily hold in our minds. Devising a strategy to analyse this material presented significant challenges at that time.

Payne and McVay’s discovery of song structure came from a technique of visualisation by the spectrographic analysis of the recordings, a graphic system whereby frequency is represented on the vertical axis, time on the horizontal axis and amplitude by contrast or colour. The “exceedingly tedious process” involved extracting 9.6-second segments of tape-recorded sound and analyzing them on a spectrograph, to graphically represent the frequency, amplitude and time in a visual image (Payne and McVay 1971, p.592). These printed segments were carefully matched, glued together and reduced in size to make them manageable to see altogether as an overview. The composited spectrograms were then traced onto new sheets of paper by hand, to clarify the whale sounds from the background ocean noise and echoes. This level of interpretation of the images of the sounds removed the parameter of amplitude, leaving diagrams representing frequency (pitch) over time. Interestingly, this choice of what information to remove from the spectrograms resulted in an interpretation of the whale song that conforms to the basic parameters of Western musical notation, implying that the sonic utterances of the humpback whale can be understood in terms of a human, culturally specific genre of music. Ironically, to depict something sonic in a visual form that is conventional for music, and then “discover” music in it, appears to be a tautology.

Payne and McVay’s spectrogram notations of whale recordings seem to implicitly suggest a reading mode like a graphic musical score, where the repeating patterns become visible, analysable and comparable to other notations. The first form of graphic analysis presented is the entire song in both spectrogram and graphic tracing. From this they deduced a hierarchical scheme of temporal structure summarised as follows: “subunit < unit < phrase < theme < song < song session”, the first of which is too small to be detectable by the naked ear, the last of which can continue for hours. (Payne and McVay 1971, p.591) The structure is clear, recognisable and repeatable, but there is distinct variation in the detail rather than the overall. They discovered that the sequence of themes stays consistently in the same irreversible order. The place of most variation occurs in the phrases that make up the distinct themes.

The second form of graphic analysis examines these slight variations in phrases. By layering phrases vertically above one another as they appear in temporal sequence, we see how these phrases “systematically change, or ‘evolve’, with each successive repetition during the theme” (Payne and McVay 1971, p.593).12 The visual presentation is suited to reveal the qualities of the phrase evolution, which in one theme is almost like stretching; in another, more like addition. It not only shows the sounds made, but also the variations in the spaces between sounds. In one of their example themes, the phrases seem to repeat with great regularity, suggesting what composers would term a recognisable rhythm. However, one of their example themes, named ‘Theme 2’, is interesting because it does not ideally fit into this analysis as it consists of one long phrase. “It may consist of a great variety of sounds, but all, or most, of them are ascending frequency sweeps or brief (less than one second) high-frequency squeaks or chirps” (Payne and McVay 1971, p.593). ‘Theme 2’, therefore, is a strange phrase/theme because there can be so much variation in the detail of spacing between sounds or units. The “interunit spacing” is only mentioned in passing and is not given consideration as a potentially important aspect of the sounds, the assumption being that the sound alone contains the information, the ‘on’ rather than the ‘off’ (Payne and McVay 1971, p.592).

The inclusion in the paper of both the spectrograms and the graphical tracing side-by-side enables the reader to compare the two levels of visual interpretation. Moreover, by noting what is excluded in this process of translation from one graphic form to another it becomes apparent that their approach diminishes the environmental contextual information within the sound recordings. Of the elements visible in the spectrograms but excluded in the tracings, the most prominent are the contextual and spatial characteristics of the underwater sound environment that are clearly audible when listening to the recordings. In the recording Songs of the Humpback Whale (1970), which includes a voice-over describing the chief characteristics of their discoveries, the echoes in particular are mentioned, “the water is very deep and the sounds are echoing off the under-surfaces of waves and from the submarine canyons and ridges on the island’s slope” (Payne and McVay 1970, track 1,). These echoes are visible in the spectrograms, as shapes with a shadow that repeats three times in very close succession, but are excluded in the tracings as they are distracting to the visual clarity required to recognise the patterns in structure. Dynamite blasts occurring in pairs every ten minutes are also excluded. Based on extended listening, Payne and McVay conclude that “the blasts do not have any detectable effect on the whale’s rendition of its song” (Payne and McVay 1971, p.586).

The sounds the whale actually emits are prioritised over what is heard and how the sound behaves and transforms in interaction with the space and other sonic events in it. This is using an idea of communication which requires removing noise – unwanted interference – from information. The more we accept the relativity of sound and theorise the relation between sender and receiver in terms of spatial context rather than direct communication, (as examined subsequently in both André and Kamminga’s work on sperm whales and Alvin Lucier’s Quasimodo) the less we can afford to exclude a consideration of the specific characteristics of the underwater context within which the sounds occur.

Sperm whale RIME’s

I discussed this notion of the spacing between sounds as carrying information, with Michel André in relation to his research into the possible communicative function of sperm whale clicks (2010). Substantiating my above critique of Payne and McVay, André’s idea about the relative importance of considering the relationship between non-sounding to sounding in sperm whales became the basis for his RIME hypothesis.

In their paper on echolocation in sperm whales André and Cees Kamminga report research into the rhythmic function of clicks (very short, sharp pulses covering a broad frequency range) used for echolocation, (a way of locating and navigating by listening to changes in returning echoes), and suggest their simultaneous potential for communication and identification (André and Kamminga, 2000). This research is significant in its implications for whale communication, as well as for sonic research methodology. The authors combine scientific techniques of analysing the time intervals between clicks to reveal dominant rhythmic patterns, with the listening observations of a Senegalese drum master, an expert in identifying dense polyrhythmic patterns. By conducting these rhythmical analyses on the apparent cacophony of sperm whale clicks, they drew on musical concepts of rhythm, which became what they call RIME (rhythmic identity measurement), to unearth potential uses of these sounds for communication.

Payne and McVay distinguish different kinds of sounds produced by the Mysticete or baleen whales, such as the humpback or fin whale, and those made by the Odontocetes or toothed whales, which includes the sperm whale, killer whale, dolphins and porpoises. Unlike the varied ‘song’ of humpback whales (Megaptera novaeangliae), sperm whales (Physeter macrocephalus) emit ‘trains’ of clicks. Regularly spaced click trains were thought to be used for echolocation, and the observation of moments of irregularly spaced clicks, named ‘codas’ (another musical term), were considered for communication. By contrast, André and Kamminga argue that “the exclusive function of echolocation attributed to the series of usual clicks might be too restrictive” (André and Kamminga 2000. p.164).

They suggest that the clicks may play an important role in helping large groups of sperm whales communicate over distances of several kilometres. “The social character and the cohesive behaviour of the sperm whale suggest a continuous exchange of information which cannot be based on visual cues, given the great distances over which the whales are separated while looking for food and the virtual absence of light at foraging depths.” (André and Kamminga 2000, p.164) The scientists noticed that during foraging they emit these click trains (not the ‘codas’) and that the whales never parted further than six miles, perhaps to stay within acoustic range of their pod. Is it possible to deduct from their sounds if and how they communicate through them? Much like the first listening of the humpback songs, the clicks from a pod of sperm whales appear to be a cacophony with no discernible order. If they contain some communicative information beyond echolocation, how could we recognise this? André and Kamminga hypothesise that perhaps “the temporal aspects of these signals are crucial to information transfer, since pulse timing is less subject to environmental distortion than wave form” (André and Kamminga 2000, p.164). By considering the contextual aspects of group behaviour in relation to the clicks and the way sound travels in the underwater environment, they began to investigate possible rhythmic patterns unrecognisable to humans. This approach stands in contrast to Payne and McVays’ interpretation of humpback whale sounds which excludes the broader underwater sonic context within which the whales ‘sing’.

The most radical and controversial aspect of André and Kamminga’s work seems to be in its hybrid, interdisciplinary methodology that combines both high-end analysis techniques accepted by the scientific community, and direct auditory observations by a human listener expert in polyrhythmic music. André intuited a connection between the rhythmic expertise and social cohesion of West African drumming and the possible functions of sperm whale clicks. He played a sample of the recordings to the Senegalese drum-master Arona N’Daye Rose, who was “spontaneously able to separate and identify the individual whales in the sample recording through their strong individual rhythmic structure.” N’Daye Rose deduced from first listening the number of whales in the group and detected a dominant rhythm around which the others were organised “belonging to what he called the leader of the group, in reference to the organisation of the rhythmic structure of an African tribe” (André and Kamminga, 2000; 164).13)14

Although the musician could identify these social details communicated through rhythms, André needed to have that observation corroborated by scientific analysis. N’Daye Rose’s analysis was “spontaneous” but it took the research team months to prove. Their paper details these levels of analysis which identify the “pulse repetition frequency (PRF)” and the “inter-click interval (ICI)” as parameters for calculation, from which they pull apart, or parse, the click trains of individual whales revealing their distinct acoustic signature. From this they determine the idea of RIME, or Rhythmic Identity Measurement, which they suggest is a learned behaviour, much like the West-African drumming traditions, shown by each member of the group. Crucially, it “identifies each individual by the rhythm of its acoustic signals – a time parameter – and not through the shape of the signal wave forms” (André and Kamminga 2000, p.166). Again in contrast to Payne and McVay’s 1970s ‘songs’, this ‘RIME’ project shows how much information can be gleaned from the spaces between the sounds, and the contextual information of behaviour and medium, rather than from only considering the content of the emitted sound itself. It also demonstrates the value of music’s knowledge to scientific research on sound.

Every new research step seems to reinforce a sense of the previously unimagined complexity of the underwater environment revealed through its sound. Payne and McVay opened up a world of sound and questions of structure, behaviour and meaning in humpback whale songs, creating a basis for future researchers to build upon. André and Kamminga suggest that the sperm whale clicks actually have a double function: echolocating and communicating at the same time, a process that demands that the whale “distinguish its own echoes against the background of other whale click trains” (André and Kamminga, 2000; 166). This insight opens up unimagined complexity and sophistication in the whales’ connection between sound and its environment, and their approach is applicable to research on other marine species.

Alvin Lucier’s Quasimodo the Great Lover

These scientific examples of humpback whale songs and sperm whale RIME’s open up ideas and questions around visualisation techniques and musical scores, long distance sounds, and possible relationships between science and art. Many musicians were inspired by the scientists’ recordings of humpback whale sounds, such as George Crumb’s Vox Balaenae (Voice of the Whale) (1971) in which the composer explores extended techniques of piano, flute and cello to imitate the kinds of vocalisations produced by the whale. More recently, the work of musicians and composers, including Rothenberg and Dunn, reference André’s scientific research. The aspect of the humpback whale songs that inspired Alvin Lucier to compose Quasimodo The Great Lover (1970) was the whale’s ability to send sounds over very long distances. Lucier attended a lecture by Roger Payne at the University of California Santa Barbara in 1969, prior to the publication and record release. His impressions of that lecture exemplify the influence these recordings had on composers of the time.

While I was there, Roger S. Payne came to give a lecture-demonstration and play his recent recordings of whale music. I, like everyone else, found it very beautiful. What struck me more than the sounds, however, was the ability of whales within a species to communicate with one another over tremendously long distances, across ocean basins in some instances. They do this by echoing their sounds within a specific temperature layer in the sea so that the sound doesn’t get absorbed into the bottom of the ocean or dissipated out through the surface. I was very impressed by that. So instead of imitating the sounds of the whales, or using Payne’s recordings, I imitated the feature that struck me strongest, their amazing long-distance sound-sending ability (Lucier, 1995: 112).

Lucier transforms this “long-distance sound-sending ability” into a musical performance that can be set up to travel through almost any medium, linking different acoustic spaces by relays of microphone-amplifier-loudspeaker, accumulating the subtle differences of a sound that is passed through them. Quasimodo the Great Lover is the last in a series of four works “which explored the acoustic characteristics of natural and architectural spaces” (Lucier, 1995: 428). The other three works included Chambers (1968) for resonant objects as portable environments, Vespers (1968) exploring spaces using echolocation devices inspired by bats, and I am Sitting In A Room (1970) based on a spoken text that is re-recorded and played back within the same space over and over again. The specific task of Quasimodo is to send a sound over a very long distance by building a system out of a signal, a medium and a receiver.  Multiplied over on itself such a system becomes a way to send the sound further than the limits of the medium allow, like a chain or relay. But in the process the sound collects artefacts or discrepancies from the system itself. I am Sitting In A Room and Quasimodo the Great Lover are both systems that explore how the accumulation of sound in loops can bring out the inherent sonic qualities of acoustic spaces. In a kind of mirror of each other, the first explores the resonant properties of a single space as activated and transformed by a voice, playback and recording loop, the second by using a repetitive process to string spaces together into an elongated form of transmission. One system is a closed loop feeding back on itself, the other moves in a forward direction. Quasimodo was inspired by the ability of the humpback whale to make sound that is heard hundreds of miles away from its source, using the temperature layer in the oceans that allows the least diffraction of the sound wave, and the most suitable frequency and amplitude levels to derive the greatest effect from the medium. Thinking beyond the medium of water, however, Lucier extends the possibilities for using different media dedicating the score “for any person who wishes to send sounds over long distances through air, water, ice, metal, stone, or any other sound-carrying medium, using the sounds to capture and carry to listeners far away the acoustic characteristics of the environments through which they travel” (Lucier, 1995: 318).

The system of Quasimodo consists of a chain of microphones and loudspeakers that passes from the first space where the sound begins, through adjacent spaces to the final performance space. In each space the loudspeaker and microphone are placed as far as possible from each other. Playing the starting sound activates the acoustics of the particular space, which is recorded by the microphone. This accumulated sound is passed to the next space by means of an audio cable where it is reproduced on the next loudspeaker. The adjacent space is likely to have totally different acoustics properties. In this way the sound of one space becomes the input for the following resonating sound space. This relay system can continue over any distance and through any number of spaces, collecting a sound that is transformed by each acoustic space it travels through. The end result, at the final location of the chain, is an accumulative sound based on transfer over distance by means of alternating transduction of sound from electronic signal to sound waves transmitted through a medium such as air, water or stone.

The specific sound input at the beginning of the chain is the first stage of this “system” (as Lucier calls it) of sound transmission, transduction and acoustics. Lucier builds directly on Payne and McVay’s analysis of the humpback whale song, using it not as a sound source in itself, but rather as a model for potential sounds and evolving structures. Sounds are not limited to vocal or instrumental ranges, timbres, envelopes and durations but can be modified by electronic, mechanical or any other means at the input stage only. Beyond the input source and mediating equipment, only the acoustic qualities of the environments will change the sound. In doing so Lucier emphasises one ‘voice’ as the source of the sound, echoing a whale’s ability to communicate and send its voice over vast distances.

The text score of Quasimodo emphasises the contextual aspects of sound, the way interaction with the environment in which the sonic signal occurs affects and transforms the specific qualities of that sound. Described by Lucier as “a guidebook of sounds suitable for acoustic testing, with suggested procedures for putting them together” (Lucier, 1995: 112), the form of the score is in three noticeable sections. The first describes all the possibilities for how and where to set-up the piece; the second, how to consider the kinds of sounds and developments to use as input material at the beginning of the chain; and the third, opening up further possibilities for variations and future instantiations. Far beyond the technical specifics, the score lists diverse kinds of spaces, environments and contexts for this to explore, “prairies, glaciers, or ocean basins … rock formations within faults, detached railroad cars on sidings, the rooms, foyers, and corridors of houses, schools, or municipal buildings … libraries, laboratories, cafeterias, offices …” (Lucier, 1995: 318).

The score outlines the kind of sounds to make and the structural progression of these sounds in the system. Both are directly related to whale sounds, with striking parallels, not only to Payne and McVay, but to that of André and Kamminga thirty years later. The sound source is described abstractly, “compose a repertory of simple sound events such as … upward and downward sweeps … accelerating or decelerating pulse trains, upward sweeps followed by tones of short duration …” (Lucier, 1995: 320). The score is more specific about the possible structures of these initial sounds which emphasise a gradual, cumulative evolution. Two aspects are prominent. Firstly, the way the sound events in Quasimodo evolve through variation is reminiscent of the way in which Payne and McVay described how whale songs “systematically change, or ‘evolve’ with each successive repetition of the theme” (Payne and McVay, 1971; 593). Secondly, the consistently irreversible order of themes in whale songs is reflected by  “taking care not to reverse the direction of a variation between two adjacent sets” (Lucier, 1995: 320). Perhaps most importantly however, unlike Payne and McVay’s analysis of Humpback whale song, but more like André and Kamminga’s analysis of Sperm whale click trains, the transformation of the sounds in Quasimodo is tied to their presence and transformation in the environments they travel through. This aspect of the sounds interaction within environment makes the piece focused on sonic qualities that move beyond the details of the sounds themselves, making clear that sound is never independent of the environment in which it occurs.

Physically experiencing sonic processes: towards a contextual understanding of remote environments

Lucier’s translation of whale sound into sonic events that we can experience with our terrestrial bodies enables us to approach a profound, visceral understanding of underwater sound. Quasimodo draws the participant into the process of sound transmission over long distances through direct physical engagement with actual environments, paralleling that of whales underwater. Only by enacting a performance of Quasimodo myself in 2009 was I able to come to this realization. By accentuating the acoustic properties of the physical spaces, the work heightened my attention to their specific characteristics and their differences. At the same time, the directional long-distance transmission of the sound through these proximate spaces enhanced an idea of continuity and forward motion, of passing thresholds, of accumulation, resonance and a relational consideration of the sounds.15 Such participation demands an imaginative leap away from scientists’ musical ideas such as song or rhythm. By situating the participant inside a sonic environment that is being activated in an unusual way, this approach has the potential to provide unexpected insights into underwater sound. In this way Lucier requires the listener to participate, to become involved on a conceptual level through listening and experiencing on a physiological level.

My own works such as Pink Noise (The Pink Noise of Pleasure Yachts in Turquoise Sea) (2010), Fishing for Sound (2010) and Swim (2010/11), build on this approach to perceptual involvement. Pink Noise employs underwater sounds I recorded using a hydrophone at a national marine reserve in Spain in midsummer. Looking down through the surface of the water, through layers of reflected light, one could imagine that the underwater world is silent. Lowering the hydrophone beneath the idyllic surface of the water I was surprised by the intensity and variety of technological sounds – loud thumps, grinds and high-pitched tones from boat engines, anchors and depth sounders. The contrast between the view above and the sound below, and the knowledge of the significant impact of anthropogenic sound on marine life, was startling. The process of making these sound recordings highlighted the necessity of technology to make this otherwise inaudible environment audible. Using the hydrophone, sitting on the deck of the boat I could see above the surface but hear through headphones the sounds beneath. I was struck by my extension of the sense of hearing underwater and the new meaning it gave to the environment I was inhabiting.

It was these contrasting qualities of the view above water and sound below that I incorporated into the physical installation. I juxtaposed hydrophone recordings with a video projection of light through layers of water from the same location. To emphasize the technologised relationship between the experience of underwater sound and the surface image, the video is projected onto the floor with headphones hanging from the ceiling. One has to physically move into the piece, stand literally in the image of the water and bend down towards its rippling surface to reach for the headphones, thus making an embodied connection between sound and image. The headphones encourage a sense of intimacy, privacy and focused immersion. The isolation and split between the media of sound and image are brought back together by ones body and imagination, drawing the audience into a physical relationship with the audio and visual components.

Video 2: Pink Noise by Yolande Harris (2010)

The sound and video performance Fishing for Sound weaves together a complex soundscape from my own underwater sound recordings, sounds used in psychological treatment for post-traumatic stress disorder (i.e., EMDR clicks), and sonified data from satellites orbiting the Earth (drawn from my Sun Run Sun: Satellite Sounders, 2008). These sounds combine with a hypnotic video (made from a first-person perspective while I navigated a sloop by sextant, exhibited in installation form as Navigating by Circles or Sextant 2008). The underwater sounds in Fishing for Sound include insect, fish, dolphin and man-made sounds of engines, depth finders and anchors collected by simple underwater microphones (hydrophones). In contrast, the electronic sounds come from satellite navigation data which I sonified, turning it into sound to make it audible to us. These are tied together by a clicking sound moving from left to right once per second, like the EMDR technique used in psychotherapy for curing traumatic memories. Although it is unknown exactly how this technique works, one explanation suggests that the alternating clicks act as a sonic guide through subconscious memories – a way of navigating through the mind. Similarly, looking through the viewfinder of a sextant, or listening via a hydrophone to an active anthropogenic soundscape beneath the apparently idyllic surface of turquoise sea, brings into consciousness elements of the environment we otherwise would not see or hear. All these different elements explore sound in the different contexts of the environment, memory and information, and all share in a common mass of background noise. Listening in these dense sonic spaces is like fishing for sounds.

Video 3: Fishing for Sound by Yolande Harris (2010)

The installation Swim uses single channel video and stereo sound to explore a similar effect (see Figure 1: Swim). In this piece, however, the point of view explores a parallel between humans and cetaceans, who live in water but must surface to breathe air. Recorded from a swimmer’s perspective out at sea, it captures their rhythm of breathing and the physical motion of the strokes. Swim presents this intense first person perspective of the swimmer, caught on the surface between air and water, between dreaming and wakefulness. Placing the viewer at the interstice of sea and air, the sound and image alternate between above and below water, cutting through the surface to explore the physicality of sound through a direct involvement with environment.

These works are sympathetic to a sense of physiological involvement in the context of underwater sound. Through an artistic and conceptual combination of media, they draw the audience into sound worlds that are unfamiliar. Rather than engaging the audience in direct physical interaction with underwater environments, the works investigate a high level of commitment to listening using a first-person perspective and multi-sensory video and sound. This approach can, I believe, move us closer to redefining the role of composers and sonic ecologists as activators of a sustainable attitude towards the sonic environment, one that is less passive and less anthropocentric than the genre of field recording, and rather more immersed in and actively committed to experiencing land and sea from a non-human point of view.

1. This research is derived from my doctoral dissertation at Leiden University and the Orpheus Institute Gent. [↩]
2. This insight is shared by disciplines ranging from ethnography to quantum physics. [↩]
3. Visualisation has become the major technique in analysis of complex data sets, but the development of audification and sonification techniques that use sound to describe sound or data, rather than sight and the visual, is comparatively new and undefined. I identify and distinguish between two overlapping approaches to making the inaudible audible: audification by scaling existing vibratory signals into human hearing range; and sonification by translating and mapping a choice of sounds onto data. Audification uses the existing signal as its basis, while sonification requires compositional strategies of mapping data (non-vibratory information) onto sounds. Confusingly, often examples of audification and sonification are used interchangably, but they are distinctly different treatments of sound, and the relationship to the original media differs in that audification remains considerably closer than sonification. The interaction of visualisation with audification and even sonification can be very powerful for understanding inaudible sound. [↩]
4. The installation Pink Noise has been exhibited in the Transmediale Festival, Berlin (2010), Issue Project Room, New York (2011) and WRO Media Art Biennale, Wroclaw Poland (2011). [↩]
5. The performance Fishing for Sound was commissioned for Sonic Acts Festival XIII, Amsterdam (2010) and has been subsequently performed at the Ear to the Earth Festival, New York (2010), Shedhalle, Zurich (2011) and Issue Project Room, New York (2011). [↩]
6. The video Swim was presented at the Ear to the Earth Festival, New York (2010), Shedhalle, Zurich (2011). [↩]
7. Interestingly, Schafer mentions the humpback whale song, although he does not include a specific reference to the scientific source despite reproducing the recognisable analysis and visualisation by Payne and McVay (Schafer 1977: 38). [↩]
8. In 2002, the International Marine Mammal Project of the Earth Island Institute commissioned a comprehensive report on the state of known knowledge of biological underwater sound by Stocker. This was republished by The Soundscape Journal of Acoustic Ecology. [↩]
9. André’s recent groundbreaking work on the affect of anthropogenic sound on cephalopods raises the following questions: “Is noise pollution capable of impacting the entire web of ocean life? What other effects is noise having on marine life, beyond damage to auditory reception systems? And just how widespread and invasive is sound pollution in the marine environment?” (André et al, 2011). [↩]
10. For example the sci-fi film Altered States (1980) very loosely based on dolphin researcher John Lilly. [↩]
11. Mary Catherine Bateson on her father Gregory Bateson. [↩]
12. Subsequent research has shown how humpbacks evolve new songs over seasons, years and large distances, although it is not known why these transformations happen. [↩]
13. André remarked “We knew there were four whales because we took notes during the recording, but all we heard was a confusion of clicks. I asked Arona how he could tell there were four different animals. He said, ‘I don’t know how, but I know.’” (Rothenberg, 2008 ;181 [↩]
14. After Payne and McVay finding music in an interpretation based on Western musical notation, it is perhaps another instance of a specific genre of music, in this case West African drumming, being used to ‘find’ similar social and spatial correspondences in whale sounds, although André and Kamminga apply it in an interesting way by exploring a musicians expert listening skills to corroborate an intuition later backed up by scientific techniques of analysis. [↩]
15. An internet version of Quasimodo such as realised by Laura Cameron and Matt Rogalsky (2009) stands in contrast to the embodied experience of physically proximate spaces. I argue that the accumulation of the acoustics of globally distant spaces has a different effect on our bodily experience of space than those linked together in physically adjacent spaces within a locale. In such an interpretation of Quasimodo one can only experience representations of distant spaces rather than the spaces themselves. [↩]

References

André, M. and Kamminga, C. 2000. Rhythmic dimension in the echolocation click trains of sperm whales: a possible function of identification and communication. Journal of the Marine Biological Association of the UK, 80: 163-169.

André, M., 2010. Sons de Mar. Available from http://www.sonsdemar.eu/sonsdemar.php

André, M. et al. 2011. Low Frequency Sounds Induce Acoustic Trauma in Cephalopods. Frontiers in Ecology and the Environment: The Ecological Society of America, doi:10.1890/100124

Bateson, G. 1972. Problems in Cetacean and other Mammalian Communication. Steps to an Ecology of Mind New York; Ballantine Books, 364-78.

Bateson, G. 1979. Mind and Nature: A Necessary Unity. London: Hampton Books.

Cameron, L. and Rogalsky, M. 2009. Internet performance of Quasimodo the Great Lover http://mrogalsky.web.wesleyan.edu/transnational/doc.html

Dunn, D. 2008. Public presentation, Netherlands Media Art Institute., Amsterdam. http://nimk.nl/eng/search/navigating-the-space-of-the-future

Harris, Y. 2010. Making the Inaudible Audible: Strategies and Disagreements. Proceedings of the 16th International Symposium for Electronic Arts, ISEA2010 Ruhr. Berlin: Revolver Publishing, 133-35.

Hoare, P. 2008. Leviathan, or, The Whale. London: Fourth Estate.

LaBelle, B. 2007. Background Noise, Perspectives on Sound Art. London: Continuum Press.

Lilly, J. C. 1962. Man and Dolphin. London: Victor Gollancz.

Lucier, A. 1995, Reflections / Reflexionen. Cologne: MuzikTexte.

Miller, S. 2007. A Conversation with David Dunn: Sound, Science, Music, Evolution and Environment. Soundscape Journal for Acoustic Ecology, 7(1).

Payne, R. and McVay, S. 1971. Songs of Humpback Whales. Science, 173(3997): 585-97.

Payne, R. and McVay, S. 1970 Songs of the Humpback Whale (recordings re-released by National Geographic1979)Rothenberg, D. 2008 a. Thousand Mile Song: Whale Music in a Sea of Sound. New York: Basic Books.

Rothenberg, D. 2008 b. Whale Music: Anatomy of an Interspecies Duet. Leonardo Music Journal MIT Press, 18.

Slabbekoorn, H. et al. 2010. A noisy spring: the impact of globally rising underwater sound levels on fish.Trends in Ecology & Evolution 25(7): 419-27.

Stocker, M. 2002. Ocean Bio-Acoustics and Noise Pollution: Fish, Molluscs and other Sea Animals: Use of Sound, and the Impact of Anthropogenic Noise in the Marine Acoustic Environment. Soundscape Journal of Acoustic Ecology 3(2):16-29.

Bio

Yolande Harris is a composer and artist engaged with sound and image in environment and architectural space. Her most recent artistic research projects Sun Run Sun: On Sonic Navigations (2008-2009), and Scorescapes (2009-2011) explore sound, its image and its role in relating humans and their technologies to the environment. These works consider techniques of navigation, sound worlds outside the human hearing range, underwater bioacoustics and the sonification of data. They take the form of audio-visual installations and performances, instruments, walks, performative lectures and writings. Yolande holds a Ph.D (Leiden University, 2011); was Sound Art Fellow (Academy of Media Arts Cologne 2006); Artistic Researcher (Jan van Eyck Academie 2003-5); has an M.Phil. (University of Cambridge, 2000); and a B.A. in Music (Dartington College of Arts,1997). Her installations, performances and lectures are presented internationally in the context of visual art exhibitions, music venues, media art festivals and fellowships, including: MACBA Barcelona, Schirn Kunsthalle Frankfurt, Hayward Gallery Touring UK, Netherlands Media Art Institute, Issue Project Room NYC, Shedhalle Zurich, House of World Cultures Berlin, STEIM and Sonic Acts Festival Amsterdam.

http://www.yolandeharris.net