Joshua Banks Mailman

Joshua Banks Mailman is an American music theorist, as well an analyst, composer, improvisor, philosopher, critic, and technologist of music.

Joshua Banks Mailman was born in New York City and attended Fiorello H. LaGuardia High School of the Arts. He gained a bachelor's degree in Philosophy from the University of Chicago and a Ph.D. in Music Theory from the Eastman School of Music of University of Rochester. Among his teachers were Charles Rosen, Ted Cohen (philosopher), Richard Cohn, and Robert Morris.^[1]

Mailman has taught predominantly at Columbia University,^[2] as well as at New York University, University of Alabama, and University of California, Santa Barbara. He has lectured at the Japan Society (NYC), the Society for Music Analysis (UK),^[3] IRCAM (Paris),^[4] the Symposium (SIMPOM) of Brazilian Studies in Music (Rio de Janeiro).,^[5] and the Symposium on Computer-assisted Composition, Istituto per la Musica, Cini Foundation (Venice, Italy).^[6]

His writings, published in numerous scholarly journals and books, contribute to several areas of music theory, analysis, and technology, spanning such topics as narrativity, phenomenology, metaphor, form as process and dynamic form, cybernetics, music visualization, and range over repertoires from Arnold Schoenberg, Elliott Carter, Ruth Crawford Seeger, and Milton Babbitt, to Gyorgy Ligeti, Luciano Berio, Alvin Lucier, Robert Ashley, Gerard Grisey, and Kaija Saariaho.

According to Katherine Lee, “Theorist Joshua Mailman has written prodigiously of… dynamic form in Western Music…[arguing] that musical form should be interpreted as dynamic or as the ‘retrospective contour of the flux of intensity of qualities, [challenging] oft-held views of musical form as architectonic or structural.” ^[7] Several writers have noted the indebtedness of their work to this idea.^{[8][9][10][11][12]} Mailman's work on dynamic form in music focuses on “emergent properties in dynamic processes”,^[13] for instance ‘’temporal density’’ (‘’interonset density’’) as a generator of phenomenologically emergent properties.^[14] The analyses and concepts that Mailman presents reveal underlying principles of electroacoustic music operating in music outside the electroacoustic realm, such as in Robert Morris’s String Quartet Arc ^[15]

Mailman has examined

“flux and flow in music by [Elliott] Carter and [Luciano] Berio, [modeling these] on multi-layered graphs showing the curve of various musical features” with the graphs aligned to indicate visually “the degree of coordination of flux among these salient musical elements, [with] a high degree of coordination [being] said to contribute to a more assertive (rather than furtive) projection of form, one that is more easily apprehended in time. Yet Berio’s Points on the Curve to Find was shown [by Mailman] to assertively project form through a completely different unconventional flux that did not depend on such coordination.” ^[16]

Other emergent properties Mailman identifies as generators of dynamic form include ‘’echo rate’’ and ‘’pitch freshness’’.^[17][10]

More generally Mailman’s “broad-reaching meditation on ‘dynamism’ in relation to musical form [... includes...] a working definition of ‘’dynamism theory’’ as an analysis that ‘asserts motion, change, process, or energy (potential motion, change, or process) as existing in the course of a piece or a performance, as it elapses’” ^[18]

Mailman’s extensive writing on cybernetic phenomenology of music engages Speculative Realism, exemplifying that “the experience, and indeed the very constitution of the world does not necessarily have to refer to linguistic mediation.”.^[19] It may reference the real but nevertheless diverge from it, constituting “[m]ixed realities…resulting in new narrative opportunities through artistic practice” ^[20] As music is enacted in time (showing rather than telling), its most important properties are not basic, instantaneous, or immediate; rather “this enactment occurs through ‘emergent properties’ that arise out the combinations of the fundamental elements of music (such as pitch, timbre, and duration) into holistic musical features,” ^[21] Mailman explains how these “emergent properties” can arise through bodily motion (using conventional instruments or ‘’comprovisational’’ technology). “Mailman suggest that, when we conceive of musical properties in terms of physical reference, musical listening can, in turn, present new notions of bodily experience.” Mailman argues that as such properties emerge they can give rise to noticeable processes.^[21]

In his lengthy 2016 article on cybernetic phenomenology (see Second-order cybernetics) Mailman uses the rules of the ancient two-player strategy board game Go (囲碁; いご), to generate brief snippets of music and discusses the ‘’emergent properties’’ that arise only indirectly from these rules, but which can be heard in the snippets. As Tim Summers explains it:

“Under Mailman’s mode of listening, music in its sounding is also a form of world-building—we perceive not only the musical material in front of us, but also that there are sets of parameters, rules, and procedures that give rise to this music. Much like game theorist Jesper Juul’s conception of learning a game’s rules through its fictions, Mailman suggest that the ‘rules’ that underpin the worlds of musical pieces are implied by the sounded material. These rules/processes can be established, subverted, fulfilled, and so on. The music plays out within these frames and processes... Mailman's ‘properties’ within ‘rules’....imply that we are hearing only one musical articulation within broader spaces of possibilities—the music we hear has the potential to be otherwise” ^[21]

Mailman has theorized and demonstrated processes for creating formalized models for various perceived musical effects or emergent properties. “Such processes of ‘listening, phenomenology, and computational modeling’” are what Mailman calls ‘’cybernetic phenomenology’’ ^[22] (See also Second-order cybernetics).

Thus, as described in various sources, Mailman does not just propose these ‘’emergent properties’’ as ineffable, but rather explicates them through analysis (which in some cases enables them to be synthesized or emulated technologically—through custom-designed algorithms).^{[23][10][24][17][25]}

Yet this explication has to be prompted by phenomenology of listening leading into a feedback loop involving some technical analytical procedures or algorithm, whose output is monitored and which is adjusted in response to that monitoring. As Benjamin Hansberry explains it:

“Mailman describes this analytical methodology [cybernetic phenomenology] as involving ‘computational analytic procedures prompted by [one’s] hearing, procedures whose output in turn enhanced [one’s] experience as a listener…Mailman…foregrounds the creative potential of such a methodology as part of the performative turn in music analysis.” ^[26]

As Hansberry describes it, this process has three stages: (1) The analyst translates their phenomenal experience into formal terms. (2) The formal terms are worked out systematically. (3) The results are “re-translated” back into phenomenal experience—except that in Mailman's cybernetic phenomenology these stages occur in a loop, where stage 3 feeds back into another iteration of stage 1. “The notion of this cybernetic phenomenology as an underlying analytical practice is a common theme in Mailman’s work.” ^[27]

Regarding the term "comprovisation", Mailman’s use of this term clearly shows that improvised and composed music are so interwoven with each other down to even their presuppositions, that only the label assigned to them allows one to, arbitrarily, distinguish them.^[28] At the same time Mailman offers a justification for the composite term as described in Arthur Farac’s account of it: Comprovisation is compositional since it “involves the composition of algorithms for generating music guided by an aesthetic concern,” and “the semi-stochastic algorithm can be considered as 'improvisation', since the determination of certain details cannot be predicted in advance” ^[29]

“In an analysis of his own comprovisational computer music practice, Joshua Mailman explains that improvisation has been an activity located between composition and performance, drawing substance from each….comprovisation as carried out in the design of interactive artworks unites planning with performance, predictability with spontaneity, and rewards the artist who engages in their art-making processes with a spirit of experimentation and an embrace of unexpected outcomes with the possibility of finding inspiration in chance events.”^[30]

Mailman’s work on comprovisation, embodiment, and interactive music involves developing and using custom-built technologies, using wireless hardware sensors to control meta-parameters of algorithms, where these meta-parameters correlate to emergent properties^[31] “Mailman …rightly critiques what he sees as a ‘universalizing impetus’ of embodied cognition that insists on a ‘natural’ relationship between musical sounds and their embodiment, suggesting instead an approach that highlights music's artifact and allows for the expansion and extension of how music is embodied.”^[32] Therefore rather than trying to manipulate typical musical properties,

“Mailman advocates listening to and analyzing music in order to discover alternative musical parameters to manipulate via body movements. These alternative elements can then become embodied in a new way through associative learning while using the IMS [Interactive Music System]. Some of the musical features Mailman demonstrates include rhythmic sparseness, viscosity of texture, and harmonic space. He pairs these features with types of body motions, such as pressing the hands closer together or further apart"^[24]

Caitlin Trevor calls this a “more enriching and creative way to build IMS” and “an inverting, boundary-pushing new perspective on the subject.”^[33]

In terms of technology, Mailman has developed the audio component of these systems using ports of Real-time Cmix (RTcmix) for iOS as well as Max/MSP. For instance, in his iOS implementations, features such as attack density, timbre (low-pass filter), or pitch-collection can be manipulated through sliders or tilting the phone.^[31]

At the same time, Mailman's IMSs also involve a visual component, live computer graphics animation, steered by the same body controlled sensors as steer the audio components.^[34] As David Wright remarks: “Visualization of music … serve[s] as the basis for experimental work in live digital visuals…[Mailman’s Montreal Comprovisation No. 1] provides an excellent example.”^[35]

In this work, as explained by Zuhal Oktem Demir (2021), the music and graphics are created by the physical movement of the user, who acts as both a dancer and musician, creating an audio-visual space with graphics shaped according to these physical movements.^{[36] [37]} Thanks to such artistic applications for music and technology of interactive systems, the visual and musical are more blended.^[37] Thus, as Demir, explains the terms by which composition and improvisation are named together, ‘’comprovisation’’, applies equally well to the self-production and execution of the computer graphics, in particular these graphics are systematically coordinated audio-visual while subjected to situated performance of the dancer-musician-usuer.^[37] These are explained, according to Demir, in terms of Mailman’s Fluxations and FluxNOISations human body interfaces, which are interactive dance systems that spontaneously generate music and graphics in response to hand and body movement.^[37] The harmonious complexity and diversity are spontaneously impressive, arising from the system at provides shaping, revealing a multi-layered multi-form experience, says Demir, who continues to describe Mailman’s ‘’improvising synesthesia’’ as follows:

Fluxations and FluxNoisations equipment is operated by a musician-dancer moving in a room in front of the infrared camera wearing wireless sensor gloves. This movement drives the algorithmically generated music and graphics. Audible and visible qualities of the music and graphics depend on the musician-dancer’s movements. The variety of the image that changes according to the artist’s movements has a wide range, Sudden or soft changes of the image vary according to spatial location such as the position of the body or the angle of the wristband. For example the slow movement of the artist brings soft qualitative changes. Sudden movement brings about sudden qualitative changes. Since the qualities of both music and graphics are thus shaped by the artist’s movement, a sudden qualitative change in music is accompanied by a sudden qualitative change in graphics. The opposite is also true; a smooth trajectory of qualitative change in music is accompanied by a smooth trajectory of qualitative change in images and vice versa. In this way the graphics noticeably coordinate with the music, creating an immersive world whose associated audiovisual trajectories are constantly directed and shaped by the movements of the human body moving through space. Prismatic particle systems and color, texture, continuity of harmony maneuver with the planned or spontaneous movements of the artist. The generative algorithms are designed to maximize the variety and consistency of the musical and visual experience, including cross-mode relationships between them and explosions and bubbling bursts of evolving particles.^[38]

Arthur Farac and Manuel Falleiros (2019) describe Mailman’s comprovisation in aspirational terms as a performing art in which interactive technological systems are essential, as they enable what was previously impossible: the design of complex streams of semi-Stochastic events that are generated complexity achieved through technological tools, that is, the use of algorithms, are in this way essential for the characterization of comprovisation. Mailman positions the method as a hybrid of composition and improvisation. From composition, the comprovising new media artist draws a tendency to "compose music-generating algorithms as guided by aesthetic concerns," and the inclusion of a "planned choreography of physical movements." From improvisation, it draws three other aspects: its potential for the creator to use "spontaneously decided physical movements," the manner in which "planned (choreographed) movements may be spontaneously ornamented with expressive nuanced deviations," in addition to the already mentioned unpredictable stochastic-algorithmically generated details which inspire and inflect the spontaneity of body movement.^[39][40] As Arthur Farac (2021) describes Mailman’s comprovisational theory thus: the existence of comprovisation would be impossible without the emergence of new technologies that can mediate interactivity.^[29]

As with his analyses of Elliott Carter’s music, Mailman's approach to Spectral Music reconsiders how to present musical formal procedures as they unfold in time, especially with regard to rhythm and texture.^[41] His analysis of Gerard Grisey’s Vortex Temporum and Kaija Saariaho's Lichtbogen employ his own custom-designed computer graphics animations to analyze texture and temporal processes.^[42]

According to musicologist Michael Gallope, Mailman is among the recent generation of music theorists who have ventured more systematic analyses of Arnold Schoenberg’s free atonal period, through the perspective of transformation theory (music).^[43] But in Mailman’s analyses of Schoenberg, transformation theory plays merely a role within a larger looser framework that focusses on chord-event hierarchies emerging from melodic contour and other motivic or contextual features, whose configurations help to ‘emancipate’ previously unfamiliar chords:

Joshua Mailman’s study, which investigates a selection of Schoenberg’s works, suggests that concepts of prolongation and structural levels are inappropriate for his music. 105 He states that “Making unfamiliar chords comprehensible (‘emancipating’ these chords by cultivating strategic comparisons between them) has really not much to do with generatively driven hierarchies of chords whose tones are ‘conceptually sustained’ over spans of time”.106 Despite the focus on Schoenberg’s use of atonality, his study, which advocates the use of Representational Hierarchy Association (RHA), suggests that we should move towards a more inclusive prolongational hierarchy, a representational hierarchy for models of music.^[44]

In particular, Mailman’s analysis of the fourth piece, ‘’die blasse Wascherin’’, from Schoenberg’s Pierrot Lunaire Op. 21, engages a modernist aesthetic perspective, but by analyzing phrase-like entities and thereby overcoming what Arnold Whittall has called the ‘Martian’-like approach that sometimes typifies pc-set analysis.^[45] This is done by identifying associational (pitch contour or other contextual) features that bring out contrasts or kinships between consecutive or non-consecutive phrase-like segments.^[46] For instance Mailman's analysis of Schoenberg's piano piece, Op. 11 (Drei Klavierstücke), No. 2 singles out an arch contour, recurring through the opening section, focusing on the chords that occur at the registral peak of these arches, he shows how they create a conflict that resolves.^[47] To do this, Mailman examines mutually exclusive aspects of their interval content, uses Klumpenhouwer networks to demonstrate a pattern in relation to these peaks, and analyzes the melodic contour activity. For this melodic activity, Mailman identifies a prominent arch contour in Schoenberg’s Op. 11, No. 2 and divides it into three subsegments either present or absent, thus each a binary state, so that:

“in terms of David Lewin’s Binary State GIS (generalized interval system), [he explains] this binary state GIS as essentially a vector of on/off switches that applied to each motivic cell can either reverse (make opposite, flip, invert) the current state of something (with a digit 1) or leave the state of something as is (with a digit 0)”.^[48]

Accordingly, in Mailman’s use of the states, each item in the binary code is a chronologically-ordered and a Binary State Generalized Interval System is used to track changes of state.^[49]

“Mailman…is of the new generation of musicians who have absorbed the text and sense of Milton's music as a natal environment and think naturally within its terms” says Benjamin Boretz^[50] Mailman is acknowledged for developing a flexible approach,^[51] and for drawing attention to and theorizing liminal periodicity (usually some “lopsided groove” arising from recurrence of two recurring time-points among others, a phenomenon called ‘time-point permeation’),^[52][53] and because of “focusing on ways in which passages of [Babbitt’s] music can both function in a serial context and reference tonality”,^[54][55][56] Mailman is acknowledged for his systematic interpretation of Babbitt’s 12-tone music as a sequence of tonal jazz chord changes,^[57] also known as ‘portmantonality’ (double entendre allusions to jazz and Tin Pan Alley songs),^{[54][58][59][60][61]} as well as, related to this, for demonstrating the analogy of jazz improvisation for Babbitt’s practice of composing note-by-note details somewhat spontaneously based on a pitch chart, somewhat analogous to a jazz lead sheet.^{[62][63][64][65]} In this way, as Zachary Bernstein explains it:

Mailman posits an improvisatory character to Babbitt’s music and his treatment of precompositional materials. Building on George Lewis’s work on improvisation, Mailman imagines Babbitt improvising among the many possibilities his partially ordered aggregates permit. Examples are drawn from Composition for Four Instruments, Semi-Simple Variations, and Whirled Series. Mailman offers a playful corrective to myths that Babbitt’s music is strictly determined.^[56]

Thus both the ‘portmantonality’ and improvisatory characterizations that Mailman offers counteract the common misconception that Babbitt’s way of composition was algorithmic, that once an array had been selected for pitches and rhythms, and once parameters for instrumental and registral projection had been determined, then the rest (the sonic details heard) would follow trivially. “But this [common misconception] could not be further from the case,” as Mead remarks. “Mailman 2019…[demonstrates how] Babbitt’s compositional technique sets limits on the choices he could make in the process of writing but it did not make these choices automatically.”^[66] In these ways Mailman’s work has fueled the newly emerging consensus that the more fruitful way of viewing Babbitt’s serialism is as “creat[ing] a musical space within which he could be creative”^[66] and as an “instance of the American tendency to put together unexpected combinations of influences, just to see what happens.” ^[64]

In his writing on metaphors for (music) listening, "Mailman offers a range of images to illustrate the diversity of listening" ^[67] As Marty explains it, whereas psychology and phenomenology of music typically emphasize a view of listening as deterministic, Mailman, inspired by George Lakoff and Mark Johnson's conceptual metaphor theory, emphasizes that, our experience of music is inescapably mediated by metaphors because metaphors are the basis of all non-basic cognition.^[68] Thus Marty quotes Mailman's assertion that metaphor is the "cognitive mechanism by which all abstract thought (from everyday language to mathematical reasoning) emerges from 'physical experience'" ^[69]

Therefore, as Ellingson elaborates: “Mailman (2012) offers seven metaphors for listening that challenge readers to reconceptualize listening in creative and provocative ways, including digestion, recording, adaptation, meditation, transport, improvisation, and computation”^[70] As Fagan puts it: 'By invoking metaphor, [Mailman] aims to "maximize listening’s experiential value", partly by opening the doors between listening and multiple other sensory experiences: touch, digestion, cognitive improvisation, transport to different spaces and temporalities. At the forefront of Mailman’s analysis are complex links between sonic phenomena, memory and time."^[71] 

Regarding the seven metaphors for (or types of) listening, poet George Quasha remarks that "all of Mailman’s types strike familiar notes in one’s experience" and speculates that "one's listening is typological according to one's personal make-up plus experience and occasion, and further, that there are no pure types except in our conceptualization of them. And in any given instance we may shift the type of listening we do many times."^[72] Thus Quasha connects with the dynamic and plural nature of music listening proposed by Mailman.^[72]

By contrast Fagan has focused on the stimulation of corporeal and narrative imaginings that these metaphors seem to reference:

"During a listening experience, argues Mailman, memory works to organise a flow of sequential sound into paratactic narratives, including stories that make sense of ontological happenings. So I am not hearing a round, repeating tumble of incremental shifts in pitch and melody. Rather, I am hearing the song of a grey shrike-thrush, returning with the spring. In remembering this song I am transported to other springs, other gardens,…” ^[71]

Citing the linguist Michael Reddy, Mailman suggests that listening is typically assumed to be a passive information transfer activity, as opposed to what Reddy distinguishes as a "toolmaker’s paradigm". As Vickery elaborates:

Past explorations of listening metaphors explicitly and implicitly recognize Reddy's (1979) conduit metaphor. For example, Mailman (2012) employed rhetorical, poetic, and cognitive representations of metaphors in his development of seven listening metaphors designed to capture the plurality of listening to music. In the listening as recording metaphor, listeners are viewed as being able to identify the clear labels of chords, melodies, and harmonies with accuracy, fidelity, and minimal effort. Mailman (2012) argues this metaphor assumes the "lossless transfer of information" (p. 3) explicitly associating this metaphor with the conduit metaphor. Mailman (2012) also develops an argument for a listening as computing metaphor (i.e., listening as mathematical, identifying and comparing beats, rests, and rhythms numerically), which is representative of the toolmaker paradigm (Reddy, 1979).^[73]

Listening as Meditation or Listening as Transport are ones that Quasha finds most tempting to map his own concept of 'axiality', which is an unknown but felt center variable with regard to some activity, that he finds, for music is best represented by the ‘routine ontology of sound’ that Mailman attributes to his metaphor Listening as Recording.^[72]

Mailman’s metaphors propose a plurality of styles of listening. In the context of an explaining plurality of musical cognitive processes, Almen recognizes:

One can perhaps listen idiosyncratically, noticing intertextual relationships that the composer—or performer, or cultural tradition, and so on—would not have recognized. [Footnote] Joshua B. Mailman's "Seven Metaphors for (Music) Listening: DRAMaTIC" (2012) is useful here in two respects. First—more generally —it advocates precisely the kind of plurality of frameworks for encountering music that I am advocating, from the perspective of metaphor. Second—more specifically—his notion of [listening] as improvisation, 'the unscripted aspect of listening but moreover its anti-scripted potential determinable by the context as well as by the listener’s volition," is akin to the present discussion [of pluralism of cognitive processes]: that of the centrifugal attentiveness that engenders pattern discovery.^[74]

Mailman has also, within his "computational-phenomenological theory of dynamic form", proposed a dialectic opposition that recognizes how "metaphors for time…tend to be classable as one of two types: where time moves over us (temporal) or where we move through time (spatial). The synthesis offered concerned the notion of vessel, which encompasses both the temporal-metaphorical approach to time (think of a blood vessel) and the spatial-metaphorical approach to time (think of a seafaring vessel).”^[75]

Notable performances include his audio-visual electro acoustic improvisational trio Material Soundscapes Collide (with Arthur Kampela and Rhonda Taylor) at the New York Philharmonic Biennial in 2016, John Cage's Ryoanji at the Miller Theatre, NYC, in 2015, and the solo audio-visual work/performance Montreal Comprovisation No. 1 at Improvisation, Community, and Social Practice (ICASP) at McGill University in 2012. His audio and audio visual works are presented in Enough Records’ 100 Years of Noise, SoundsRite, and The Open Space Web Magazine. Mailman's music has also been performed by Thomas Piercy, Taka Kigawa, and Carson Cooman.

Mailman has appeared in Robert Hilferty’s documentary film Babbitt: Portrait of a Serial Composer, in Ryan Camarada’s documentary film Royalty Free: The Music of Kevin MacLeod, and on TV in the ABC’s News Nightline segment “Why Some Songs Make us Sad” where he likened the bluesy, minor-key pitches in Pharrell Williams’s song "Happy" to the appeal of salted caramel.

Joshua Banks Mailman is the son of immigration lawyer and writer Stanley Mailman,^[76] second cousin of feminist painter Cynthia Mailman and of film producer Fran Rubel Kuzui, and second cousin-in-law of Nobel prize winning nuclear physicist Melvin Schwartz.

• "An Imagined Drama of Competitive Opposition in Carter's Scrivo in Vento (with Notes on Narrative, Symmetry, Quantitative Flux, and Heraclitus)" Music Analysis (2009)
• "Seven Metaphors for (Music) Listening: DRAMaTIC" Journal of Sonic Studies (2012)
• "Improvising Synesthesia: Comprovisation of Generative Graphics and Music" Leonardo Electronic Almanac (2013)
• "Schoenberg's Chordal Experimentalism Revealed Through Representational Hierarchy Association (RHA), Contour Motives, and Binary State Switching" Music Theory Spectrum (2015)
• "Cybernetic Phenomenology of Music, Embodied Speculative Realism, and Aesthetics Driven Techné for Spontaneous Audio-visual Expression" Perspectives of New Music (2016)
• "Pragmatist Ironist Analysis and Embodied Interactivity: Experimental Approaches to Sensor-Based Interactive Music Systems inspired by Music Analysis" in Music, Analysis, and the Body: Experiments, Explorations and Embodiments (2018)
• "Portmantonality and Babbitt's Poetics of Double Entendre" Music Theory Online (2020)

• Joshua Banks Mailman’s page on Leonardo
• The ABC News Nightline segment “Why Some Songs Make us Sad”
• IRCAM lecture and visualization for Grisey’s ‘’Vortex Temporum’’