Joint center analysis, a technique commonly used for posture and motion analysis in human modeling and biomechanics, turns out to be quite revealing when used on pianists in order to determine if there is a significant difference in movement when pianists are asked to play in one of two ways: focusing on correctness or on enjoyment. These precise techniques of tracking and measuring allow for classification, identification and comparison of movement patterns with regard to shape and location, thus providing "objective correlates" against which to test our subjective impressions and judgments.

Part of the process employed in joint center analysis involves capturing the motions of performing musicians by infrared camera tracking of sensors placed on relevant parts of their bodies. When these dots subsequently are connected on the resulting video, a byproduct of this analysis of a performing musician emerges:  a 3D video rendering showing an animated point light “stick figure” which may be observed by simple inspection in its own right. This animated avatar is a concrete manifestation of the abstracted formal motion of the musician. This is a truly new tool in the history of methods available to us for empirical music research, allowing for the exhibition of qualities of the performing musician that previously only could be abstracted in our imagination.

fMRI technology permits us to observe the brain activity of someone watching and listening to such an avatar. In the experiment that provides the basis for this presentation, four pianists each performed the same two short pieces while fitted with sensors in a motion capture laboratory.  Unaware of each other’s experimental experience, each was first asked to perform each piece as correctly as he or she could, and, the second time around, simply to enjoy playing the piece. Eight (other) subjects were then placed in an fMRI scanner  –  four trained musicians and four non-musicians who were identified as appreciators of classical music. Each of these eight fMRI subjects watched and listened to performances by the avatars (the performances were varied across fMRI subjects). The subjects each had to answer the same battery of questions about each performance witnessed while in the fMRI machine - questions posed so as to be answerable on a 7-point Likert scale.

One result was that the enjoyment mode was more attention-getting for the audience members  than  the correct mode, though primarily for musicians -- less so for non-musicians. Interestingly, this attention grabbing may take the form of “‘empathic” emotional connectedness with the musicians as the enjoyment mode tends to produce a more emotional (right hemisphere) response in them (as seen in the Parahippocampal Gyrus) , whereas the correct mode tends to activate frontal regions in the left hemisphere  -- regions that might be associated with a technical "evaluation" of the performance as perceived by musicians, but not by non-musicians.  

Further analysis of the data suggests that evidence is provided that is consistent with the activation of mirror neurons in the musicians, but not in the non-musicians. Musicians activate this region in both the correct and the enjoyment mode, but more so in the enjoyment mode. The activation of this region is pretty much non-existent in the non-musicians who do not know how to play the piano, but presumably could move their fingers and wave their arms in a similar but meaningless way, so there is probably some minor activation of this region in them as well .

This presentation will thus

(1)    Briefly outline the details of the experimental setup.

(2)    Show what motion capture analysis alone reveals about the movement characteristics of pianists when playing in varied intentional states.

(3)    Present the implications that the coupling of motion capture with fMRI analysis suggests for further research on the relationship between the quality of engagement a musician manifests in performance and the reception of the performance by audience members.

(4)    Discuss some of the new perspectives this experiment and ones like it can cast on the role of form in philosophical aesthetics of music