• Project Home
• Staff
• Publications
• Links
• Software and Documentation
• Results

Categories of results generated during M⁴S

The results of the M⁴S project fall into five major categories:

1. Development of features for the automatic analysis of melodies and music in symbolic formats.
2. Psychological tests concerning the cognitive validity and adequacy of analytic features and similarity algorithms.
3. Development of the technical and conceptual tools to derive melodic information for a large collection (i.e. a corpus) of melodies.
4. The usage of melody features as predictors in models of human memory for melodies and other application models.
5. Development of general methodologies for music (cognition) research

1. Feature development

The features developed and used in M⁴S are implemented in the open source program FANTASTIC and are documented in the corresponding software documentation (see section Software and Documentation). The features implemented in FANTASTIC comprise features of different types:

• Phrase summary features that summarise the content of a melodic phrase, conceptually similar to the features that Steinbeck (1982) proposed but incorporating as well recent knowledge and techniques on tonality induction and melodic contour.
• Repetition features that measure the repetitiveness of short melodic-rhythmic sequences or motives (called m-types) which are the building blocks of melodies. These features are mainly inspired by characteristic text constants from computational linguistics that describe the frequency distribution of words in written text (see Baayen, 2001).
• Features based on the frequencies and frequency densities of phrase summary features in a corpus of pop music. These features build on the assumption that the frequency with which melodic features appear in real music are very important for music cognition (Huron, 2006).
• Features that are based on the corpus-derived frequencies of m-types as building blocks of melodies. These features are motivated by the important role that word frequencies play in text information retrieval and text categorisation (Landau et al., 2007) and verbal memory (Tse and Altarriba, 2007).

2. Feature testing

Over the course of M⁴S we have conducted several studies that tested the cognitive adequacy of algorithmically derived melodic features in various psychological experiments. The results of these studies have been reported at several conferences and journal papers are currently in the publishing process. Among the features that have been tested intensively are:

• Segmentation of long melodies into shorter melodic phrases (ICMPC08 presentation on segmentation; Pearce et al., 2008)
• Different models of melodic contour (ICMPC08 presentation on polynomial contour; ESCOM09 presentation on melodic contour)
• Perception of melodic accents (ICMPC06 presentation on accents; Müllensiefen et al., in press)
• Harmonic content and chord labeling (Rhodes et al., 2007; Müllensiefen et al., 2007)

3. Developing tools for using music corpora

One of the original ideas of M⁴S is the approach to incorporate human background knowledge about music into models of music cognition. As an approximation to human knowledge about western popular music we use a large corpus of commercial popular songs and describe it in terms of the statistical distributions and regularities of melodic and generally musical features in it. We then use this statistical information about western pop music in general to assist modelling cognition in specific tasks and individual melodies. Beyond cognitive modelling the statistical description of a corpus of music can also be interesting in persepctive of stylistic categorisation, music information retrieval, analytic musicology. The three steps for dealing information with a music corpus that we took in M⁴S are:

1. We acquired a large collection of very accurate MIDI-transcriptions of 14,063 pop songs ranging from the 1950s to 2006 and encompassing most pop music styles from Geerdes MIDI Music and curated it to become a coherent corpus of pop music
2. We devised AMuSE (in collaboration with the Andrew W. Mellon funded MeTAMuSE project), a database system for musical knowledge representation and reasoning.
3. We developed concepts and methods to apply corpus-based music knowledge to cognitive modelling and musicology, drawing parallels and differences to approaches in corpus-based linguistics (Müllensiefen et al., 2008).

4. Using melodic features and corpus knowledge in models of music cognition and other applications

There are several areas where we successfully applied and tested the tools and knowledge gained during M⁴S:

• Models of melodic memory:
  • Having investigated explicit and implicit memory under several different conditions (Halpern and Müllensiefen, 2008) we applied melodic feature analysis to the melodies in a recognition memory experiment to answer the question which structural features make melodies easy or hard to remember (ICMPC08 presentation on melodic memory). Going one step further than the traditional analysis of accurate memory performance, we also asked which melodic features cause wrong memories, that is are responsible for the illusion that a melody has been heard before (False Alarms) or has not been heard before (Misses) when the opposite is actually true. The results are currently being written up in a journal paper.
  • We re-analysed data from a highly influential memory study (Sloboda and Parker, 1985) using several melodic similarity and analysis algorithms and discussed the results from the older study in the light of the new computational approach (IRCAM-Symposium09 presentation; Müllensiefen and Wiggins, in prep.).
  • We are currently analysing data from a study using the recall paradigm from Sloboda & Parker (1985) employing the features implemented in FANTASTIC and the similarity algorithms from SIMILE. The hypothesis of this study is that the same melodic features are responsible for recall memory as for recognition memory of melodies (see above).
• Predicting court decisions in cases of music plagiarism: Using algorithms from SIMILE and corpus-derived frequencies of m-types (see above) as features in in Tversky`s similarity model (Tversky, 1977) we were able to predict 90% of the US court decisions in a sample of cases regarding melodic plagiarism (Müllensiefen and Pendzich, in press; ESCOM09 presentation on plagiarism). Since this result was obtained using a small sample and did not yet make full use of features we implemented in FANTASTIC, we consider this experiment a pilot study for which a follow-up with a larger sample is already planned.


• Item difficulty in the Montreal Battery of Evaluation of Amusia (MBEA): In collaboration with Lauren Stewart and Andrew Cooper from Goldsmiths` Psychology Department, we are currently analysing the performance for different melody items in a subtest of the MBEA (Peretz, et al., 2003). The aim of this analysis is to show a relation between the difficulty of the item in the test as measured by an Item Response Model and its musical structure as measured by a melodic feature analysis with FANTASTIC.


• Features of hit songs: Rather as a proof of concept than an in-depth investigation and in collaboration with Reinhard Kopiez from the Musikhochschule Hannover we analysed the melodies from all 14 songs from the Beatles album Revolver (1965) using features from FANTASTIC. With these features we predicted the popularity of the 14 songs as measured by the fact whether any cover version of the song had entered the British, German or US-charts. Just using two features (pitch range and pitch entropy), we were able to "predict", or rather partition, the 14 songs with 100% accuracy into popular and less popular songs. We hope to continue this line of investigation in the future

5. General methodologies for music cognition research

Apart from results that are in direct connection with the targets of the project we also developed concepts and methodologies that are relevant for a wider circle of researchers in music cognition and musicology.

• DagMuCorR: A description scheme for music corpora: Together with other participants of the Dagstuhl seminar on Representation of Musical Knowledge we developed Dagstuhl Description Schema of Music Corpora for Research (DagMuCorR). This description schema should enable music researchers who maintain a corpus of music to describe their corpus in a standardised way and publish this information very easily and very quickly on the Internet. This in turn should inform other interested researchers about the content and availability of a music corpus that might be interesting for their own research.
• Dealing with ambiguity in music cognition: In several of the studies carried our during M⁴S we found that different experimental participants would give different reactions when listening to the same music stimulus. While this ambiguity of music is appreciated as a very important aesthetic quality of music it also constitutes a challenge for empirical research that is rarely dealt with explicitly (quite often differences between participants are buried in some form of average of the dependent variable). We found different solutions to this problem for e.g. melodic segmentation (ICMPC08 presentation on melodic segmentation), chord labeling (ISMIR07 paper on harmonic labeling), and melodic accent perception (Müllensiefen, Pfleiderer, and Frieler, in press). We also addressed the general problem in several talks (e.g. Music and Brain talk, 2008).
• New statistical methods in music psychology: We surveyed, discussed, and evaluated on several occasions (e.g. Weihs et al., 2008; Müllensiefen, in press; IPEM09 seminar) recent developments regarding statistical methods used in the music cognition community, such Functional Data Analysis, Tree Models, Bayesian Methods, and Sequence Learning. We highlighted the problems and challenges that these new methods set out to tackle as well type of answers researchers can expect from these techniques.
• AMuSE (Advanced Musical Score Encoding): Devised and developed in close collaboration with the MeTAMuSE project, the AMuSE system allows tools on all levels to work together on very different types of musical scores, additionally providing score-based, graphical or midi presentations of research results. Many of the tools developed for M⁴S are built with or rely on AMuSE, and the system has been taken up by several other projects withing the group.