Projects

In the 2016-17 academic year, I am interested in supervising projects in the general areas of Music Information Retrieval, Colour and Sound Perception, Computer Language Implementation and Automated Music Processing; some project topics are suggested below:

hip-hop Twitter bot

The project aims to create an agent to participate in discussions on Twitter related to music. Specifically, can we identify participants who might be interested in some kind of music, and can we craft responses that are in any way plausible? The hip-hop community maintains an extensive online database of lyrics and analyses; the project would mine this database for textual patterns, and use it both to recognize discussions of hip-hop on Twitter and to generate responses.

Deliverables might include some of:

• implementation and running of a scraper for Genius;
• a twitter bot which aims to classify tweets into those which are relevant to hip-hop in some way and those which are not;
• an extension to the twitter bot to have it reply to tweets classifed as related to hip-hop;
• experimental validation of the accuracy of the bot, by comparison with human judgment.

perceptually-motivated colour space library

This project aims to implement a set of library functions implementing perceptually-motivated colour space handling. It will: provide conversion functions between various colour spaces; be able to compute the perceptually closest representable colour in some colour space to a given input colour; functions to generate colour palletes with smoothly varying perceptual features; and may integrate with existing colour-handling peripherals.

Project deliverables might include:

• implementation of a range of colour space functionality, including at least colour space conversions, appropriate distance computations, and colour palette generation;
• a range of test cases, demonstrating the empirical correctness of the implementation;
• API-level documentation of the library functionality;
• User-level documentation, including an example application as appropriate to the language used for development of the library.

hypermedia portal for renaissance polyphony session tapes

This project is to produce an interactive hypermedia portal to help navigate, interrogate and visualise a dataset of four days of audio recordings (session tapes of renaissance polyphony). Deliverables might include:

• a website capable of presenting appropriately individual elements of audio, image and textual data;
• suitable means for navigating between data and comparing multiple instances of related data, such as manuscript images, different channels of the same take, different takes of the same musical material;
• commenting facilities, to allow users to record personalised textual associations between media items
• user studies and web analytics, to measure user engagement

mobile phone app for music fragment recognition

This project aims to build a mobile phone application to perform efficient retrieval of musical fragments from a database, in order to provide information to Opera-goers of motifs in the music they are listening to. The student would work on processing input audio from the device's microphone, and matching it to a set of canonical examples held as part of the programme. Deliverables might include:

• a library to perform real-time feature extraction on audio, runnable both on a desktop and a mobile device;
• a preprocessed database of audio samples prepared for feature matching;
• an implementation of linear search through the database of audio samples to find close matches to recent audio input;
• (advanced) a more sophisticated search technique to speed up the search;

optical music recognition on non-traditional music notation

Optical music recognition (OMR) can be made to work acceptably on examples of standard music notation. However, music comes in a wide variety of forms, and some repertoires of music are less well-served by modern tools. The student would work with existing OMR toolsets to develop and evaluate optical music recognition methods for music notations such as tablature, mensural or neumes.

Deliverables might include some or all of:

• an implementation of OMR for a particular collection, based on an existing OMR framework such as Gamera;
• an evaluation of the accuracy and robustness of OMR on a sizeable collection of music;
• a website to enable crowd-sourcing of corrections to outputs of OMR tools.

Background reading:

• C. Dalitz and T. Karsten, Using the Gamera framework for building a lute tablature recognition system
• T. Crawford and L. Pugin, Evaluating OMR on the Early Music Online collection

Python and C++ experience recommended

music use and propagation on social media and the web

The project is to study user behaviour on music-related Social Networks, and to see if there's any useful information. As an example, consider two sites: Musicbrainz, a site where interested parties input, edit and collate metadata about audio recordings, and RapGenius, where users input, analyse and discuss the lyrics of hiphop and rock songs. Can links be made between activities on these sites? Do they share a userbase? Do the communities (on the user or the media level) have any structure in common?

Deliverables might include:

• an implementation of an automatic or semi-automatic alignment (linking) method between the same works and/or users on the two social networks;
• an implementation of a mechanism for retrieving relevant data from social networks
• research on the graph and community structure in two social networks, identified through user behaviour or clusters of works;
• comparison between the structures present in the two sites, and evaluation of the structures in terms of the musical activity they represent.

Background reading:

• K. Page et al., Semantics for music analysis through linked data: how country is my country?

compilation of pD to a low-level language

The project aims to convert pD programs to a lower-level language, in the hope of making them more computationally efficient and/or more reusable. The aim would be to write a compiler for (a subset of) the graphical and reactive language pD, to produce a program with the same effect but written in a language of your choice (such as C++, Java or LLVM IR). Deliverables might include:

• a parser for the pD patch textual format;
• a simple interpreter for a subset of the pD patch format;
• a compiler for that subset;
• experimental measurements for any gains in efficiency from the compilation process.

sonnet rhyme scheme detection from spoken poetry

The idea of this project is to implement a system which, to the extent possible, determines the rhyme scheme used in a sonnet given a spoken word recording of it. The sonnet might in principle be in English, Italian or Urdu (or indeed other languages).

Deliverables might include some of:

• implementation of a system to perform speech recognition, at least to phoneme level, from recordings of spoken poetry;
• implementation of a system to segment the phoneme stream into lines, based on heuristics and given that the phonemes are known to be in sonnet form;
• implementation of an algorithm to determine the most likely rhyme scheme given a phoneme stream representing a sonnet;
• a web interface to visualize of the results of the analysis steps;
• evaluation of the implementation on a corpus of recorded sonnets.

a bot for a roguelike game

Roguelike games have simple interaction and very complex mechanics; it is very hard to write an implementation of good, let alone optimal play. This project is about exploring the space of artificial intelligence techniques that can be applied to roguelikes, in particular to Dungeon Crawl: Stone Soup, which has a lua scripting interface available to players. Deliverables might include some of:

• an assessment of the available lua scripting hooks, and review of some previous attempts at writing bots for DC:SS;
• an implementation of a simple rule-based bot capable of exploring dungeon levels;
• integration of more rules to allow the bot to win combats under some circumstances;
• using more advanced AI or Machine Learning techniques (e.g. expert systems, neural networks, genetic algorithms) to learn and implement better behaviour

interactive fiction and real space

This project aims to augment an engine for running parser-based interactive fiction with ways of interacting with the engine based on geolocation or other aspects of real space. The engine will need extending to be able to accept input of other kind than text; this can be done along the lines of Andrew Plotkin's modifications for the commercial game Hadean Lands. Deliverables might include some or all of:

• modifying the Quixe or Glulxe source code to include a custom Glk event;
• mapping that custom event to some text that a piece of interactive fiction accepts as input;
• triggering that custom event when some condition is met (e.g. when the user is close enough to a given location, or when a certain QR code is scanned);
• a game using the real-space functionality as part of the game mechanics.

randomized testing of lisp compilers and interpreters

Lisp is a programming language with the unusual property that programs written in it are represented as data structures that other Lisp programs can manipulate; this gives programmers the ability to write programs that interrogate or transform other programs. This project would develop systems for automatically generating Lisp programs, and verifying that their evaluation by contemporary Lisp systems is correct, where this is provable, or at least consistent with different implementation strategies.

Deliverables:

• a system for automatically generating conforming side-effect-free Lisp programs;
• an implementation of a differential tester of Lisp programs;
• a set of defect and bug reports generated by running the differential tester on currently-available open source Lisp systems.

Background reading:

• P. F. Dietz, The GCL ANSI Common Lisp Test Suite
• W. M. McKeeman, Differential Testing for Software