IS52020B: Perception and Multimedia Computing

Teaching materials

Slides, lab sheets, and other lecture material will be made available at the module website after the corresponding lecture.

Coursework

There will be two assignments, each worth 15% of the total credit for this module.

Syllabus

Visual perception: cones, rods and the eye; optical illusions; colour vision; colour spaces and profiles; motion perception and Gestalt psychology.
Animation: approaches to animation; perception in video and film; making animations; visualisation.
Sound, hearing and music: sound and the ear; frequency, pitch and harmony; melody; rhythm; digital audio formats and compression.
Signals: the nature of signals; special signals; audio signals and sampling; frequency, amplitude and phase; the Fourier representation.
Systems: linearity and time-invariance; impulse responses and convolution; spectral analysis; convolution by spectrum multiplication.
Audio and image filtering: EQ; filter design; subtractive synthesis; echo and reverberation; resampling; image representation; two-dimensional convolution and image effects.
Multimedia information retrieval: retrieval, fingerprinting and similarity; features and distance measures; systems for multimedia information retrieval.

Reading List

Essential Reading

The essential material in the course is based around two subject guides, shared with the University of London External Programme. Final drafts of the two subject guides can be downloaded here:

M. Casey with S. Rauchas, Creative Computing II: interactive multimedia (vol. 1: Creative signals and systems);
C.S. Rhodes with S. Rauchas, Creative Computing II: interactive multimedia (vol. 2: Perception, animation and information retrieval).

In addition, substantial excerpts and some extra materials are available at the External Programme's news page, and on the External Programme's page for volume 1.

In addition to the guides, it is essential that students remain familiar and extend their skills with Processing, and additionally gain fluency with either R, an environment for statistical, numerical and signal processing and visualization; the following titles may be of assistance:

C. Reas and B. Fry, Processing: A Programming Handbook for Visual Designers and Artists;
D. Shiffman, Learning Processing;
W.N. Venables and D.M. Smith, An Introduction to R

Additional Reading

In addition to the core technical material, it will be helpful for understanding (technical or cultural or both) to read around the subject. Some suggestions:

J.D. Foley, A. van Dam and others, Introduction to Computer Graphics;
A.V. Oppenheim and A.S. Willsky with S. Hamid Nawab, Signals and Systems;
R. Dawkins, The Blind Watchmaker;
G. Deutscher, Through the Language Glass;
R.P. Feynman, Lectures on Physics, Chapters 35-36;
M. Stokes, M. Anderson, S. Chandrasekar and R. Motta, A Standard Default Color Space for the Internet – sRGB;
S. Handel, Listening;
D.M. Howard and J. Angus, Acoustics and Psychoacoustics
D.J. Levitin, This Is Your Brain On Music
O. Sacks, Musicophilia
I.H. Witten, A. Moffat and T.C. Bell, Managing Gigabytes: Compressing and Indexing Documents and Images
C.J. van Rijsbergen, Information Retrieval