Field Composition
A multi-channel A/V performance as part of MA/MFA Computational Arts Degree show.
3-Ch Audio-driven generative moving images.
8-Ch Spatial soundscape composition. The spatial paths of 27 audio inputs will be controlled in real time. All audio samples come from the artist’s personal recordings from daily field trips. The whole piece is built based on Max/MSP/Jitter and external Spart5 made by IRCAM.
produced by: Taoran XU
(To get binaural listening please put on headphone while watching the video)
Concept and background research
My interest towards environmental sounds, space and human intialed back to the MFA 1st year's study when I created my first soundscape related work Headscape(http://doc.gold.ac.uk/compartsblog/index.php/work/headscape/). Headscape is the realisation follows my individual binaural study in acoustic perception which I did for Compuational-arts Based Research module in term 2. In this research I studied the capability of how headphone and sound can manipulate the perception and acoustic images in the listeners' heads based on one case : A soundwalk piece made by Janet Cardiff.
In order to explore further, during the second year study I audited Phonography and Sonic Art Practice modules taguht by Iris Garrelfs and Sonic Art Techniques moduel taught by Ian Stonehouse from Music department. Iris was very kind to give me two individual tutorials discussing my works and ideas and she guided me to names such as Barry Traux, Denis Smalley and Trevor Wishart who work in the field of the soundscape composition and multichannel sound diffusion and sound transformation.
10-11 November 2018, Sound - Image conference 2018 @Grennwich University
I attent the conference and its night concert with special guest Denis Smalley featuring his new multichannel composition work.
It was my first time listening a multi channel piece and it was pretty impressive of how a multichannel sound system and a great piece could offer the sense of immersion to you. I still felt very luck that attent the concert and saw Denis Smalley in person.
30 Nov 2018,Barry Truax: talk and listening event @SIML, Goldsmiths
"Soundscape Composition: From the Real to the Virtual", a talk and listening event given by composer Barry Truax, Professor Emeritus at Simon Fraser University, Canada. “ During the seminar, Barry talked about his composition strategies and played several his multi channel works in the SIML. I still fascinated by his idea of finding the perfect balance between abstract and real in terms of composing recored sounds. And it was time I realized that people can perform multi channel pieces at SIML. Luckily, not long before I saw Denis Smalley in person I met Barry Traux in person as well.
04/2019 - 07/2019, 10 weeks short course
After having researched for a two terms I decided to get my hands dirty so I took part in the short course led by Dr Marcus Leadley, who is the manager of Electronic Music Studio of Goldsmiths. The 10 weeks study involved getting used to know different mics and how to mice them, such as shotgun mic, binaural mic, contact mic, omni directional mic and coil mic; A field trip towards Greenwich , most of the sounds I used in my piece was captured during that trip; Mastering basic Pro tools software operation; Using Pro tools HD and multi channel plugin ''Spat" in the 8 - channel studio. The course built me confidence about creating a multichannel soundscape piece and perform it at SIML.
Technical
Audio:
The whole system was implemented on Max/MSP environment. The sound spatialisation part was relying on Spat 5, which is an Max/MSP external specializing at sound diffusion and acoustic modeling, developed by IRCAM.In order to get best listening experience using Spat 5, you need to 1) measure the speaker array’s position in the venue 2) calibrate the gains and delays for each speaker.
1)Measurement
I used a laser measure to roughly measured the space and the speakers. The coordinate system follows the coordinate system used in Spat 5. After the measurement I created a virtual speaker array system in Spat 5, software based.
2) Calibratation
a) gain calibration
Spat offers an object to deal with gain calibration, in order to get proper calibration result you will need to plug in a omni directional mic, set it in the center of the room(corresponding the 'listener' position in the spat). It will capture the sound output by each speaker and based on signal levels the mic provides, the spat will caculate a calibrated result. And then apply that result. In this case, I borrowed an AKG 414B mic from the department(Never knew we have so many good stuffs from the tech office), set its capture pattern into 'omni' and get the final result.
b) Delay calibration
The delay calibration scheme was similar to the gain calibration with same mic setting. Get the result from corresponding delay calibration object and apply the result
Visual
The visual was driven by the sound, to be more specific, the current gains of each speaker were sending to a Processing sketch from Max via OSC commnucation. The gains values will be used as of inputs the algorithm of the generative moving image. The Processing sketch will be generating different water bubbles in different raduis and brightness according to the current gain values being sent from Max. Then, the Processing sketch will send each frame to Jitter via Syphon communication for better visual aesthetics.
The visual design mainly concerns the A/V sensorial perception and integration. Since I am using recorded sound, not genrated, there is no need to make a waveform based or other FFT based visual in terms of sensorial perception. The audience will maily notice the sounds have their directions and distances.
So I set 8 bubble generating areas in the Processing sketch correspoding to 8 speaker's position. In this way, no matter which sounds are playing or moving, the visual will aesthetically be percieved as the speakers will be projecting the sounds differently according to the current positions in the space of the sounds. The closer the louder the brighter the and more obvious.
Just like the fuild water can be interfered , the sounds in the SIML will be interfered as well.
Composing Details
As a non - musician, my composing strategies might differ from the academic ways. I treat my field recordings as my material sources and I selected certain parts of the materials as objects. Sound objects. Put the sound objects into where they should be to create a sound sculpture. Cutting the objects from the tracks as to detach the sounds from their origins, like simulcaras, and then put thos simulacras under a new context, my own artistic context.
Get sound objects
Similar to making a sculpture, the first thing I did is to listen every recording I have and extract insteresting sound fragmentations from the tracks. I used Pro Tools to pick up the sounds I wanted. And I used basic panning and filtering techniques to 'polish' the objects, to get rid of their origins.
Place the sound objects
Placing sounds in the spat, testing and listening.Room effect
The spat can create artificial room reveration and delays, which are useful to create acoustic effect for the sounds which help to create immersion and narrative. Manualy to control the room effect from dry to wet.
Set Spatial Motion
Inspired by Trevor Wishart's outstanding book<On Sonic Art>, chaper 10, I made different spatial motion patterns for my different sound objects. Watch the video to see further description.
Sonic Narrative
1) Crabs in a cave, talking and chatting. (Sounds at bottom, put your ears at the bottom)
2) Cave somehow starts to shake, bats flying out. Crabs are escaping from the cave.
3) Strong water wave is coming and hinting the cave.
4) Cave is destroyed and crabs are gone, wave is fading out, world back to peace again. Birds are singing.
5) Finalising with multiple airplanes.
About Crabs:
I lived somewhere near Canary Wharf and one day early morning I went to the Billinggate Seafood Market bought two crabs. I kept them in the my bathroom's sink. Some moment later when I came back to the bathroom I heard them chatting, a very unique and inetesting sonic phenomenon. Felt like they were talking to each other about how to escape from the place. I recorded the conversation. Have a listent at:
https://www.instagram.com/p/ByuJk-ZhCL2/?igshid=1ew4kopjn9els
About water:
I lived by a river bank and one of my favorite outdoor excerices is to go down to the river bank and listen. When a ship is passing by, it generates several generations of waves. The waves arrive at the bank at different times, different directions, differnt velocities. I can hear the sounds coming from left to right, i can hear the inner rythem, it is orchestra. I am facsinated by the listening experience. Have a listen at:
https://www.instagram.com/p/BxBEKxHF1pt/?igshid=l78solpskhj8
About bird:
One day this Spring, I could not sleep and stayed up to the next morning. Sunshine and birds greeted me when I sat in the balcony, I recorded that moment. Have a listen at: https://www.instagram.com/p/BxOyX9IFzUg/?igshid=1h6l6yeja0b4q
About airplane:
One of my favorite spot in London is the campus green of Goldsmiths. When I lying down on the grass I feel extrmely pleasantly satisfied. I can always see airplanes in the sky every time I look up and the airplanes in London are flying so low comparing with my hometwon in China. I can feel the planes just flying above my head and I hear the sounds from the engine clearly. It is a new listening experience to me as well. So decided to finalising the piece with multiple airplanes landing and taking off. Inspired from the listening experience i have when i lying on the campus green but emphasis or augmented the experience by my artificial composing.
Further Developement
1) Better sound 'polishing' technique, the sound objects could be polished in further detail.
2) Spatial motion: More organic movement, could apply swarm or boid algorithmes.
3) More sophisticated layering.
Reference
1) http://forumnet.ircam.fr/product/spat-en/
2) https://www.youtube.com/watch?v=BZUdGqeOD0w&t=313s
3)
4)
Special Thanks
Adan / Thanks for the sound checking, mixting desk tuning
Ben / Thanks for helping me during the change over, saved me from the opening night(wrong osc, no visual)
Chris / Thanks for helping me during the change over, sound check with Motu
Chritina/ Thanks for helping with organizing the program
David / Thanks for helping me during the change over
Dr Marcus Leadley / Thanks for lending me Soundman binaural mics during the exhibition so that I could record for everyone.
Pete / Thanks for trusting me and lending any equipments I wanted, let me do anything I wanted
Valeria / Thanks for GoPro Fusion shooting and rendering and filing
Ru Pert / Thanks for static wide, close moving footages shooting
Ziwei / Thanks for shoot extra footages for me