More selected projects

Sonification Drawing 

This interactive programme takes the audible data captured by laptop's built-in microphone and transferrs them into pixels on the Processing canvas in real time. 

produced by: XU Taoran

Introduction

This interactive programme takes the audible data captured by laptop's built-in microphone and transferrs them into pixels on the Processing canvas in real time. And the proframme runs in following steps:

1.  0 - 30 seconds. Dots generation:

The programme captures audible data from ambient environment and process the data. Processed data will be assigned to properties of each dot object. Each dot object has its position, radius, color. Based on each dot's properties, the programmes draws them on the cavas; All the dots are stored in a array list.

2. 30 - 60 seconds. Dots spining:

The programme captures audible data from ambient environment and process the data. Processed data will be assigned to the rotation angle for each dot.  Based on the its rotation angle, current dots will spin according to the center of the cavas. 

3.  After 60 sceonds. Clearing

The programme removes every object of the array list and clear the canvas by calling background(255); Go to step 1;

Concept and background research

As my optional module of this term is Advanced Audiovisual Processing instructed by Dr. Mick Grierson.  I would like to practice what I have learned from the course on the project. Plus, I find Islamic patterns beauty as they are symmetrical in a way. So the idea to make symmetrical image based on sound came out.

Technical

IDE: Processing v3.2.1 

Processing Library: minim (for audiable data input, fft analysis) 

1. Positions 

In order not to generate dots that are to close to each other I set the condition that the distance between new dot and the previous dot should be larger than 20 pixels;

In order to relate dots' postions with audible input I referrenced code from CodeCircle https://live.codecircle.com/d/ZqvP8eT7iZdnT3NmX 

for (int i = 0; i < in.bufferSize(); i++)
    {
      bing+=in.left.get(i);
    }

 bing = bing/in.bufferSize();

for (int i = 0; i < in.bufferSize(); i++)
    {
      bong+=(in.left.get(i)-bing)*(in.left.get(i)-bing);
    }

Variables 'bing' and 'bong' are then used for later processing.  

2. Rotation

In order to make symmetrical image I call rotation() function and drawing eight times based on the position of one dot.

So for every dot object there will seven counterparts on the cavas. 

 

Future development

1. More variation of 2d primitives

2. Use same interactive mechanism to interact with live video input instead.

3. Process audiable data in detail, i.e make a selection from low frequency and hight frequcency.

Self evaluation

Not so bad though the images made by the programme are more or less the same and recipients at St. James Church became bored after several iterations. I heard people stamping, laughing, whistling, snapping and singing at the exhibition as well.  

 

References

1. McLaren Standard Dev https://live.codecircle.com/d/ZqvP8eT7iZdnT3NmX 

2. Minim Examples/Basics/MonitorInput