Ridge of Sentiment
Ridge of sentiment’ is an interactive projection mapping which adapts human’s brainwave and with nature theme through the visual and the sounds of nature.
produced by: Yunjeong Jang(yunny)
Introduction
Have you thought about how your brainwave looks like?
Have you been influenced by nature or put your influence on it?
‘Ridge of sentiment’ utilises mainly the machine called EEG(Electroencephalography) which detects a person’s brainwave applying electrophysiological monitoring method. In this project, the EEG scans your attention and meditation levels, capturing them in current time.
Through ‘Ridge of sentiment’, I intended to represent the interaction between the humans and nature. I reckon that The biggest similarity between brainwave and nature is the diversity. The most of the elements I used in this project were deeply inspired by nature. I represented this concept through the sounds of nature and the graphical projection. The visual concept presented through the hexagon shape object structure.
Concept and background research
My initial questions before I had schemed this project were how computational factors can affect the nature, and the way human’s brain gets influenced by the digital materials and the nature around us. Thus, I decided to take forward the research which was done during ‘Computational Arts-based Research&Theory' class which I participated last year, using deep academical approach, I investigated the relations between brainwaves and art specifically figuring out the way to influence art piece structure in relation to human’s emotion represented in brainwave forms. Moreover, the investigation I implemented last year encouraged me to gain a general knowledge about how brainwave works and practised with brainwave machine called EEG.
During the specification of the theme, the greatest agony was making the connection between the points of interest; Especially nature and brainwaves those I desired to represent via the project. As a result, I ended up discovering the similarity among various factors. I decided to the coherence as ‘diversity’. Also ‘chaos theory’ has inspired me in terms of underlying a state of disorder.
The core questions which I try to represent in my work are:
What if the diverse nature may be the container for the chaos system which I preset using computational object and existing within the apparent randomness of chaotic complex systems? What if that tangled chaotic systems can be manipulated by brainwaves?
From my perspective, not only nature has enormous diversity such as colours, shapes, sounds and etc, but also brainwaves are partly predictable and appear in different forms. As a result, I subtly utilised the concept which includes nature, brainwave and digital material by means of visuals which is generated by codes, sounds and object I mapped.
Visual
First of all, I have got inspired the hexagon shaped object by honeycomb and nature hexagon basalt column in the Giant’s Causeway of Northern Ireland. Those are created only by nature itself and none of the human has manipulated those original shapes. Generating the biggest amount of outcomes with the minimal power is one of nature’s fundamental aspect, and the hexagon shape is ideally fit for this purpose.
Secondly, the colours I selected to create graphics are mainly came from the natures especially flora and sky. The core characteristic of nature pallet is high saturation and vivid ratio, so I intended to avoid an indistinct spectrum like pastel tone. Also, to represent the brainwave more efficiently, I was graphically inspired by Mindswarms by Random quark (Theodoros Papatheodorou and Tom Chambers) in terms of the colourful visuals, curved waveform and how candidate’s real time feeling state affects visualisation.
Brain Waves
There are 5 types of brainwave forms; Gamma, Beta, Alpha, Theta and Delta. Each brainwave refers to the different condition of the brain Gamma, Beta and Alpha are the awakening statement and Theta and Delta are activated during the sleep. In my project, I used eSense Attention/Meditation Meters. Beta is related to attention level which is a user’s mental focus or concentration. Meditation level is connected to Alpha and it measures calmness or relaxation.
Between those two parameters, attention level is more attractive to me so, I mainly aimed to calculate the audiences’ real time attention status in the congested exhibition place affected by external factors such as noise and visual disturbances. At the same time, meditation statue among the crowded ambience was also fascinating.
Sound
All the sounds I used extruded from nature and affected by brainwave status (apart from background wind).
Attention level is related to the ocean sounds. Rain soundscape refers to meditation level.
When meditation level is low, the audience can hear the thunder. On the other hand, crispy wave noise means high attention level. Additionally, there are particular wave ranges which represented through birds and wasps sounds.
fig 1, 2, 3
Technical
To create the interactive art piece using brainwave, technically 5 core aspects could be considered those are implanting brainwave to the coding program, visual and sound implements, building object, projection mapping and brainwave forms lightings, all codes are running in processing. I coded 3 processing sketches for running much smoother with less error, one is for sounds and the others represent to visual.
Brainwave
The EEG machine I selected is ‘Neurosky Mindwave2’ which has reasonable price and provides the possibility to analyse all specific brainwave forms. Also the company Neurosky offers some libraries to convert raw brainwave data into the scaled date which possible to apply. I choose ‘ThinkGearSocket’ library with ‘Neurosky’ library for processing codes. Those libraries include java file which I can modify, are written easily and enough understandable for me.
Visual
The 5 hexagons can be divided into 6 each triangle and I created 8 scenes applying to triangles. About the below image, it refers to each scene with different 8 colours.
The colour yellow green and red appear kaleidoscope shapes with boolean rectangle/rotation mode those two colours have basically the same code structure but different colour. Also, those interact brainwave in colour changing and rectangle/rotation style. Basically, the scenes are drawn by ellipse with PGraphic and clipped image. That inspired by ‘Jeremy English’ design.
In my codes this project, I applied attention/meditation level for various purposes by 0-100 gauge. Colours, shapes, the number of random lines and etc. have been adapted in a particular range of eSense levels. Moreover, through modulo, PVector, Array, variables and lots of different functions, I have achieved what I intended to display aesthetically.
fig 4/ colour codes
Sound
All the sound sources came from the website ‘Soundbible’ and are checked as entirely royalty free. To apply the sound with codes, I selected processing library ‘Minim’ and also put brainwaves data to play sounds in the certain position.
Object
Instead of one big hexagon, I preferred 5-6 hexagons with different size to have the more beautiful outcome. The object is made by 3mm MDF wood with laser cutting, and designed via adobe illustrator. I initially planned small triangles in the hexagon objects with the diverse height of triangles. So, I decided to pile some triangles and few of them has wood blocks for being above following different height perspective. In addition, I painted the brown woods to white colour to reflect the projection lighting more efficiently.
The overall size is 1300*1040mm, the big hexagon is 600mm*520mm and the small hexagon is 400mm*346mm.
fig 5/ final object shape in exhibition place
Projection Mapping
To map conveniently, I determined to use ‘MadMapper’. However, importing processing sketch to MadMapper needs a library ‘syphon’. Here I had a small problem. Syphon can accept only P3D sketch but my pre-coded sketches were all 2D. After that, I changed some parts of existing codes to P3D because when I ran the code as P3D, so many parts were missed or worked the wrong way.
In the MadMapper, I drew triangles, rectangles and hexagons as making the specific design to be in accord with the initial scheme.
Lighting
For the better understanding of audiences about the brainwaves, I decorated one side of the walls with 5 lighting pieces. 5 lighting wires are representing each brainwave forms: Gamma, Beta, Alpha, Theta and Delta and all lines have different colours. The material is called as ‘EL wire’ which is convenient to bend and glows gorgeously. I nailed each peak of the forms then hung the wire zigzag.
fig 6/ lighting design in the exhibition space
In general, I had struggled technically until the last moment. I have learned processing and open frameworks for a few last years. To create better algorithms and embodiment of my ideal are always the hard job for me. I analysed unfamiliar functions and try to understand the whole algorithms in the library were the harsh challenge. But through this project, I realised my huge improvement in coding.
Future development
Now I have two future projects related to this project.
First of all, I want to create an opposite perspective on the method I used. Here visual and sound are generated by brainwave and people can check the brainwave instantly. That means my project is up to human’s brainwave input. But, what if I make something when people choose the emotional mode, then analysing the brainwave and produce specific graphics and sounds to reach the brainwave value which refers to particular emotion. I reckon that it is possible because I tested many people’s brainwave during the exhibition and figured out when I put the headphone to people, their meditation level was rising quite quickly. Moreover, I am thinking about the future project not only as an art piece but also a useful tool for psychological cure therapy with much user friendly way.
Secondly, creating VR world using brainwave sounds fascinating to me. I have interested in making new eco-system with the new world and I have few works about this. Using VR and if the VR world may be generated and manipulated by the user’s brainwave are an interesting topic.
Self evaluation
I have studied processing for a few years since I decided to work in media art. But shamefully, I always was not confident in coding. When I faced even a bit complex algorithm, I was struggling neither understand the code nor use the algorithms and functions. For 5 months, I studied processing again step by step and checked myself if I had got critical missing. Especially creating algorithms has big pain for me because I should make a plan logically, trained myself and I spent the most of time to study processing for the better quality of codes. As a result, I have got an enormous improvement. Now I can understand Java library, also can modify it and, as a result, making decent codes myself.
Frankly, when the exhibition was left only a week, I already put 6 scenes in one processing sketch. (But I used the method that making each scene in a separate sketch and put everything into one sketch with libraries) There was a trouble. When I inserted one more scene, processing had totally crashed without any error messages and too laggy. So I separated into two sketches and it starts working better but still has some lags. I did regret my decision to use processing instead of openframeworks. I felt openframeworks has a stronger engine than processing.
The peoples' behaviour during the exhibition was unexpected by me. Some of them are really interested in my work and asked thousands of questions not only technical but also concept and what I wanted to represent through the project. But a couple of times they broke my EEG machine in a short time and I spent a few hours to fix it. Next time, I need to prepare more precisely for special occasions during the show. Besides, some people asked me the questions those are related to the brainwave shape in a certain place or exotic condition of a human. Also, the most people could concentrate on my work quickly in the crowded exhibition space and at the same time felt relaxed due to music. That may be different for each person but the most users did not seem to have unique data except few of people.
References
concept and background
[1] Sumita Sharma, Blessin Varkey, Krishnaveni Achary, Jaakko Hakulinen, Markku Turunen, Tomi Heimonen, Saurabh Srivastava, Nitendra Rajput. (2018) Designing Gesture-Based Applications for Individuals with Developmental Disabilities. ACM Transactions on Accessible Computing 11:1, pages 1-27.
[2] Seymour, Valentine. “The Human–Nature Relationship and Its Impact on Health: A Critical Review.” Frontiers in Public Health 4 (2016): 260. PMC. Web. 15 Sept. 2018.
[3] Mystery solved: Why nature loves the hexagon, and why without it you’d be a puddle of goo
Mystery solved: Why nature loves the hexagon, and why without it you’d be a puddle of goo
[4] Why the Natural World Looks the Way It Does, by Philip Ball, published by The University of Chicago Press. © 2016 by Marshall Editions.
[5]Artefacts using human brainwaves
Artefacts using human brainwaves
[6] Mindswarms – Random quark (Theodoros Papatheodorou and Tom Chambers) fig3
[7] FU-CHIEN KAO, SHINPING R. WANG, YU-JUNG CHANG , Department of Computer Science & Information Engineering Da-Yeh University. “Brainwaves Analysis of Positive and Negative Emotions.”, WSEAS TRANSACTIONS on INFORMATION SCIENCE and APPLICATIONS, 2015, pp. 200-208
[8] https://steemkr.com/stemng/@greenrun/hexagon-in-nature-why-is-the-honeycomb-hexagonal fig1
[9] https://www.justbeedrinks.co.uk/nature-prefers-hexagon/ fig2
Technical
[1] kaleidoscope design by https://www.openprocessing.org/sketch/30186# 'jeremy english’
[2] ThinkGear socket library https://github.com/borg/ThinkGear-Java-socket
[3] Sound Sources http://soundbible.com
[4] Processing library ‘Minim’
[5] Processing library ‘Syphon’
[6] https://processing.org/reference/
[7] https://processing.org/reference/PGraphics.html