An playful experiment in projection mapping on three cubes; a story about people and things being trapped and constrained.
produced by: Jesse Wolpert
I chose three words for each scene to inform the direction of the section. Each scene had one word that indicated the shape of a 2D primitive to work with. The other two words reflected the aims and tone of the scene. My aim was to create three very different scenes to explore the visual effect on my struture.
Concept and background research
The overarching idea in this project is that of people and things being trapped and constrained in a structure. I feel it nicely mirrors the constraints of the coding itself and the associated frustrations it brings.
The story of scene 1 was inspired by the five house moves I have made over the last year. I wanted to convey the idea of being trapped inside packing boxes and feeling adrift and dislocated. This scene highlights the physical structure of the boxes and begins by outlining the edges with a waterfall of white dots. I continued to play with the physical volume of the boxes by allowing viewers to see the creation of the outside shell of the boxes and then placing myself and my eyes in the boxes. I then reinforced this physicality by creating a sheet of green slime that slides gradually over the boxes to convey their geometric structure.
Scene 2 covers the boxes in lines using noise and randomness. It is divided into two halves and the patterns created act as scene changes between the two other narrative scenes (1 and 3). I was interested in the idea of a pattern growing and how the box structure could disappear behind the sheer number of lines.
The third scene has an intended cartoon like quality. As the project criteria required that there be no user interaction I decided to instead create interaction between the different faces of the cubes. I deliberately used objects typically found around the house to reinforce the theme of moving and packing up. It is very different in nature from the other two scenes as it primarily uses strict timing to precisely control the interaction between the objects.
For scene 1 I was somewhat influenced by the visual comedy of Monty Python and the film Brazil.The second scene was largely inspired by fairy tales where characters get trapped in buildings covered in ivy - like sleeping beauty. When making scene 3 I often thought about Tom and Jerry and all the times that Tom seems to get hit and then come back to life.
The whole project was created in openframeworks using the ofxpimapper addon. Our group chose three stacked cubes as our projection mapping surface. I first had to make the hard artistic decision regarding how to map my frame buffer objects to the nine surfaces of the cubes. In the end, two of my scenes (scenes 1 and 2) rely on just one fbo each to cover the whole object. As scene 3 requires very different things to happen in each of the faces I resorted to making three fbos to cover the nine faces.
One of the hardest thing to balance I found was the relationship between randomness and precision. The scenes all required a level of randomness - the waterfall of dots, the floating eyes, the circular slime, the red scribble, the blown away tennis balls, the vibrating ping pong balls etc. But a lot of precision was also required: the places where the waterfall dots end up to create the box edges, the exact placing of the sides of the packing boxes, the eyes peeking through, the moment the slime hits the box, the rotation of the sides to reveal the inside of the boxes and the timing of the interactions of blowing, hitting and colouring. When trying to get things to happen at precise moments I struggled with how to effectively know exactly when something would happen. As the frame rate was unreliable I spent a lot of time playing with frame number and elapsed time to create perfect sequences. Since these timings were usually linked to videos I found using the frame number of the video to be the most accurate timer to use.
I tried two different methods of approaching the technical side of things. The first method involved finding a piece of code similar to my requirements and then adapting it for my precise aesthetic. This was the method I used for the black and white noise growth section of the second scene. The second method required me to write the code from scratch to try and accurately create the idea in my head. I used this method for the second part of the second scene when I create the growing red scribble. The first method means you can quickly get something up and running but it is often a longer task to understand what is going on if the code is not commented. The second method is more satisfying and I found tended to lead to more interesting results.
I initally found it was often difficult to predict how hard an idea was to implement and I ended up going down a lot of dead ends. As the project progressed I became much better at understanding the coding structure required to achieve my goal and was able to be more realistic about the time required to complete each section.
I feel this project could be developed into a much longer piece about the nature of posessions and the way they can trap us. I think the essence of projection mapping lends itself well to story telling. The ability to locate the narrative both inside and on the objects creates a very satisfying flow for the storyline. If I were to develop this piece further I would incorporate user interaction to drive some of the visuals in order to allow the viewers to have more direct engagement with the narrative. I am not sure I would add any sound as I feel the piece is reminiscent of old silent black and white movies and cartoons. There would also be scope to add some projections that spill outside of the structure as a metaphor for freedom and escape.
I see this project as an experiment in projection mapping and as such I believe it is successful. As a finished work of art I think it has a number of problems; the two narrative scenes are only loosely connected, the storylines are slight and incomplete, some of the sections are too static and don't play well to the strengths of projection mapping and the possiblities of using the shapes effectively are not well mined.
During the pop-up exhibition I found people responded well when the cubes became packing boxes, when the moving eyes appeared inside the boxes and when the ping pong balls were hit and vibrated. I feel these sections all have potential to be extended and deepened into longer more complex sequences. During the pop-up I noticed that the rotating eyes, the blowing tennis balls and dice section did not hold the viewers attentions well and I feel they all need a lot more thought and development to function well in this piece.
My aim was to create a short and funny experimental piece that fulfulled the assesment criteria - and I feel this has been largely achieved. However if I were to do this project again I would try to be much clearer about my narrative arc and be braver about trying to fulfill my artistic vision rather than working to the level of my programming skill.
And finally - next time I plan to avoid the graph below - by closing my eyes, getting up from my desk and walking around *regularly*.
1. All the code from Workshops in Creating Coding by Theo Papatheodorou, https://learn.gold.ac.uk/course/view.php?id=6852
2. Noise Drawing by Zach Lieberman, https://github.com/ofZach/recreatingThePast_fall17/tree/master/noiseDrawing