Physics, Springs and Interaction
This project explores the combination of physics based systems and webcam interaction. Allowing the user to control and influence the system presented to them by applying forces to the particles on screen, I wanted to create an engaging and fun interactive experience.
produced by: Jakob Glock
Concept
The concept behind this project was to explore how user interaction, through the use of a webcam, could be used to create an engaging interactive experience. Using optical flow and physics based particle systems as a basis, I wanted to explore the possibilities of how these two techniques could be combined together to create a project people could both play with and enjoy.
Process
In creating my project I took an iterative process, which involved creating small experiments away from the master branch of my project. These experiments would then be tested for a couple of days to gauge how my interest levels changed over this period of time. Depending on how this test period developed, the experiment would either be thrown away or merged into the master branch. This process allowed me to explore various avenues in how the users movements controlled the particles on the screen and how this control could be fine tuned to produce the effect I wanted. It also enabled me to try out different aesthetic choices which lead me to my final program.
Technical
My project builds upon the OpenCV "Camera Controller" example from week 14 which utilises the webcam and Optical Flow to create a field of velocities which describe the movement of the user. With this knowledge in hand, I realised that these velocities could be used to move particles across the screen. My initial experiments where extremely basic and created thousands of white particles over the areas of detected motion, while engaging and fun to use, creating thousands of particles a frame proved to be too CPU intensive and lead me to optimise the OpenCV section of my code. I decided to make use of multiple threads and put all of the webcam and OpenCV code into its own thread, this freed some processing power in the main thread and made my program more reliable and faster.
Implementing my particle system took various forms, my initial experiment as explained above was extremely simple, but I soon realised that I could set my system up like a Flow Field. Using this technique a particle is always reading the OpenCV data to grab a velocity and apply it to its movement as a force, this allowed me to create a defined number of particles which the user could then move around the screen.
At this stage my program was just a series of white dots on the screen, so I decided to represent the webcam image as a series of smaller tiled squares to allow the user to play with there own image, whilst still employing the same forces to the particles. I felt that this was a great improvement to my project, but it was not the aesthetic I was looking for. Whilst playing with the program I felt that the image became too cluttered and confusing far too quickly and decided to explore a different route.
My next route was to explore using springs, I made this choice as the user could interact with the particles and move them around but they would always be restricted in there movement and return to the original position. I also decided to give each particle a colour based on its position in a grid which was read from the webcam image, this displays the users image in front of them in a pixelated fashion. This resulted in a very organic fluid like motion that left the users image swaying back and forth, I felt that this slight deformation of the image caused by there motion would be just enough to entice the user into interacting with the program. My final decision was to make it possible for the user to effectively break the springs, this results in areas of pixels being able to move freely away from the grid, but after a certain amount of particles have been disconnected they will slowly rebuild themselves until the original image is restored. Juggling the parameters of the forces and limits of my program proved to be a challenging aspect of my program, and I think these would change depending on the performance of the program on specific hardware and depending on the location it was installed, I would take the approach of testing and deciding on these values on a case by case basis.
References
- Camera Controller - From week 14 of Workshops in Creative Coding Term 2
- OpenCV addon