More selected projects

Anatomy of an AI

This projection mapping piece was a visual exploration of the hidden anatomical elements of AI, including obscured labour costs, data costs and environmental and ecological degradation, all ideas laid out in Kate Crawford and Vladan Joler's paper/anatomical map 'Anatomy of an AI System'.


produced by: Lou Terry

Introduction

My initial decision was what surface to project on.  In Kate and Vladan's paper, sierpinski triangles (a potentially infinite structure of smaller triangles within triangles) are used as a visual metapahor and also a pragmatic diagram of the hidden anatomical structure of an AI (Amazon's Alexa specifically).  This was therefore a shape I particularly wanted to code into my piece, and so the Olga Kit (a surface made for projection mapping assembled from triangular faces) seemed right.  I also wanted my piece to be unfamiliar, and abstract (which the Olga Kit was), to mirror the nature of hidden anatomical elements of AI systems.  Below is the final structure we built from the Olga Kit.

Concept and background research

I wanted to communicate Kate and Vladan's ideas in a visually abstract way.  For me, some of the most effective projection mapping projects from previous years made use of darkness and negative space.  I wanted too to do this, and so began the first section by only projecting brief flashes on certain faces of the Olga Kit.  In this way the overall shape of the Olga is obscured, you only see part of it at any one time - much like Amazon's Echo, which exists in the physical world as a small, contained and benign cylindrical object, but whose underlying structure in fact includes much more.  From the vast amounts of resource extraction and environmental destruction from lithium mines in Bolivia, Chile and Argentina, alongside Cobalt mines, and the huge abuses of human labour that go into these (a child as young as 7 working in a Congolese Cobalt mine would have to work 700,000 years 24/7  to earn the amount Jeff Bezos earns in a single day), to underpaid (and sometimes unpaid, as in the case of Google's ReCAPTCHA) workers labelling data training sets, to heavily controlled and dangerous hardware manufacturing factories in China, to the millions of cargo containers and ships transporting these parts across the globe, to the carbon emissions from running machine learning networks that learn and enact on data fed from our commands to 'switch on the light Alexa', Amazon's Echo extends far beyond the object we see in our living room.  The Olga was then my metaphor for this hidden structure, and the image (taken from Crawford and Joler's map) of a mechanical Turk (both an 18th C. Hungarian contraption/illusion that appeared to be an 'automaton' that could play chess against its user, where in fact hidden inside was a human labourer moving the pieces, and also the shameless title of Amazon's crowdsourcing software) seemed the perfect image to accompany this opening scene.   Eventually the triangles evolve to form volcano-like structures on the corners of the Olga.

"According to the Aymara legends about the creation of Bolivia, the volcanic mountains of the Andean plateau were creations of tragedy.  Long ago, when the volcanos were alive and roaming the plains freely, Tunupa - the only female volcano – gave birth to a baby. Stricken by jealousy, the male volcanos stole her baby and banished it to a distant location. The gods punished the volcanos by pinning them all to the Earth. Grieving for the child that she could no longer reach, Tunupa wept deeply. Her tears and breast milk combined to create a giant salt lake: Salar de Uyuni. As Liam Young and Kate Davies observe, “your smart-phone runs on the tears and breast milk of a volcano. This landscape is connected to everywhere on the planet via the phones in our pockets; linked to each of us by invisible threads of commerce, science, politics and power.” (Crawford & Joler)

The next section involved a tracing out of the perimeters of the triangles, expanding out like strands of data networks between nodes.  Images of Alexa, Lithium and a diagram of a human brain quantified into sections of intelligence, then filled the triangles.  Following this, a repeated structure of warping triangles grew and shrank into the abyss.  They had a cage-like quality, which echos the treatment of Amazon's employees (Bezos has even designed a cage with a seat and automatic arms from which warehouse staff can sit and navigate the warehouses).  They also resemble the tiered structure of lithium mines (images of which are also shown during this stage in the piece).



In the final section the previously mentioned sierpinski triangles finally appear.  The triangles begin to rotate and traces of their previous positions remain.  They resemble plant-like structures, and simultaneously images of the palaquium gutta tree emerge.  In the mid-1800s, palaquium gutta (native to Malaysia) were used by the British, as the trees were found to produce a natural latex that was useful in making transatlantic telecommunications cables, stretching thousands of kilometers across the seabed.  The land where the trees grew was completely deforested and the trees are now extinct.  Thus, one of the first ecological catastrophes of the modern technological boom occurred.

Technical

The projection was all made using OpenFrameworks.  The sierpinski triangles were coded using help from this Youtube video

I added my own elements by semi-randomly changing the amount of triangles contained within the sierpinski, whilst leaving traces of the previous ones.  I also had the triangles rotate, applying noise at various levels to help achieve an organic movement.

One of the hardest things to code was the 'network paths' drawn around the perimeters of the triangles.  This is because each face was a separate fbo source, and there was no way I knew to get the sources to communicate with one another.  Therefore I had to use a lot of 'if' statements and a lot of variations of the same fbo source in order to choreograph the correct times for each line to be drawn, so that they appeared as if being traced across the whole Olga surface.  It was also difficult to get the lines to grow from point A to B, or 'be drawn' between nodes.  In order to do this I used the formula to find a point between two points (x1, y1) - P*(x2, y2) where P is between 0 and 1.  I mapped an ever-increasing variable 'phase' to 0 and 1 and applied it to 'P' to achieve the 'drawing' effect.

For the warping triangles I applied multiple sine functions to achieve the ebbing and flowing, one large one, and smaller ones that came into effect at the extremes of the scalings, to give the triangles the illusion of hitting the faces of the Olga and bouncing from underneath, as if they were trapped in cages themselves.  I also applied noise to these functions to help achieve organicness.

Future development

I think a nice development would be to have the fbo sources communicating with one another, and the 'network paths' (white lines drawn around the perimeters of the triangles) could then be coded in less laborious fashion, and could interact with one another - there would be more room for chance in the way the networks grew across the surface, allowing them to spread out perhaps many times over the duration of the piece. 

Kate and Vladan's paper also goes through many more hidden anatomical elements of AI (I picked just 3 to focus on).  But with a longer piece I think further exploration into the other elements they mention, such a data labelling, quantified human intelligence, corporate eavesdropping, (though touched on in my piece), could certainly be explored more fully.

I think another thing (which I have done with the video) would be to add sound to the work.  What would be really nice, is to have the sound interacting with the visual stimuli from the piece (or vice-versa), and I look forward to learning how to do that this term.

Self evaluation

I think the decision to make use of negative space, and begin by just projecting onto parts of the shape was a good one, it gave the shape an uncanny and unfamiliar feel and made it look as if it was appearing out of nowhere.  I also think leaving the central pyramid black throughout most of the piece worked well, as light from the corners where projections were occurring reflected onto the dark central pyramid, creating a third projection, and the initial projection becoming simulatenously a projection and a projector.  I think this works with the themes of hidden elements within elements, and sierpinski triangles. 

As you might notice, on the section where lines are being drawn around the perimeters of the triangles, it is not a continuous line, but there are in fact gaps along certain edges.  This was not meant to be the case (and in the projection from my original project, the line is a continuous entity).  However because of the nature and time constraints of getting everyone's varying shapes and orientations of sources onto a single project, it was not possible to get all my sources positioned correctly in time.  It would be nice to see this section as it was intended, though I didn't mind too much. In fact, the asymmetry occurring from this actually echoed other intended asymmetric elements of the piece (such as the positioning of the warping triangles, which, when fading out, do so in slightly varying directional configurations, giving the piece an 'almost' symmetry, but not quite) which I think aesthetically looks more interesting than pure symmetry.

I think the images, though useful in giving context to the abstract shapes of the piece, are sometimes hard to see.  Originally, I was planning on putting just one image in the central pyramid.  However when I tested this out, the protruding nature of the pyramid warped the images so much that they became distorted in an undesirbale way.  I think if I was to do this again, I would create a flat central square, not a pyramid, and project the images onto that.  This would also resemble a screen, which would work with the 'old TV' aesthetic acheived by the flickering black lines coded over the image.

Another future development would be to install this piece in exhibitions that centred aroun themes of environmental and ecological issues with tech, human and environmental exploitation, data and machine learning, and AI.  Afterall, this piece is supposed to provoke thought and is a political piece, and so needs to exist in the real world.

References

Barney, "Coding Challenge #9 Recursive Triangles (Sierpinski Triangle)" (June 12, 2018). https://www.youtube.com/watch?v=fwDkUxrFb0s, accessed Feb 2, 2020.

Crawford, K. & Joler, V., “Anatomy of an AI System: The Amazon Echo As An Anatomical Map of Human Labor, Data and Planetary Resources,” AI Now Institute and Share Lab, (September 7, 2018).