BitSpace is a dynamic 'object-to-think-with' that aims to explore the decision making abilities of computational matter. It reifies probabilistic spaces of data and information within 8 bit encoding systems aiming to make the bit, the logical unit - not necessarily a numerical digit, and the Byte more conceptually accessible for learners.
produced by: Luke Demarest
Previously, I helped guide learning processes at artist studios, workshops, and hackerspaces while learning how to create Do it With Others (DiWO) communities and environments. However, I recently started my first formal teaching job on a part time basis and despite consisting of similar people, space, and material, creating a similar environment in more formal settings can be a bit difficult. I am not entirely sure why.1 This realization doesn't necessarily come as a surprise, but the teaching experience has lead me to reflect on what makes my default spaces so enjoyable. At the very least, it seems that groups that ultimately focus on community instead of testable outcomes have the most growth in regards to knowledge of self and knowledge of material. In this reflective frame of mind, I set out to build an initial prototype of an interactive learning tool for my younger self - roughly the age of the current students I interact with. I thought I could at least tease out some of my own subconsious preferences and those of others.
To inform my own practices of learning and play, I started reading papers by 'metacognition-ers' ( Piaget, Gatto, Papert, Stallman, Lessig, Tufte, and Victor) whose many thoughts and theories are embedded in learning tools currently in use. Most of the sources I read were gathered through researching the work and influences of Bret Victor and exploring his ideas of humane representation of thought. I started creating the prototype by building little sketches that tapped into the conceptual veins of Dynamicland and DataBasic - really jiving with the principles-based dynamic reality (DR) and the awareness that a tool for ease of use is not the same thing as a tool for ease of learning. I eventually concentrated on building a single moderately dynamic tool that, for now, abstractly addressed a basic data literacy topic -understanding the bit and Byte- required for computational literacy.
Photo documentation of some of the initial software sketches can be found at the end of this blogpost.
Create a single tool that attempts to (1) explain the bit (2) show how the base-2 number system works (3) hint that data and information are different concepts.
- A basic data literacy need for novice learners is understanding the bit as a logical unit instead of strictly being a numerical unit of 0 or 1. “Since the early 1960's, computers have been used even more often for problems in which numbers only occur by coincidence; The computer's decision making capabilities are being used, rather than its ability to do arithmetic” (Knuth, 1968). A bit is a space with two options - no or yes, off or on, dark or light, no voltage or voltage, and, of course, 0 or 1. This distinction can help start the process of peeling away stereotypes about who is allowed or able to work with numbers. It could further make sure that these stereotypes don’t invasively mutate into other spaces. Everyone can make decisions and actions - were just using a temporarily unfamiliar syntax.
- Separately show how the base 2 number system works. Just as how learning a second natural language allows you to ask questions about the nature of language, learning a second number system allows you to ask questions about the nature of number. 10 and 10 can mean two different things in two different systems. This exposure can hint towards the difference between data and information.
- Show a data 'layer' and an information 'layer' in a single image. "GDI defines information, broadly understood, as syntactically well-formed and meaningful data." (Floridi, 2015). I think engagement with and understanding of different, but intertwined 'bit' concepts can lead to a deeper engagement with the world, not just the initial world of computation. Within the right framework, a learner can replace a perception of a data-scarce world with the perception of a near data-omnipresent world. Learning is always possible if you are continously querying your system. However, its also important to share with the novice learner that these 'bit' concepts generally still have quite fuzzy definitions. "The word ‘information’ has been given different meanings by various writers in the general field of information theory. It is likely that at least a number of these will prove sufficiently useful in certain applications to deserve further study and permanent recognition. It is hardly to be expected that a single concept of information would satisfactorily account for the numerous possible applications of this general field." (Shannon, 1993). This prototype could eventually be coupled with a visualization of Floridi's data classifications that can guide 'data gathering' exercises.
I try to keep to a tight and consistent aesthetic across my work. Almost always use monochrome. Almost always focused on elements of form and process. I sporadically maintain a visual reference of artwork that ‘feels right’ to feed what seems to be a mostly subconscious process. This project in particular was largely influenced by my everyday London environment of organic and mass produced objects. A few photographic references are included below. I'd like to think that students also see these forms in their London day-to-day and begin to build intuition of underlying processes of structure and form across material.
Technical / Process
This project has three major technical areas (1) data representation (2) interaction (3) information representation.
- I concentrated on using eight physical marbles (first eight different colors, then all white) for representing eight bits or one Byte in the BitSpace. BitSpace was a term I orignally misused in the place of bit-length, but I continued to use it since I like its sense of an omni-directional space opposed to a linear space perhaps associated with the term 'length'. I experimented with papers, coins, rocks, and bottle caps along the way as physical representations. However, I did most of the machine learning training with coins. Marbles, on the other hand, had the least distracting connotations and the best prolonged rolling on the table surface providing a bit of unpredictability - perhaps inviting play. ‘Bitspace’ locations that have a marble present are classified as ‘on’, ‘Bitspace’ locations without a marble present are classified as ‘off’. I used computer vision with OpenFrameworks and the standard OpenCv addon to classify these ‘bit states’. I used the PS3EyeCandy camera for video source and disabled white balancing and color correction to normalize the video frame data. This was my first extended project working with Computer Vision.
- I built the interaction between the physical and virtual elements in two ways. First, when using eight different colored marbles I created a color tracking system for each of the eight different colors - noting the location of each bit on the table and then would flip a bit’s state when it was entering/leaving a designated ‘BitSpace’ location. This color tracking CV option worked, but was prone to some detection inconsistencies. Increasing color thresholds between the target colors and the tracked colors help correct errors, but the color readings were still affected by shadowplay - as I was trying both overhead and under-the-table video feeds - and required longer processing time as the full resolution video frame was analyzed (for each of the eight colors). The second implementation greatly improved the speed and accuracy of classifying 'bit states'. I used a supervised machine learning algorithm to determine the 'bit states' via Wekinator, which allows you to choose, train, and use supervised machine learning algorithms within your codebase over Open Sound Control Protocol(OSC). There are two general groups of supervised learning algorithms - classification and regression. Since there are two discrete classifications in my model (on/off for each of the eight bits), I focused on classification algorithms, which return pre-set mutually exclusive outputs defined in the training data, instead of choosing a regression algorithm, which can return a range of outputs outside of the training data. There are a number of ideal classification algorithms available to use - two comparison charts are included below. I made my final implementation with AdaBoost (Adaptive Boost). With increased training data, Adaboost is capable of creating very complex class boundaries and it is less prone to overfitting and entropy related issues than other algorithms. I created my model with 12 features. Instead of analyzing the entire full resolution color image like before, I created a 25% decimated grayscale image (without color data) and then segmented the bitspace area of that image into 10 values of average brightness. These 10 values were used as the first 10 features for determining if eight bits were classified as on or off. This change in resolution reduced redundant and irrelevant pixel data. Since I had issues with shadows and movement previously with standard color tracking, the final two features I included were optical flow averages for the x and y directions of the frame. This optical flow implementation was also created with the OpenCv addon and was based on an initial Workshops in Creative Coding exercises. These features provided the model direction for the variable shadow movement during analysis. This is especially useful during the training process. Generally, more training data results in a better coverage of your feature space. However, there is an exception to the rule. If you increase a training data set with only data that is (nearly) identical to each other, your classifier won't be able to significantly improve its learning since its not receiving any new data about the problem. Providing redundant training data also results in an unnecessary increase in training time since it provides little to no benefit on accurate classification. Knowing this, the 10 brightness values could belong to that exception. The 10 features coupled with the directional flow values provides more informative data for the learning problem. While training the model, I recorded short training examples with a slow transfer rate to ensure unique data while making small shadow movements from various directions. After training, I ran my model at a much faster rate. This proved to be the better implementation for the task as a fast and accurate processing rate is important for real time interaction. This technique was recommended by Rebecca Fiebrink.
- I used a genetic algorithm in the form of an L system for visualing the information 'layer' to show the base-2 nature of semantically structured data. It is designed to generate a population of eight eight-bit BitSpaces, but only one bitspace is chosen to be visible from each particular generation. When a bit in the chosen bitspace is turned off or on, a new 'bit' mating pool is created and that mating pool then repopulates the overall population. After selection, generation, and phenotype creation for the new BitSpaces, one new eight-bit BitSpaces with the correct current eight bit value is displayed in the window. Branch length and branch angle of rotation for the L system are created via pseudo-random values selected from it's 'parents'. Each level of the L system represents the probablistic options associated with each bit in a Byte. A learner can count each option that the bits hold or read the on/off values of each bit. This can lead to learning how to read binary numbers. If the level’s nodes/options are black it indicates that that bit is classified as off. If the level’s nodes/options are white it indicates that that bit is classified as on. The total base 10 value of the of the base-2 byte is displayed below the BitSpace information visualization and above the data visualization. The information 'layer' aims to show that a single eight bit value can have different expressive forms depending on what semantic system it is encoded in. As an aside, the three parts to the singular project touch on each major area of research conducted by Claude Shannon, the least known being his PhD work genetics.
I learned a lot from the current iteration of this project, but the code could be more flexible and the physical table could be more professionally constructed. With more time and resources I can develop the physical and virtual more in tandem. I had an original goal of creating a mobile unit, I am happy that I have succeeded with the portable projector and linux system. I think the current implementation is accurate and fast enough for consistent performance in long term gallery settings. However, I’d like to run more experiments with other classification algorithms to make it more and more robust.
In the future I’d like to make BitSpace responsive to more or less than 8 bits. It could be useful in walking up and down bit-lengths to cover 32 and 64 bit computers and the longer bit-lengths in public-key cryptography. In a Dynamicland fashion, I’d also like to make this project directly onto the marbles. My earlier experiments did this, but it was out of the scope of this version of the prototype. Lastly, I think this prototype is more beautiful than clear. I think it could lose some satisfying elements in exchange for being more effective at relaying some data concepts.
This process and prototype served as a solid learning experience for creating tools for learners in the future. I learned a lot through this work and intend to take it further.