I’m fascinated by systems where material is transformed from one domain to another in order for something unique to happen. This kind of transformation is a fundamental part of all digital media and electronic arts, but we generally lack a cohesive and shared understanding of how these changes in domain or material state impact art processes. Trying to articulate why this feels important is surprisingly difficult, so rather than try to discuss directly, we’ll look at some specific examples of what I’m talking about.
Seth Brundle of The Fly (1986) develops a novel way to beam objects from one telepod to another, but the transformation between these different material states wreaks havoc on living tissues. This is demonstrated in a typically Cronenburg way with a horribly mutilated teleported baboon. Eventually, in a moment of postcoital inspiration, he fixes the bug in his teleporter and manages to teleport a living baboon (without turning it inside out). His realization was that the computer needed to be programmed for creativity when dealing with living beings. Things get wacky when he gets sad-drunk and beams himself (along with a housefly). In their natural state, Seth and the housefly are discrete beings, and the idea of them combining together in their natural state is inconceivable. Seth’s incredible metamorphosis requires first transforming his matter into another domain that allows for recombination and then transforming back into a standard human material form. While teleporting, the fly and Seth are just signals.
Signals
You are likely familiar with the classic reverb effect in audio. Reverb is a wonderful example of a multi-domain system that gets around the limits of analog circuitry. Most analog audio processes are some form of amplifier, filter, or distortion that uses electrical components to reshape the signal. Reverbs transform an incoming audio signal into vibrations across some material - usually a spring coil or a metal plate - and then picks up those vibrations with a transducer and converts it back into an analog electrical signal. The resulting sound mimics an enclosed acoustic space around the original sound, which has a wonderful smoothing effect on instrumental sound that appeals to our ears. By transforming the audio signal into physical vibrations, we are able to alter the signal in ways that would be much harder to achieve through purely electronic means.
Drawings
Andre Masson championed a Surrealist approach to drawing called Automatism which sought to draw out the unconscious through the act of drawing without conscious aims. In Masson’s drawings, we find emergent forms that coalesce from the automatic movement of the hand, which show a mental process of thinking on paper and free associating.
When an artist picks up a pencil to sketch, they are able to transform several thoughts and whims into marks on a page through tiny movements of fingertips that hold the pencil to paper. Going from mental abstractions to concrete contours and forms allows for the artist to work with those ideas as image material, enabling different tools and processes to shape them that are not available in the abstract space of the mind. Drawing is thus a way of processing a thought through a different material with its own emergent properties, limits, and translation artifacts.
Going further, there are many other ways to alter this image in its present material domain. You could erase the marks, add an addition medium to the page like gouache, or even cut up or mangle the paper itself. You could paste pictures from a magazine on top. An artist could also translate the drawn idea to another medium, like paint on a canvas. Each medium offers new ways to work on what was once just an idea.
Lasers and Monsters
From the very early days of cinema, filmmakers used to make extensive use of a wonderfully baroque device called an optical printer. Optical printers worked by projecting a frame of film onto a surface that is then rephotographed by a syncronized film camera. Because the film image is projected, the VFX artist can perform optical manipulations on the image like zooming or unfocussing the lens, or adding a color filter. In the case of old sci-fi movies, laser blasts could be added to a scene by painting them onto a piece of acetate layered over the image. This physically altered or layered image is then photographed by a film camera.
The Psyche
Carl Jung theorized the Transcendent Function, which involves the transformation of unconscious thoughts into conscious ones. By transforming the unconscious material into conscious thought, the individual can then work towards reintegrating them and becoming whole. One of the ways this works is through symbols reveals in dreams. An individual has dreams in which symbols are shown that allow for unconscious material to become conscious, thus allowing for self reflection and conscious thought processes to manipulate that material and incorporate it into their self-image.
"Once the unconscious content has been given form and the meaning of the formulation is understood, the question arises as to how the ego will relate to this position, and how the ego and the unconscious are to come to terms. This is the second and more important stage of the procedure, the bringing together of opposites for the production of a third: the transcendent function. At this stage it is no longer the unconscious that takes the lead, but the ego." (The Transcendent Function, CW 8, par. 181)
In general, we can also think of discourse, writing, and other forms of communication to be a way of transforming that which is internal - conscious or unconscious - into something that can be worked with externally using the tools of language. When a thought becomes language, we gain a different set of tools and resources to manipulate that thought, including the interpretations of that language by others.
Reality and Computers
The border between reality as we perceive it and the internal workings of a computer is deceptively complex, especially considering how embedded our lives have become with smartphones and laptops. In broad terms, the translation between these two domains involves several steps:
Sensors, transducers, and input components convert physical properties or energy into analog electronic signals that mimic the environment.
Analog to Digital Conversion - discrete samples of the analog signal are taken at precise time intervals, and are represented as numbers
Interpretation/Synthesis - The numbers from one or several sensor readings are combined together to begin forming a digital representation of reality
The benefit of this transformation is, of course, that digital representation is a powerfully mutable abstraction. Working with an image as a grid of color data allows for specific manipulations to occur that would be much more difficult to achieve using traditional media. It’s also not completely without imperfections and artifacts based on the limitations of that transformation.
Spectra
We tend to think of digital color as a grid of RGB (red, green, blue) color data, which offers many benefits, chiefly that it mimics how light works optically. If all three values are 0, then the pixel is black. If all three values are 255 (8-bit values), then the pixel is white. You can also think of these values as coordinates of a 3D space XYZ, which is how LUT (lookup table) color grading works. Via a transformation, you can also convert this RGB data into HSL (hue, saturation, lightness), which allows you to shift the hue of the color value, or the saturation, without affecting the other perceptual qualities of the color. There are even more color spaces that you can transform between, each of which are optimized for specific operations and perceptual qualities.
In the audio world, it is fairly popular to use something called a Fourier Transformation (FFT) to convert the familiar signal domain waveform into a deconstructed array of sine waves that combine together to approximate the incoming audio, also known as the spectral or frequency domain. The frequency domain is useful for audio analysis, since you can track the energy in specific frequency bands and describe the audio in a variety of ways. It also makes it possible to achieve precise filtering as well as a number of unusual audio effects - even combining spectral qualities of two audio sources like an auditory version of The Fly. The transformation between signal and frequency domains also tends to mark the sound with a signature “spectral sound” due to limitations in the algorithm.
Lost (and found) in Translation
Most meaningful translations between material domains will inevitably result in some loss of fidelity to the original signal, and many of these will impart a kind of signature onto the material. If our goal was to broadcast a perfect representation and to have that remain consistent, then we would describe these artifacts as degradation of the signal or glitches. In most cases, we accept these imperfections because we value the other results and affordances offered by transforming material into another domain.
As many have discussed elsewhere, we can also embrace this entropy or signal degradation as aesthetic characteristic of the process, in the way that Japanese potters appreciate minor imperfections that show the craftperson’s hand. The core premise of Glitch Art, at least in theory, is a celebration of the artifacts that arise as we manipulate and transform digital material across different abstract domains. Another approach is to consider that every transit into another space is an opportunity to create something new, to liberate aspects of that material, and to enter into a new relationship with it. Often a solution to a seemingly impossible problem can be found by simply transforming it into a different domain.