Multimodality
Infinite Communication Between Digital Entities – Part 2
James Woodfill Code Practice Cart, mixed media, 2019
In this swiftly changing world, where each day unfurls new advancements, artists and writers stand at a crossroads, pondering their future amidst the rise of AI-driven creativity. As technology empowers anyone to construct novels or images, painters and authors alike are left to wonder about the sustainability of their craft. This era of rapid technological evolution brings forth poignant questions about the survival and transformation of traditional artistic and literary pursuits, casting a shadow of uncertainty over the once-familiar landscape of creativity.
The air is thick with anticipation – and for many of us it feels uncomfortable. The question that keeps us up at night – has humanity unwittingly ceded its imagination to silicon? – doesn’t seem to have many reassuring answers. Given the explosive success of ChatGPT in 2023, many people assume AI has somehow grown as powerful as a magic beanstalk. So powerful that it threatens our collective soul. As Stephen Wolfram has noted, perhaps the real conundrum is that “…writing an essay turns out to be a “computationally shallower” problem than we thought.”
Whatever the aftermath of this widespread adoption of automatons, many of us will no doubt be cradling our wounded pride like limp Albatross wings in our arms. And yet, through the inevitable storm and corresponding gloom, innovation sits on the horizon. Revolutions in thought always beckon the curious forward. As many are bemoaning the end of traditional art as we know it, rather ironically, it seems curious artists are the best equipped to deal with this terra incognita.
Part of our Joseph Nease Gallery’s 5 Artists talk in October, 2020: AI-savvy artists Kathy McTavish and David Bowen discuss the power of setting up landscapes in their work.
The reality is that today’s contemporary artists have been working in some capacity, either directly or indirectly, with both multimodality and artificial intelligence for decades. Whether or not Generative Adversarial Networks (GANs) like Midjourney can create paintings that are as good as any artist is completely besides the point. Art has always been about something far more limitless and perspective driven than any one medium can encapsulate.
The uncomfortable truth pertaining to the rapid advancement of AI is that it’s ultimately an issue of defining what it is to be human. If our creative endeavors continue to prove “computationally shallower” than we realized, then what does that say about us?
We should forgive ourselves for concluding that the most human trait is not to reason, but to question. You might say to create, but what is creation if not a desire to know? To be curious. This is far less satisfying than assuming we have the capability to know in any sort of superior sense, but certainly less constraining. And right now, we could do with less constraints.
How do we maintain our infinite curiosity in the face of infinite computing power? The obvious answer is that we weaponize our curiosity. We don’t assume that we know everything about everything. We realize that knowledge is specific to the observer. The computational irreducibility of life cannot be leapt over by artificial intelligence – our lives still must be lived. Our perspectives must be experienced. Our art must be created.
What is art, if not the infinite creation of interpretation? This, it turns out, is precisely the sort of thing multimodalities help us with. If art is the boundless creation of interpretation, then this inherently human endeavor is vividly mirrored in the concept of multimodality. This idea serves a dual purpose:
a) Technically, it harnesses diverse data types - text, images, audio, video - to enrich information retrieval and interpret user queries with greater contextual depth.
b) More broadly, it signifies the fusion of varied philosophical outlooks, weaving a complex array of potential realities, each distinct in its foundational principles.
In the digital realm, file formats act as modalities, with our capacity to ascribe meaning to them being limitless. Generative AI further amplifies this dynamic, expanding the horizons of how we perceive and interact with these diverse modalities. Thus, the computationally shallow becomes the meaningfully infinite.
William Carlos Williams once wrote:
“so much depends
upon
a red wheel
barrow
glazed with rain
water
beside the white
chickens”
And artificial intelligence is no exception to Williams’ observation. The context is what matters. As the wave function collapses into a single meaning, we mustn’t lose sight of the fact that it could have been a blue wheelbarrow, in a snow storm, next to a group of half-frozen ducks. So much depends…
What we direct AI to accomplish, even if we don’t always have direct control over how it generates it, defines the meaning of the outcome. This is not the death of meaning-making – it’s the adolescent stage of a new renaissance that is already proving to be more charged with creative energy than any preceding it. Amid a backdrop of predictions about jobs being taken through automation, robbing people of purpose, and dwindling economies, artists as leaders are suddenly more essential than they’ve ever been before.
Artist David Bowen talks with curator Christopher Atkins about Bowen's exhibition, on water, at Joseph Nease Gallery. Utilizing robotics, custom software, sensors, and data, Bowen constructs devices and situations that are set in motion to interface with the physical and virtual world. These devices and situations create a dissonance that leads to an incalculable, changeable situation, resulting in unpredictable outcomes. The phenomenological outputs are collaborations between the natural form or function, or between the mechanism and the artist.
Contemporary artists understand multimodality better than anyone else so it seems a logical conclusion, even if overlooked, that they’re going to pave the way for humans through this artificial madness. One doesn’t need to look far and wide to find this is the case.
As an example, from July to September of 2023, Joseph Nease Gallery exhibited artist Sara Pajunen’s solo exhibit, “Listening through context”.
“HOW TO BUILD THE FUTURE”, A pre-war mining inspection film tells stories of hierarchy in gender and race, regional and national priorities, working conditions and methods, social protocol and more.
This piece began with a field recording that captures a train carrying taconite pellets adjusting its position on an elevated ore dock (CN Docks, West Duluth). Using contact microphones to record the resonance of the train through metal, Pajunen allowed this recording to dictate the composition: placing herself as composer and instrumentalist in a supporting role. The aim is to force the ear to question our trained anthropocentric sonic focus on music and human-made instruments.
Trained as a violinist, Pajunen’s work is a rich tapestry of field recordings, drone imagery, and archival material. Her projects, such as “Mine Songs: Sounding an Altered Landscape” and “The Places We Know,” showcase her ability to weave together diverse file formats to explore themes like environmental connection and cultural narratives. This multidimensional approach reflects a profound grasp of how different sensory experiences can combine to create a more immersive and meaningful artistic expression. This is the model for the new search experiences being developed by OpenAI, Microsoft, and Google.
Could we possibly envision a better guide through the future of media if we wanted to? And Pajunen is by no means alone. The gallery has had the privilege of showcasing the generative art (long before generative AI was in the zeitgeist) of Kathy McTavish. As a media composer, cellist, and installation artist, McTavish blends data, text, code, sound, and visual media to create immersive, generative environments (something which suddenly seems far ahead of its time).
Kathy McTavish, "z3r0" installation
McTavish’s background in mathematics, ecology, and music theory informs her approach to creating dynamic, multi-layered experiences. Her work transcends traditional gallery spaces, engaging with various physical structures and their inherent textures and sounds. Her installations, including her innovative use of a computer-controlled quilting machine, showcase her ability to interweave digital and physical realms, inviting deep reflection and transformation.
Or how about David Bowen’s recent installation, “outsourced narcissism,” which exemplifies a profound grasp of robotics, AI, and the philosophical ramifications of social media. His setup, featuring a computer-controlled robotic arm with a camera and a mirror, uses a neural network trained to recognize the robot itself.
David Bowen outsourced narcissism, computer, robotic arm with a camera attached, mirror, monitor, cables
This self-identification process, marked by a high certainty level, triggers the robot to post selfies on Instagram, showcasing a unique interplay of technology and self-awareness. The continual retraining of the AI with its own images creates a feedback loop, enhancing the robot's proficiency in self-recognition and social media engagement, embodying a cutting-edge fusion of art, technology, and identity.
Every artist represented by the gallery has something profound to contribute to our forward movement through an automated future. That word ‘automated’…it sounds corrupted. Or corruptible. Almost as though it means ‘anti-human’. But that’s merely because we haven’t been paying attention to how artists have been using automation in their work.
Art's evolution towards greater conceptualization is influenced by technological advancements. Since the Industrial Revolution, tools like the photograph and phonograph have shifted art from mere representation to exploring deeper meanings. Technologies like cinema, television, and digital media further this trend, enabling artists to experiment with time, motion, and virtual realities. The advent of the internet and AI technologies, generative algorithms, and virtual reality have all expanded the canvas, allowing artists to explore and express complex concepts like identity, existence, and the human condition in more abstract, immersive ways. Reflecting the ever-changing relationship between technology and human creativity.