Exploring Art and AI in Georgia Tech's School of Literature, Media, and Communication

An-AI generated image of a woman painting
An image generated by the DALL-E stable diffusion engine. DALL-E is one of the tools Art & AI students use to exploring the boundaries of AI art.

Painting may seem old school, but it’s a profoundly technological practice. From the tech behind brushes to the mathematics of perspective, artists of all kinds have long placed themselves where science, technology, and art meet.

But it’s never been more challenging to pinpoint where a relentless ocean of technology splashes over the ever-shifting sands of artistic expression amid the rise in artificial intelligence tools such as DALL-E and ChatGPT capable of generating images, text, and even music from relatively simple text prompts.

This landscape has proven fertile ground for School of Literature, Media, and Communication (LMC) resident painter Mark Leibert, who for years has turned to algorithms and computational methods to help inspire and refine his work.

Leibert and collaborators from the Georgia Tech Research Institute (GTRI) have gone even further, recruiting a team of student researchers in the Art & AI Vertically Integrated Project (VIP) to explore the artistic implications of AI and the technological implications of art. In fact, they were well ahead of the recent public buzz over artificial intelligence tools, having launched four years ago — before DALL-E and ChatGPT became household names.

“We have a really unique situation here where we can think about how these techniques can be used for artistic means and how to use those creative neurons to approach some of these new technologies,” said Leibert, an LMC professor of the practice.


Art and Computation Central to LMC’s Mission

The Art & AI VIP is one of many arts-and-technology efforts housed entirely or partly in LMC, a unit of the Ivan Allen College of Liberal Arts. The most recent initiative is Interactive Media Arts, a partnership between the Ivan Allen College and the Georgia Tech Library, that features many LMC faculty.

Other projects include Professor Brian Magerko’s widely acclaimed EarSketch coding-through-music platform; work examining the shared DNA behind quilting and computing spearheaded by Anne Sullivan, an LMC assistant professor whose doctorate is in computer science, and Professor Janet Murray’s continued exploration of the impact of digital technology on storytelling. The School also recently added a new artist-in- residency program, in collaboration with the GT library, to further examine the nexus of art and tech. It also has invited musicians to a residency program with similar aims, that one in conjunction with the School of Music.

“LMC has always been about looking at how technology is shaping literature, art, and storytelling in general,” said Kelly Ritter, chair of the School. “The Art and AI VIP is one way we’re doing this, but whether it be a new center dedicated to exploring the nexus between media arts and research, the extraordinary work of faculty such as Brian Magerko and his team, who are helping teach young students to code, or how technology and its impact is suffused throughout our writing and communication program, this sense of interplay between technology, textual study, and media arts is central to what we do.” 

AI: ‘Solutions in Search of a Problem’

An AI-generated image of a man in a gray jacket
DALL-E's take on Leibert teaching.

DALL-E’s AI-generated take on Leibert leading a class.

The VIP draws students from across campus, many interested in exploring the creative potential of technological degrees or in scratching a creative itch they aren’t always able to satisfy amid a heavy courseload in computing or engineering.

“Tech has great resources for technology and amazing resources for literature, media, and communication. This seemed like one of the few bastions in which the two were really intermingling,” said Jude McClaren, a second-year computational media student.

Leibert formed the VIP team in 2020, initially partnering with now-retired GTRI researcher Betty Whitaker. He’s now working with Ethan Trewhitt, another GTRI researcher, who helps students dig deeper into the sometimes mystifying ways in which AI and artistic intent interact.

“AI systems, in some cases, are solutions looking for a problem, and giving them new outlets helps you recognize where the limitations of those systems are,” said Trewhitt, a senior research engineer at GTRI. “Then the creative motivations of the people using these systems can create new ways to solve problems that can then be used toward what you would consider more practical applications that aren’t necessarily art. So it’s kind of a feedback loop.” 

Exploring the Intersection of Art and AI

An abstract image with faintly depictly a cow against a patterned blue background
VIP student Diana Kim created this artwork from an acrylic painting she made depicting a literal interpretation of the poem “Two Headed Calf” by Laura Gilpin. She fed the painting and poem through an AI model with instructions to focus on key lines of the poem. “The image captures the melancholy beauty that the poem portrays in such an interesting way,” Kim wrote in a class description of the project

In its four years, students in the Art and AI VIP have explored the intersection of the two fields through various projects. In recent years, students have tried using AI to write horror, create moving portraits, animate children’s books, explore new takes on old music, creative interactive, emotion-sensing art, and tools to visualize poetry and music. 

In the Spring 2023 semester, an interdisciplinary group of students with experience in coding, illustration, 3D-modeling, computer animation, website design, marketing, and AI systems worked on creating a human-like extension to the ChatGPT AI tool.

You can see a gallery of some of the recent work they have done in a database maintained by the Art & AI VIP.

Their techniques include generative adversarial networks, diffusion models, transformer models, and natural language processing models, as seen in publicly available tools such as DALL-E, EbSynth, and ChatGPT.

“I would say the bulk of the students are working in the deep-learning space, which tends to be a little more black-box kind of functionality,” Trewhitt said. “The fun thing about this is there really isn’t such a thing as failure, so it’s safe to really play with things that can surprise you.”

While Trewhitt is still working through ways to dive deep into the technology behind such systems in a way that works for a team with a range of backgrounds, some more technical than others, some teams have surpassed his knowledge in some areas.

“We had students working on a music video project recently, and by the end of the semester, they definitely understood it better than I did. I had dabbled in it, but they had spent hours really getting it working the way they wanted, peeling back the layers of a complex system to get it to do what they wanted it to do.”

Michelle Lee, an Art & AI VIP member and second-year computer science student from New York, is working on a music visualization program. She said the VIP has allowed her to use her computing knowledge in a creative way that she hopes will help after graduation.

“I hope to bring creativity wherever I go,” Lee said. “I hope to work at a tech company, and I’d really like to bring this kind of creativity to make my work different and unique.” 

Thinking About the Ethics of AI and Art

Tools such as DALL-E and ChatGPT have made headlines in recent months. The coverage ranges from astonishment at the tools’ ability to render true-to-form, if sometimes absurd, images or text to outright alarm at the potential the tools have to deceive or perhaps even put artists and other creatives out of work.

It’s a topic of discussion in class, of course. Leibert, Trewhitt, and the students have discussed how emergent technologies, such as the AI tools used in class, can scramble methodologies, disciplines, and careers. They’ve also discussed how working with art creates opportunities to be nimble, reframe conventional thinking, and surface novel ways to solve problems. Leibert says that can, in turn, lead to the creation of new tools or even new careers. Case in point: the rising demand for people skilled in “prompt engineering” — in other words, able to coax quality results from a tool that can sometimes be mysterious and even erratic.

That kind of work is something creative people have done for a long time, Leibert said.

“Artists and writers have worked with generative prompts and games for much of the last 100 years,” he said. “Sometimes it has been the flip of a coin, consulting I Ching, or selecting from a deck of prompt cards. Tools like Photoshop and Illustrator have been a part of my process for at least two decades, and working with this group has increased my appreciation for the work computer scientists do.”

Ethical issues also come up, of course, such as who is doing the creating — the human, the machine, or some mixture of the two — and who can claim ownership over the resulting works.

“This is a very new and fast-moving field, so we spend a lot of time talking about the technologies and the creative and conceptual potential of AI in art. But the nice thing about what we are doing is that we are able to have these conversations about ethics in an informal manner as we work, so this kind of ethical thinking becomes as much a part of what we do as the technical inputs or the artistic outputs.”

Students in the VIP are well aware of the ethical issues and committed to using AI for good. And many view the use of AI to make or remix art as natural, just another tool in an artists’ toolkit.

“AI is a legitimate way of making art,” said Marin Hyatt, a third-year computer science student from New York. “I think, like anything else, AI is just a tool, similar to Photoshop. It’s just a different way to make art.” 

Interested in More?

The Art and AI VIP isn't the only project examining art and computation in LMC, of course. The School features more than a dozen centers and labs exploring various aspects of art and storytelling. If storytelling is your thing, check out the Prototyping eNarrative Lab and the work visionary director Janet Murray does thinking about digital storytelling. If you’re fascinated with the idea of bridging real and digital spaces, visit the Digital World and Image Group, led by Associate Professor Michael Nitsche. You also can learn about the nexus of design and social justice, the interplay of music and technology, and more.

Other AI Examples

Blurb / Gallery Set

Interactive Media Arts

Interactive Media Arts is a new center housed in the Ivan Allen College of Liberal Arts that seeks to enhance artistic expression on campus, preparing students for careers as creative makers and innovators, and helping faculty develop creative approaches to improving scientific and technological research and outreach.

The center is brand new, but plans include an incubator for creative scholarship where artists and researchers can work together. The goal is to help researchers think creatively about public engagement and how art can help the public understand the work of scientists, engineers, and technologists.

The center faculty also hope to create opportunities for more interdisciplinary research in which media arts can drive public engagement, understanding of, and action on complex issues.  

To learn more about Interactive Media Arts, visit its website.

Expressive Machinery Lab

The Expressive Machinery Lab, led by Professor Brian Magerko, investigates how understanding human creativity can enhance technological experiences, enabling expression and meaning-making with computing.

Research areas include improvisation with computers, using expression as motivation for learning computing, measuring creative social interactions, and engaging learners through embodied interactions. Applications range from AI-based digital performance to tangible interfaces for education.

The lab's current projects include the NSF-funded EarSketch platform, which uses computational remixing of hip-hop to engage underrepresented populations in coding education, and LuminAI, an interactive installation that invites "a human interactor / performer to co-exist in the same space as an artificially intelligent (AI), computer-controlled performer." Magerko's current research on LuminAI will eventually result in a live performance featuring dance students at Kennesaw State University performing with the AI.

To learn more, visit the Expressive Machinery Lab website.

Code Crafters

Code Crafters is a research project that explores the connections between quilting and computational thinking. By developing instructional workshops for adult quilters, the project leverages existing social structures, skill sets, and engagement to improve computer science literacy.

Led by Anne Sullivan, an LMC faculty member whose doctorate is in computer science, Code Crafters seeks to broaden public awareness of computational thinking. The project focuses on reaching a diverse population, including women from various backgrounds, to broaden participation in computing and enhance public engagement in computational thinking.

AvantSouth at Georgia Tech

Georgia Tech is bringing together the finest minds and voices to explore artificial intelligence — the opportunities, the risks, and above all the ethical and responsible stewardship of AI. To see our presenters and register to attend Avant South on Sept. 28 – 29, 2023, visit avantsouth.com.