A field guide to AI systems

May 15, 2023

Studying artificial intelligence (AI), but in the way the biologists of yore did with the exotic species they found on their expeditions. This is what TU/e researchers Pei-Ying Lin and Janet Yi-Ching Huang are planning to do with the 10,000- euro Mingler Scholarship they managed to secure.

The Mingler Grant, awarded by the Society of Arts and The Young Academy, is intended for projects on the interface between science and art.

The project is called ‘Field Guide to the Artificial Intelligence Bestiary’ and will result in an interactive field guide, inspired by the historical bestiaries: richly illustrated volumes describing both existing and fictional animal species. However, this new guide will not describe mythological creatures, but their contemporary – and just as mysterious and fascinating – counterparts: AI systems such as Chat GPT, DALL-E and Midjourney.

We can’t help but chuckle at many of the theories used by scientists in the past to explain things they couldn’t wrap their heads around yet. Pei-Ying Lin (1986) is fascinated by these stories from scientific history. Gods, ghosts and monsters were held responsible for all kinds of things. Diseases were explained based on a disbalance of fluids in our bodies and astronomers had the sun revolving around Earth – and possibly a flat one at that.

But it’s easy to judge based on contemporary knowledge; people didn’t know any better back then. “And we’re at the same point when it comes to AI systems,” Lin and Janet Yi-Ching Huang (1983) assert. Which sounds odd, seeing how we humans developed artificial intelligence ourselves. Huang: “We get how algorithms work, but don’t understand the reasoning ability these generate. AI systems are black boxes.”

AI and biology

In addition to being an artist, Lin is a PhD candidate at IE&IS, where she works on the multidisciplinary project Creative AI Machines. Huang has a computer science background and is an assistant professor at Industrial Design. “We’re both from Taiwan, but we met in Eindhoven. It turned out we had many mutual Taiwanese friends.”

Janet Huang (left) and Pei-Ying Lin. Photo: Holly Krueger

Huang: “I often help artists with the technical side of their digital art projects, and Pei knows many of them from the art world.” Huang is also the one delivering the technical knowledge to the Mingler Scholarship project, Lin says: “She has a better grasp of the workings of AI than I do.”

The two got to talking, which led to inspiring conversations about AI, thanks in part to the other perspective Lin contributed – not just as an artist, but also owing to her background in biology: she holds a bachelor in life science.

Huang: “Pei was able to identify unexpected but highly accurate parallels between the workings of AI and, for example, genetics and ecosystems. Those are still very different disciplines, of course, but using terms and principles from biology makes it easier to talk about AI.”


Such a vocabulary that is helpful in conversations about AI isn’t an unnecessary luxury, Lin and Huang say. “For the moment, discussions about AI are mainly held in academia, while the systems influence everyone’s lives. So the debate should shift to society at large and that’s what we want to help bring about with our project: we want a very wide range of groups to chime in.”

As it concerns a project that touches upon art, the two researchers feel like they have space to talk about AI systems ‘as biologists’, even though that might not make a whole lot of scientific sense. “Art can be a bit messy. And it can involve some phantasy,” Lin says.

How does she see herself: as a scientist or as an artist? Or maybe as a designer – for the Virophilia art project she created dishes based on viruses, for instance. “I wear several hats, and for this project I’ve donned the artist one, which means I will deliberately ignore all scientific literature,” she says with a laugh.


Nonetheless, art plays a serious part in the project, Lin emphasizes. Often art is only used as a tool to communicate scientific insights. With the bestiary project this is different, as art contributes to the acquisition of those insights. It is exactly because we don’t have a firm grasp on the workings of AI yet that the eye of the artist can make such a contribution. 

Huang: “Imagine a number of points making up a graph. A scientist will look for the line that connects them all, whereas an artist might think: that one big anomaly, what kind of measurement result is that? As we don’t know yet which data are significant and which aren’t, not looking for the connecting line right away may yield interesting input.”

In print

The field guide for AI systems is due to be published in March of next year. A lot needs to be happen before then, Lin and Huang know. “The mere task of defining the ‘creatures’ to observe is already complex.” After all, the way AI systems such as Chat GPT work is influenced by the data files available to them and by the input of users. “In studying an AI system, should we look at the code, at the code including the data file, or at the code including the data and the user community?”

There are also challenges in a methodological sense, Lin says. “We will be ‘interviewing’ AI systems, but we have to try to do so without influencing them with our subconscious intentions.” They want to safeguard objectivity by inviting a great many groups – including international ones – to collaborate.

The two researchers are still on the fence about paper versus digital. Lin’s dream is to have a gorgeous field guide in print, but: “Paper has its limitations when it comes to displaying complex matters such as the connections in the AI ‘ecosystem’. And another advantage to a digital guide is the possibility to interact with the public.”


Given their experiences so far, how do the two researchers view the current discussion on the risks AI poses? Will AI save or destroy humanity? Doesn’t their approach – equating the AI system to a dodo or unicorn – portray too innocent a picture of artificial intelligence?

Huang: “That binary way of looking at things – utopia versus dystopia – is especially useful for theoretical discussions. There’s an entire range of possibilities in between, which interest us much more.”

Lin makes a comparison with biology. “Thinking based on two extremes – life (or survival) and death – has something Darwinian about it, but biologists have moved away from those views. We now know that survival of the fittest isn’t the whole story. Life always finds a way. Take the first life forms on earth: oxygen was lethal to them, but now it’s a precondition to life. That’s exactly how it will go with AI.”


This article was previously published by TU/e Cursor