Page 38 - The Black Box Book

AI AS A CURATOR
Experiments with the use of AI in the analysis of digitized art collections
show that AI software for image recognition is not universally applica-
ble but requires training software on a suitable dataset for the stated
goals. Emilly L. Spratt discussed the possibilities of applying AI as a cu-
rator from the perspective of art historians in a paper Dream Formula-
tions and Deep Neural Networks: Humanistic Themes in the Iconology of
the Machine-Learned Image published in 2017. (Spratt, 2017) She
expressed her belief that using machine learning for image analysis “can
engage the humanities, complement existing socio-cultural theories, and
offer the possibility of new methodologies for image analysis that take
cognitive psychology into consideration.“ (Spratt, 2017, p. 2) She com-
plemented her claims with a description of her own experiment using
machine learning to analyze a digitized art collection. She took software
available on the open-source cloud platform called Grad_CAM (Gradi-
ent-weighted Class Activation Mapping) by Perception Lab at Georgia
Tech and applied it to selected images from the collection of the Metro-
politan Museum of Art. (Spratt, 2017, p. 4) The results of the experiment
were quite ambivalent. The Grad-CAM, trained on contemporary images
available online (ImageNet), made mistakes (albeit interesting ones) when
applied to medieval paintings or even photographs from the beginning of
the 20th century. In the example of Georgia O’Keeffe, photographed by
Alfred Stieglitz, the software interpreted the image as “a man holding a cell
phone in his hand.” (Ibid., p. 4) This is one of many pieces of evidence that
the AI software is highly biased according to the data set of images used
for its training, and it is, therefore, necessary to take this into account
when designing such projects (see the project by Lukáš Spilka Digital
Curator, 2022). Despite initial unease about the results of applying AI as
a curator of art collections, Spratt and some other researchers believe
that the use of AI in art iconography opens up a new evolutionary phase
of the discipline, which Spratt called “the iconology of the age of artificial
intelligence.” (Spratt, 2017, p. 12)
The idea of AI as an artificial curator has intrigued not only art his-
torians eager for a technically supported understanding of the content
of vast collections of traditional art but also artists and curators testing
the potential of digital media. Several notable projects have emerged in which
AI software is used in the role of curator. For example, Amy Alexander has
created a still-running project, an endless document, What the Ro-
bot Saw (*2019 - ), involving AI software which automatically searches
YouTube content. “A social media AI turned documentary filmmaker,
the Robot continuously makes its way through the world of low engage-
ment online video, carefully organizing and describing the people and
scenes it features in its documentary. The film is constantly curated,
edited, titled and archived algorithmically from among the least viewed
and subscribed YouTube videos uploaded over the past several hours.”
(Alexander, 2022) The resulting video not only showcases social network-
ing outsiders, whose only audience would typically be an AI software but
38
also reflects the massive use of AI as “engagement algorithms” and as
a tool for automated surveillance of social network users by reversing
the function of the algorithm.
Joasia Krysa is the initiator of two experiments with AI (machine
learning) as a curator, entitled The Next Biennial Should be Curated by
a Machine: B (NSCAM) and AI-TNB, which were put forward as a part
3
of the Liverpool Biennial and in collaboration with the Whitney Museum
of American Art. In the first project – ‘(NSCAM)’ – the creator team, led
by Christiane Paul, focused on generating an exhibition concept using
AI trained on a dataset made up of the content of the Liverpool Biennial
and The Whitney Museum of American Art’s archives of curatorial texts:
“Processing datasets (including curatorial texts) linguistically and semiot-
ically, the AI system ‘ learns’ their style and content, breaking and mixing
them together. The generated texts are then presented to the user, with
a degree of interactivity and ‘ branching’, iteratively rewriting small parts
of its own text at random.” (Krysa, Impett, 2021) 2
In the second project - ‘AI-TNB’ - Joasia Krysa and her team fo-
cused on using AI as a shadow curator for the current Liverpool Biennial
exhibition of spring 2021, curated by Manuela Moscoso. The AI curator
created a parallel online exhibition of works displayed physically in differ-
ent locations in Liverpool: “The resulting ‘curatorial AI system’, or an AI
Biennial, is an exercise in interaction through large datasets, using com-
puter vision and natural language processing techniques with a focus on
human-machine co-authorship.” (Krysa, Impett, 2021)
These experiments aim to explore the potential of AI as a new actor
in digital curating and creativity, to try to transfer curatorial ‘know-how’ to
AI through machine learning, and to explore the possibilities of extending
current curatorial practice.
CROWDSOURCED BEAUTY OF AI ART
However, AI software trained by machine learning may also be able to
generate images similar to those whose common features it has ‘learned’
to recognize. The AI-assisted image generation was initially presented as
‘dreaming’ or ‘hallucinating’ of the machine system. The surreal nature of
these images was emphasized by the visual language of the AI’s generative
lever outputs (Deep Dream Generator, 2015). In the next stage, families
of images were generated through AI, whose visuals played with the lim-
inal form between impressionistic blurring of contours and the transpar-
ency of layers of imagery from which the human face usually emerged.
(Edmond de Belamy from La Famille de Belamy by the artistic collective
Obvious, 2017) However, the latest tools generate images in any known
art style (Midjourney, 2022), perfectly imitating human work. These are,
however, the outputs of a combinatorial game at the level of content and
form, i.e., a form of ‘remakes’ or ‘remixes’ of digital imprints of products
2 This project is described in this book in a chapter written by one of the
members of the creative collective going by the name of UBERMORGEN.
39