Page 290 - The Black Box Book

test whether this ability could be transferred to a neural network algorithm,
CURATOR OF ART TECHNIQUES
Recognizing the technique of a work is one of the basic skills not only
of connoisseurs of “art historical craft”, but of virtually anyone who has
come into contact with the visual arts. Even young children in kindergarten
learn to distinguish between drawing (crayons) and painting (watercolors),
and thus acquire the basics of this kind of classification. In the world of
museum collections, however, the situation is often a little more compli-
cated. We find techniques detailed in much greater depth on labels in
gallery halls or on reproductions in exhibition catalogs, and there are even
hundreds of distinct types of techniques in the Digital Curator database.
These range from succinct descriptions such as “woodcut” or “oil paint-
ing” to elaborate statements indicating the often very subtle nuances in
the methods and materials used. For example, the technique for Gustav
Klimt’s famous Kiss is described as follows:
“Figure surface: gold leaf, gold paint (gold powder dispersed in a me-
dium), silver, platinum, lead, oil painting, on canvas painted with zinc
white: Metal (brass), painted with glazes.” (Belvedere Museum, 2022)
The record indicating the artistic technique of the given work has
a purely practical character. It serves in categorizing and cataloging
the collection, is used for searching and filtering, or helps in navigating
museum systems and online catalogs. Additionally, the process of “creat-
ing art objects” is the main focus of numerous theoretical and historical
treatises. One need only think of the publications devoted to the impor-
tance of oil painting in Dutch art or the invention of copperplate and its
influence on South German Renaissance art. Alternatively, one can look
up relevant keywords in the Google Scholar database and see how many
results are returned, where for example, for the term ‘woodcuts and en-
gravings’ the application returns 35,500 entries, for ‘oil painting’ there are
374,000, and for ‘lithography and etching’ as many as 534,000. (Google
Scholar, 2022)
To distinguish a painting from a drawing is an easy task, but it takes
a trained eye and a certain level of proficiency to tell an etching from an
engraving or lithograph. The aim of the first experiment was therefore to
290
and if so, to use this program to check and complete the metadata about
the artifacts – despite the fact that all of the Digital Curator data came
from museum collections, the record of art technique was available for
only a part of them. This information was completely absent from certain
collection objects, while it was present but unclear for others (curators
used terms that could not be unified during data cleaning). Additionally,
not all curators have complete trust in their data, so if the algorithm could
acquire a certain degree of confidence, it might be used to spot incon-
sistencies in records already in existence.
In this experiment, a supervised learning method was applied, where
the resulting program can independently classify the items under inves-
tigation into predetermined categories. In this case, the “teachers”, i.e.,
the persons preparing the sample data, were the museum staff them-
selves, who prepared the metadata contained in the Digital Curator
database. This was a secondary use of their work – probably very few
curators and collections managers anticipated that the data they were
collecting concerning artistic techniques would be used to derive these
cognitive capacities.
The first step was to select groups of art techniques and collect
hundreds of sample reproductions from which the algorithm would ex-
tract its skills. For this purpose, it was possible to use part of the existing
database, find out which art techniques are most frequently found in it,
and prepare the necessary number of study samples according to them.
The machine learning dataset was composed of over 3,000 photos sorted
into 14 categories following numerous experiments and iterative changes.
Among them were: watercolor, woodcut, photography, pen drawing, pencil
drawing, charcoal drawing, etc.
During the learning process, the algorithm decomposes the sample
images into individual pixels and through the changing distribution of
weights among the neurons, it looks for a configuration that can capture
the shared properties as precisely as possible. To accomplish this, it di-
vides the sample data into two groups – while the training set is used for
learning, the test set provides information about accuracy – and each time
the weights are rearranged, the program attempts to estimate the catego-
ries (art techniques) for the test set and then determines what the success
rate of classified images. The algorithm gradually improves as a result of
this feedback and subsequent iterations until it reaches its limit, at which
point the learning process stops. The attached graph illustrates the pro-
gression of model improvement – while the algorithm achieved less than
30% accuracy on the first attempt (assigning the correct art technique
to 30% of the reproductions in the test set), it reached its peak with an
accuracy of about 65% on the 20th iteration (Figure 3) In other words,
there is a two-thirds likelihood that the algorithm will correctly attribute
an art technique to an unknown reproduction if we now ask it to do so.
291