Page 291 - The Black Box Book

test whether this ability could be transferred to a neural network algorithm,
and if so, to use this program to check and complete the metadata about
the artifacts – despite the fact that all of the Digital Curator data came
from museum collections, the record of art technique was available for
only a part of them. This information was completely absent from certain
collection objects, while it was present but unclear for others (curators
used terms that could not be unified during data cleaning). Additionally,
not all curators have complete trust in their data, so if the algorithm could
acquire a certain degree of confidence, it might be used to spot incon-
sistencies in records already in existence.
In this experiment, a supervised learning method was applied, where
CURATOR OF ART TECHNIQUES the resulting program can independently classify the items under inves-
tigation into predetermined categories. In this case, the “teachers”, i.e.,
Recognizing the technique of a work is one of the basic skills not only the persons preparing the sample data, were the museum staff them-
of connoisseurs of “art historical craft”, but of virtually anyone who has selves, who prepared the metadata contained in the Digital Curator
come into contact with the visual arts. Even young children in kindergarten database. This was a secondary use of their work – probably very few
learn to distinguish between drawing (crayons) and painting (watercolors), curators and collections managers anticipated that the data they were
and thus acquire the basics of this kind of classification. In the world of collecting concerning artistic techniques would be used to derive these
museum collections, however, the situation is often a little more compli- cognitive capacities.
cated. We find techniques detailed in much greater depth on labels in The first step was to select groups of art techniques and collect
gallery halls or on reproductions in exhibition catalogs, and there are even hundreds of sample reproductions from which the algorithm would ex-
hundreds of distinct types of techniques in the Digital Curator database. tract its skills. For this purpose, it was possible to use part of the existing
These range from succinct descriptions such as “woodcut” or “oil paint- database, find out which art techniques are most frequently found in it,
ing” to elaborate statements indicating the often very subtle nuances in and prepare the necessary number of study samples according to them.
the methods and materials used. For example, the technique for Gustav The machine learning dataset was composed of over 3,000 photos sorted
Klimt’s famous Kiss is described as follows: into 14 categories following numerous experiments and iterative changes.
Among them were: watercolor, woodcut, photography, pen drawing, pencil
“Figure surface: gold leaf, gold paint (gold powder dispersed in a me- drawing, charcoal drawing, etc.
dium), silver, platinum, lead, oil painting, on canvas painted with zinc During the learning process, the algorithm decomposes the sample
white: Metal (brass), painted with glazes.” (Belvedere Museum, 2022) images into individual pixels and through the changing distribution of
weights among the neurons, it looks for a configuration that can capture
The record indicating the artistic technique of the given work has the shared properties as precisely as possible. To accomplish this, it di-
a purely practical character. It serves in categorizing and cataloging vides the sample data into two groups – while the training set is used for
the collection, is used for searching and filtering, or helps in navigating learning, the test set provides information about accuracy – and each time
museum systems and online catalogs. Additionally, the process of “creat- the weights are rearranged, the program attempts to estimate the catego-
ing art objects” is the main focus of numerous theoretical and historical ries (art techniques) for the test set and then determines what the success
treatises. One need only think of the publications devoted to the impor- rate of classified images. The algorithm gradually improves as a result of
tance of oil painting in Dutch art or the invention of copperplate and its this feedback and subsequent iterations until it reaches its limit, at which
influence on South German Renaissance art. Alternatively, one can look point the learning process stops. The attached graph illustrates the pro-
up relevant keywords in the Google Scholar database and see how many gression of model improvement – while the algorithm achieved less than
results are returned, where for example, for the term ‘woodcuts and en- 30% accuracy on the first attempt (assigning the correct art technique
gravings’ the application returns 35,500 entries, for ‘oil painting’ there are to 30% of the reproductions in the test set), it reached its peak with an
374,000, and for ‘lithography and etching’ as many as 534,000. (Google accuracy of about 65% on the 20th iteration (Figure 3) In other words,
Scholar, 2022) there is a two-thirds likelihood that the algorithm will correctly attribute
To distinguish a painting from a drawing is an easy task, but it takes an art technique to an unknown reproduction if we now ask it to do so.
a trained eye and a certain level of proficiency to tell an etching from an
engraving or lithograph. The aim of the first experiment was therefore to
290 291