Page 301 - The Black Box Book

The first part of the technical realization was carried out in a similar
way as in the case of the classification of artistic techniques. A neural
network capable of recognizing whether a particular image originates from
the Rudolphine circle was developed; – the attached graph (Figure 8)
shows the learning process here as well and the resulting estimation
accuracy rate of over 90%. The next step was to use this algorithm to
classify 1,640 paintings created between 1612 and 1812, two centuries
after Rudolph’s death, and to select the works that the program deemed
to be most consistent with the learnt rules.
At the moment, two sets were available: the first group contained
actual paintings by Rudolphine painters, the second, slightly larger, in-
cluded younger works that resembled them in unspecified ways. In total,
there were 1,000 paintings. How to examine and compare these works?
The Inception and UMAP neural networks used to cluster the images
according to formal similarities and the Yale DHLab PixPlot visualization
tool capable of drawing an interactive map with the clusters thus created
served this purpose (Pilka & Hrdličková, 2022).
Looking at the atlas, we can see an “archipelago” of images, with
the distances between each point corresponding to the degree of formal
affinity (Figure 10). The closer the reproductions are, the more similar
features they share. In some cases, the space between the icons is quite
large and empty - indicating that there are no works in the analyzed data-
set with characteristics that correspond to a specific location. Conversely,
locations where there is a high concentration of points indicate a greater
Figure 8: Training an algorithm to detect
number of visually close images. It is irrelevant if the similarity is due to the style of Rudolf II painters. The graph shows the lear-
color, brightness, contrast, proportions, composition, shapes, edges, or ning process of the neural network. The horizontal axis
surfaces, or any other phenomenon. All of these features were taken into shows the iterations, and the vertical axis shows the
accuracy that the algorithm achieves with respect
account by the neural network method and reduced to a two-dimensional to the test set. Author: Lukáš Pilka
map.
In addition to the atlas itself, the program includes navigation, which
is comprised of the indicated areas. In collaboration with art historian
Tereza Hrdličková, we attempted to choose and name such groups that can
make (at least partially) sense from a curatorial or art historical perspec-
tive. These clusters were partially created automatically using a sorting
algorithm and partially drawn on the map by “human hand.” Some of these
collections are temporally homogeneous, containing exclusively works
by Rudolphine or “post-Rudolphine” artists. In other instances (such as
the Landscape with trees group) (Figure 12), however, there is a pretty
substantial mixing of the two datasets and an obvious continuity between
the two time periods. It is precisely such clusters that may represent a type
of semi-finished product or a signpost for potential future qualitative study.
After looking at the atlas, we find that the characteristics of
the clusters are different. The relationships between the grouped works
are sometimes clear, and sometimes more ambiguous; sometimes they
contain works with similar motifs, created with the same technique,
or classified in the same genre; and sometimes these clusters share
a similar color palette, compositional style, or painterly handwriting.
300 301