Self-Improving Robotic Brushstroke Replication

Abstract

Painting robots, like e-David, are currently unable to create precise strokes in their paintings. We present a method to analyse given brushstrokes and extract their trajectory and width using a brush behaviour model and photographs of strokes painted by humans. Within the process, the robot experiments autonomously with different brush trajectories to improve the reproduction results, which are precise within a few millimetres for strokes up to 100 millimetres length. The method can be generalised to other robotic tasks with imprecise tools and visible results, like polishing or milling.A cam is a rotating disk, on which a follower rests. The changing radius of the cam moves the follower up and down, thus translating rotational to linear motion. This is for example used in combustion engines, where a camshaft opens and closes the fuel valve.Arts 2018, 7, 84 3 of 28 one revolution. The followers then transfer the motion through amplification mechanisms to the end effectors Droz (2014).The "Musician" is a machine that plays a functional miniaturised organ by pushing the instrument's keys. The "Writer" produces a 40 letter text by reading each letter from a reconfigurable wheel and using an ink-quill to write on a sheet of paper Mahn (2013), Schaffer et al. (2013). This automaton is interesting as it represents one of the early programmable machines and introduces the concept of encoding information in shaped metal plates-a process that was later used to store sound information in vinyl records.The "Artist" is capable of producing four different paintings: a portrait of King Louis XV; the couple Marie Antoinette and Louis XIV; a dog with "Mon toutou" written next to it; and a scene of a butterfly pulling a chariot Mahn (2013). Each scene consists of many lines drawn with a pencil that the automaton holds in its hand. These sketches are literally hard-coded into the metal cams inside the machine and cannot be reconfigured easily. The automaton and two of its stored drawings can be seen in Figure 1. Furthermore, the painter periodically blows away the graphite dust left behind by its pencil using an integrated air pump. This feature is not found in any contemporary painting robot, despite this being potentially useful for selectively drying paint Schaffer et al. (2013). Stroke Reproduction ResultsAn application of stroke reproduction is copying human writing, as shown in Figure 9. The example presented here was performed by the robot using only its optical systems. The prototype writing was photographed, each individual stroke was extracted and analyzed with the method described in Section 4.4. Additionally, the distance between strokes was preserved. Then the robot was able to write the presented text on a separate sheet of paper. In principle every type of writing or sketch can be reproduced with this method, as long as no self-overlapping strokes are used.for the deformed brush lagging behind the intended trajectory. For example, in areas of high curvature 417 in the prototype stroke, the first attempt will common...

Recent Developments Regarding Painting Robots for Research in Automatic Painting, Artificial Creativity, and Machine Learning

“…Machine Learning Groundwork: Through the enhanced data collection methods we plan to build a corpus of brushstrokes, which consist of pairs of trajectories and resulting strokes. Using this dataset we intend to improve upon previous efforts at static stroke analysis [2] and use state-of-the-art machine learning approaches to allow the system to continuously improve its painting abilities on a technique level.…”

“…Apart from e-David, a multitude of other painting machines are currently used in the art world and in research. Historical examples have been discussed extensively in [2], thus in the following we only list contemporary examples.…”

“…The old Reis RV60 has been replaced with an ABB IRB 1660ID and modifications have been made to tooling and workholding, which are discussed in detail in Section 3.3. A more in-depth description of e-David and its purpose can be found in a previous paper [2]. So far the main focus of e-David was on painting techniques based on the Hertzmann Algorithm [3], with improvements which allowed the use of visual feedback, limited palettes of colors, and stroke placement optimization.…”

“…An explicit conversion from pixel space to millimeter space must be performed, based on the known size of the target canvas. At this interface we can also use known data about stroke width and its relation to brush pressure (see [2]) to compute the required brush pressure for a required stroke width.…”

Non-Photorealistic Rendering Techniques for Artistic Robotic Painting

“…The painting machine applies different layers of strokes on the canvas and produces beautiful artworks that mimic those produced by humans. Recently, the brush technique has been further improved by J. M. Gülzow et al [16], who developed a method to analyze human brushstrokes and replicate them by means of the robotic painting system with impressive results.…”