Artistic robotic pencil sketching using closed-loop force control

Wu, Panlong; Hung, Yi-Chin; Shaw, Jin-Siang

doi:10.1177/09544062221096946

Cited by 7 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to drawing path planning, aspects such as pen lifting planning and coordinate system configuration require comprehensive planning and constraints to ensure that the robotic arm operates without any unforeseen incidents. The pen lifting method and coordinate settings used in this experiment were consistent with those established by Wu et al 7 For the liftoff path, two intermediary points were set to ensure that the robotic arm moved vertically during liftoff and touchdown, thereby preventing unnecessary marks on the paper. For the coordinate system, the tool centre point (TCP) was employed as a reference for the robotic arm's movement instructions.…”

Section: Methodsmentioning

confidence: 99%

“…The force control method used in this experiment was similar to that employed by Hung. 7 A force sensor was installed on the end effector of the robot to read the force values. Coupled with a PID controller, this setup ensured that the drawing force exerted by the robotic arm remained stable within an error margin of 0.5 N. This stability enabled the robotic arm to complete the sketching tasks while maintaining equilibrium.…”

Section: Force Controlmentioning

confidence: 99%

“…In 2022, Wu et al employed TSP with nearest-neighbour algorithms to plan pencil sketching paths efficiently. 7 The robotic arm successfully created recognisable sketches. Scalera et al developed a robotic painting system that used the watercolour technique and a sponge as the painting media.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Using genetic algorithm for drawing path planning in a robotic arm pencil sketching system

Shaw,

Lee

2024

Proceedings of the Institution of Mechanical Engineers, Part C:

Self Cite

View full text Add to dashboard Cite

The purpose of this study is to implement a robotic arm pencil sketching system through a novel drawing path planning method. Firstly, to enhance the craftsmanship and uniqueness of the artwork, the original image is transformed into an animated style using AnimeGANv2. After image processing, the image is divided into boundary and background layers for drawing path planning. In the realm of path planning, traditional methods often involve manual path design by artists, and only recently have automated path planning methods been proposed. However, these recent approaches still have some limitations. For instance, a pursuit of efficiency can lead to overly uniform strokes, making the artwork resemble a print rather than a genuine piece of art. Therefore, this study introduces a new path-planning approach based on the traveling salesman problem (TSP), utilising various algorithms for path-planning experiments. Different stroke effects generated by distinct algorithms are applied to appropriate areas. The aim is to let the generated artworks exhibit the technological appeal of algorithms while retaining the distinctive characteristics of pencil sketches. For robotic arm control, a force sensor is integrated onto the arm, and a proportional-integral-derivative (PID) controller is employed to monitor and precisely control the drawing force. Experimental results demonstrate that artworks created using genetic algorithms to generate drawing paths exhibit distinct colour-layering effects, emphasising the characteristics of pencil strokes. In comparison with previous methods, this approach effectively addresses the issue of overly uniform and rigid strokes in past techniques.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Force Controlmentioning

confidence: 99%

See 1 more Smart Citation

Using genetic algorithm for drawing path planning in a robotic arm pencil sketching system

Shaw,

Lee

2024

Proceedings of the Institution of Mechanical Engineers, Part C:

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, this research also uses neural style transfer technology to extract the content and style features of the image, and performs edge detection and further layering on the newly generated image. The results show that the system has good effectiveness in painting and the painting time is less than 30 minutes [10]. In order to allow non-professionals to operate robots as easily as professionals, researchers such as Jens P introduced text-based programming that minimizes robot manufacturing.…”

Section: Literature Reviewmentioning

confidence: 99%

Image Generation of Animation Drawing Robot Based on Knowledge Distillation and Semantic Constraints

Wang

2024

ijacsa

View full text Add to dashboard Cite

With the development of robot technology, animation drawing robots have gradually appeared in the public eye. Animation drawing robots can generate many types of images, but there are also problems such as poor quality of generated images and long image drawing time. In order to improve the quality of images generated by animation drawing robots, an animation face line drawing generation algorithm based on knowledge distillation was designed to reduce computational complexity through knowledge distillation. To further raise the quality of images generated by robots, the research also designed an unsupervised facial caricature image generation algorithm based on semantic constraints, which uses facial semantic labels to constrain the facial structure of the generated images. The outcomes denote that the max values of the peak signal-to-noise ratio and feature similarity index measurements of the line drawing generation model are 39.45 and 0.7660 respectively, and the mini values are 37.51 and 0.7483 respectively. The average values of the gradient magnitude similarity bias and structural similarity of the loss function used in this model are 0.2041 and 0.8669 respectively. The max and mini values of Fréchet Inception Distance of the face caricature image generation model are 81.60 and 71.32 respectively, and the max and mini time-consuming values are 15.21s and 13.24s respectively. Both the line drawing generation model and the face caricature image generation model have good performance and can provide technical support for the image generation of animation drawing robots.

show abstract

“…For example, [ 23 ] utilized paint brush techniques to create highly detailed portraits, but this required a large number of strokes, resulting in lengthy drawing times; a single portrait took 17 h to complete a drawing with 9000 strokes. Other methods, such as those that use pens or pencils [ 10 , 25 , 26 , 27 , 28 ], are also time consuming and may fail to preserve facial details in the drawings.…”

Section: Related Workmentioning

confidence: 99%

Artistic Robotic Arm: Drawing Portraits on Physical Canvas under 80 Seconds

Nasrat,

Kang,

Park

et al. 2023

Sensors

View full text Add to dashboard Cite

In recent years, the field of robotic portrait drawing has garnered considerable interest, as evidenced by the growing number of researchers focusing on either the speed or quality of the output drawing. However, the pursuit of either speed or quality alone has resulted in a trade-off between the two objectives. Therefore, in this paper, we propose a new approach that combines both objectives by leveraging advanced machine learning techniques and a variable line width Chinese calligraphy pen. Our proposed system emulates the human drawing process, which entails planning the sketch and creating it on the canvas, thus providing a realistic and high-quality output. One of the main challenges in portrait drawing is preserving the facial features, such as the eyes, mouth, nose, and hair, which are crucial for capturing the essence of a person. To overcome this challenge, we employ CycleGAN, a powerful technique that retains important facial details while transferring the visualized sketch onto the canvas. Moreover, we introduce the Drawing Motion Generation and Robot Motion Control Modules to transfer the visualized sketch onto a physical canvas. These modules enable our system to create high-quality portraits within seconds, surpassing existing methods in terms of both time efficiency and detail quality. Our proposed system was evaluated through extensive real-life experiments and showcased at the RoboWorld 2022 exhibition. During the exhibition, our system drew portraits of more than 40 visitors, yielding a survey outcome with a satisfaction rate of 95%. This result indicates the effectiveness of our approach in creating high-quality portraits that are not only visually pleasing but also accurate.

show abstract

Artistic robotic pencil sketching using closed-loop force control

Cited by 7 publications

References 19 publications

Using genetic algorithm for drawing path planning in a robotic arm pencil sketching system

Using genetic algorithm for drawing path planning in a robotic arm pencil sketching system

Image Generation of Animation Drawing Robot Based on Knowledge Distillation and Semantic Constraints

Artistic Robotic Arm: Drawing Portraits on Physical Canvas under 80 Seconds

Contact Info

Product

Resources

About