Autonomous Robot-Guided Inspection System Based on Offline Programming and RGB-D Model

Bedaka, Amit Kumar; Mahmoud, Alaa M.; Lee, Shao-Chun; Lin, Chyi‐Yeu

doi:10.3390/s18114008

Cited by 12 publications

(3 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple applications benefit from a more accurate close-range depth measurement, for example wound measurement systems [ 2 , 3 ], robot-guided positioning to grip novel objects [ 4 ], or defect inspection systems [ 5 ]. Depth sensors have been used for markerless real-time tool tracking at medium range, in a multi-sensor laparoscopic training setup [ 6 ], or for automating surgical manipulation tasks [ 7 – 9 ] using a robotic surgical assistant, and for gesture recognition in the operating room [ 10 ].…”

Section: Related Workmentioning

confidence: 99%

Comparative evaluation of three commercially available markerless depth sensors for close-range use in surgical simulation

et al. 2023

View full text Add to dashboard Cite

Purpose Minimally invasive surgeries have restricted surgical ports, demanding a high skill level from the surgeon. Surgical simulation potentially reduces this steep learning curve and additionally provides quantitative feedback. Markerless depth sensors show great promise for quantification, but most such sensors are not designed for accurate reconstruction of complex anatomical forms in close-range. Methods This work compares three commercially available depth sensors, namely the Intel D405, D415, and the Stereolabs Zed-Mini in the range of 12–20 cm, for use in surgical simulation. Three environments are designed that closely mimic surgical simulation, comprising planar surfaces, rigid objects, and mitral valve models of silicone and realistic porcine tissue. The cameras are evaluated on Z-accuracy, temporal noise, fill rate, checker distance, point cloud comparisons, and visual inspection of surgical scenes, across several camera settings. Results The Intel cameras show sub-mm accuracy in most static environments. The D415 fails in reconstructing valve models, while the Zed-Mini provides lesser temporal noise and higher fill rate. The D405 could reconstruct anatomical structures like the mitral valve leaflet and a ring prosthesis, but performs poorly for reflective surfaces like surgical tools and thin structures like sutures. Conclusion If a high temporal resolution is needed and lower spatial resolution is acceptable, the Zed-Mini is the best choice, whereas the Intel D405 is the most suited for close-range applications. The D405 shows potential for applications like deformable registration of surfaces, but is not yet suitable for applications like real-time tool tracking or surgical skill assessment.

show abstract

Section: Related Workmentioning

confidence: 99%

Comparative evaluation of three commercially available markerless depth sensors for close-range use in surgical simulation

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Some commercial software, such as Robotmaster ® 2023 (Saint-Laurent, QC, Canada), Visual Components Robotics OLP 4.8.0 (Espoo, Finland), and Dürr ® DXQ3D.onsite process simulation tool (Stuttgart, Germany), offers advanced CAD-model-based path-planning functions specifically for spray-painting tasks in which it is only necessary to input some spray parameters to obtain a robot trajectory and a paint-deposition simulation as outputs. Other approaches, as in [ 10 , 11 ] in which an OLP platform based on Open CASCADE Technology (OCCT) [ 12 ] was presented, used CAD information to create a 3D scan and a glue application trajectory and to generate a robot program. Reference [ 13 ] presents a form of CAD-based trajectory optimization, but an initial trajectory must be defined as an input before the algorithm can optimize the point position to achieve the desired paint thickness quality on the surface.…”

Section: Introductionmentioning

confidence: 99%

Autonomous Trajectory Planning for Spray Painting on Complex Surfaces Based on a Point Cloud Model

Nieto Bastida,

Lin

2023

Sensors

Self Cite

View full text Add to dashboard Cite

Using teach pendants or offline programming methods can generate tool paths for robot manipulators to carry out production activities, such as spray painting on objects of different geometries. This task, in which the complexity of painting the surface is one of the main challenges, requires highly skilled operators. In addition, the time spent setting up a robot task can be justified for the mass production of the same workpiece. However, it is inconvenient for low-production and high-variation production lines. In order to overcome these challenges, this study presents an algorithm to autonomously generate robot trajectories for a spray-painting process applied to objects with complex surfaces based on input 3D point cloud data. A predefined spherical mesh wraps the object, organizing the geometrical attributes into a structured data set. Subsequently, the region of interest is extracted and isolated from the model, which serves as the basis for the automatic path-planning operation. A user-friendly graphical user interface (GUI) is developed to define input parameters, visualize the point cloud model and the generated trajectory, simulate paint quality using a color map, and ultimately generate the robot’s code. A 3D sensor is used to localize the pose of the workpiece ahead of the robot and adjust the robot’s trajectory. The efficacy of the proposed approach is validated first by using various workpieces within a simulated environment and second by employing a real robot to execute the motion task.

show abstract

“…Each analytical model separately studies the different characteristics of the UGV propulsion performance. The point of view of achieving a balance between the capabilities of the mobile robot to orient, plan, and estimate the positions of the obstacles and the factors limiting the progressive field ability [22][23][24] takes into account the fact that both the speeds in rectilinear or turn motion and driving autonomy are influenced by the unstructured character of the terrain, which could be sand, grass, concrete, etc. Their conclusion is that an approach that makes corrections on coefficients (following real-time measurements) is much closer to reality.…”

mentioning

confidence: 99%

The Integration of Collaborative Robot Systems and Their Environmental Impacts

et al. 2020

View full text Add to dashboard Cite

Today, industrial robots are used in dangerous environments in all sectors, including the sustainable energy sector. Sensors and processors collect and transmit information and data from users as a result of the application of robot control systems and sensory feedback. This paper proposes that the estimation of a collaborative robot system’s performance can be achieved by evaluating the mobility of robots. Scenarios have been determined in which an autonomous system has been used for intervention in crisis situations due to fire. The experimental model consists of three autonomous vehicles, two of which are ground vehicles and the other is an aerial vehicle. The conclusion of the research described in this paper highlights the fact that the integration of robotic systems made up of autonomous vehicles working in unstructured environments is difficult and at present there is no unitary analytical model.

show abstract

Autonomous Robot-Guided Inspection System Based on Offline Programming and RGB-D Model

Cited by 12 publications

References 28 publications

Comparative evaluation of three commercially available markerless depth sensors for close-range use in surgical simulation

Comparative evaluation of three commercially available markerless depth sensors for close-range use in surgical simulation

Autonomous Trajectory Planning for Spray Painting on Complex Surfaces Based on a Point Cloud Model

The Integration of Collaborative Robot Systems and Their Environmental Impacts

Contact Info

Product

Resources

About