Pallet recognition and localization using an RGB-D camera

Xiao, Jing; Lu, Huimin; Zhang, Lilian; Zhang, Jianhua

doi:10.1177/1729881417737799

Cited by 19 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Overall, our pallet docking accuracy can be controlled within ±6 mm and the coverage range is from 1–3 m and from −15° to +15°. When compared to the measures in the literature [ 32 ], the coverage range is roughly the same, but it does not test for severe pallet angle deflection. More importantly, the pallet accuracy in the literature [ 32 ] is only ±3 cm, which is much lower than our ±6 mm accuracy.…”

Section: Experiments Resultsmentioning

confidence: 99%

“…When compared to the measures in the literature [ 32 ], the coverage range is roughly the same, but it does not test for severe pallet angle deflection. More importantly, the pallet accuracy in the literature [ 32 ] is only ±3 cm, which is much lower than our ±6 mm accuracy. Similar coverage and coverage angle experiments to ours were carried out in the literature [ 33 ], but their maximum error of recognition was 10.5 mm after only 135 experiments, which was 75% higher than our maximum error of 6 mm.…”

Section: Experiments Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning-Based Intelligent Forklift Cargo Accurate Transfer System

Ren

Pan

Yao

et al. 2022

Sensors

View full text Add to dashboard Cite

In this research, we present an intelligent forklift cargo precision transfer system to address the issue of poor pallet docking accuracy and low recognition rate when using current techniques. The technology is primarily used to automatically check if there is any pallet that need to be transported. The intelligent forklift is then sent to the area of the target pallet after being recognized. Images of the pallets are then collected using the forklift’s camera, and a deep learning-based recognition algorithm is used to calculate the precise position of the pallets. Finally, the forklift is controlled by a high-precision control algorithm to insert the pallet in the exact location. This system creatively introduces the small target detection into the pallet target recognition system, which greatly improves the recognition rate of the system. The application of Yolov5 into the pallet positional calculation makes the coverage and recognition accuracy of the algorithm improved. In comparison with the prior approach, this system’s identification rate and accuracy are substantially higher, and it requires fewer sensors and indications to help with deployment. We have collected a significant amount of real data in order to confirm the system’s viability and stability. Among them, the accuracy of pallet docking is evaluated 1000 times, and the inaccuracy is kept to a maximum of 6 mm. The recognition rate of pallet recognition is above 99.5% in 7 days of continuous trials.

show abstract

Section: Experiments Resultsmentioning

confidence: 99%

Section: Experiments Resultsmentioning

confidence: 99%

Deep Learning-Based Intelligent Forklift Cargo Accurate Transfer System

Ren

Pan

Yao

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…3D cameras and processing algorithms of point clouds (3D-Keypoints, depth segmentation, etc.) were also used in [12,19,20] and [21] for this same task. Note that the purpose of all these works is automatic pallet detection, but using only views of the side faces.…”

Section: Related Workmentioning

confidence: 99%

MOSPPA: monitoring system for palletised packaging recognition and tracking

Castaño-Amorós

Fuentes²,

Gil³

2023

Int J Adv Manuf Technol

View full text Add to dashboard Cite

The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked on pallets to be supplied to its customers. Human operators usually classify these pallets according to the physical features of the cardboard packaging. This process can be slow, causing congestion on the production line. To optimise the logistics of this process, we propose a visual recognition and tracking pipeline that monitors the palletised packaging while it is moving inside the factory on roller conveyors. Our pipeline has a two-stage architecture composed of Convolutional Neural Networks, one for oriented pallet detection and recognition, and another with which to track identified pallets. We carried out an extensive study using different methods for the pallet detection and tracking tasks and discovered that the oriented object detection approach was the most suitable. Our proposal recognises and tracks different configurations and visual appearance of palletised packaging, providing statistical data in real time with which to assist human operators in decision-making. We tested the precision-performance of the system at the Smurfit Kappa facilities. Our proposal attained an Average Precision (AP) of 0.93 at 14 Frames Per Second (FPS), losing only 1% of detections. Our system is, therefore, able to optimise and speed up the process of logistic distribution.

show abstract

“…However, this method suffered from pallets varying in size and shape, as it could not capture enough features from the 2D depth information (Mohamed et al , 2020; Molter and Fottner, 2018). Some authors suggested using plane segmentation and template matching or registration to deliver more precise results (Xiao et al , 2017). However, the speed and accuracy of pallet recognition are heavily influenced by pallet types and the quality of point cloud data, potentially imposing severe requirements requiring a demanding high-fidelity depth camera and a powerful computer, significantly increasing cost.…”

Section: Introductionmentioning

confidence: 99%

A systematic strategy of pallet identification and picking based on deep learning techniques

Li¹,

Ding²,

Li³

et al. 2023

View full text Add to dashboard Cite

Purpose This paper presents a comprehensive pallet-picking approach for forklift robots, comprising a pallet identification and localization algorithm (PILA) to detect and locate the pallet and a vehicle alignment algorithm (VAA) to align the vehicle fork arms with the targeted pallet. Design/methodology/approach Opposing vision-based methods or point cloud data strategies, we utilize a low-cost RGB-D camera, and thus PILA exploits both RGB and depth data to quickly and precisely recognize and localize the pallet. The developed method guarantees a high identification rate from RGB images and more precise 3D localization information than a depth camera. Additionally, a deep neural network (DNN) method is applied to detect and locate the pallet in the RGB images. Specifically, the point cloud data is correlated with the labeled region of interest (RoI) in the RGB images, and the pallet's front-face plane is extracted from the point cloud. Furthermore, PILA introduces a universal geometrical rule to identify the pallet's center as a “T-shape” without depending on specific pallet types. Finally, VAA is proposed to implement the vehicle approaching and pallet picking operations as a “proof-of-concept” to test PILA’s performance. Findings Experimentally, the orientation angle and centric location of the two kinds of pallets are investigated without any artificial marking. The results show that the pallet could be located with a three-dimensional localization accuracy of 1 cm and an angle resolution of 0.4 degrees at a distance of 3 m with the vehicle control algorithm. Research limitations/implications PILA’s performance is limited by the current depth camera’s range (< = 3 m), and this is expected to be improved by using a better depth measurement device in the future. Originality/value The results demonstrate that the pallets can be located with an accuracy of 1cm along the x, y, and z directions and affording an angular resolution of 0.4 degrees at a distance of 3m in 700ms.

show abstract

Pallet recognition and localization using an RGB-D camera

Cited by 19 publications

References 18 publications

Deep Learning-Based Intelligent Forklift Cargo Accurate Transfer System

Deep Learning-Based Intelligent Forklift Cargo Accurate Transfer System

MOSPPA: monitoring system for palletised packaging recognition and tracking

A systematic strategy of pallet identification and picking based on deep learning techniques

Contact Info

Product

Resources

About