Processing and Forecasting the Trajectory of a Thrown Object Measured by the Stereo Vision System

Mironov, Konstantin; Vladimirova, I. A.; Pongratz, Martin

doi:10.1016/j.ifacol.2015.09.155

Cited by 12 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work in [ 32 , 33 ] used stereo vision to collect samples of mechanically thrown tennis ball trajectories. However, they enabled researchers to measure positions of ball in camera-related coordinate system with millimeters of accuracy (even outliers in some measurements).…”

Section: Related Workmentioning

confidence: 99%

Intelligent Tracking of Mechanically Thrown Objects by Industrial Catching Robot for Automated In-Plant Logistics 4.0

Qadeer

Shah

Sharif

et al. 2022

Sensors

View full text Add to dashboard Cite

Industry 4.0 smart manufacturing systems are equipped with sensors, smart machines, and intelligent robots. The automated in-plant transportation of manufacturing parts through throwing and catching robots is an attempt to accelerate the transportation process and increase productivity by the optimized utilization of in-plant facilities. Such an approach requires intelligent tracking and prediction of the final 3D catching position of thrown objects, while observing their initial flight trajectory in real-time, by catching robot in order to grasp them accurately. Due to non-deterministic nature of such mechanically thrown objects’ flight, accurate prediction of their complete trajectory is only possible if we accurately observe initial trajectory as well as intelligently predict remaining trajectory. The thrown objects in industry can be of any shape but detecting and accurately predicting interception positions of any shape object is an extremely challenging problem that needs to be solved step by step. In this research work, we only considered spherical shape objects as their3D central position can be easily determined. Our work comprised of development of a 3D simulated environment which enabled us to throw object of any mass, diameter, or surface air friction properties in a controlled internal logistics environment. It also enabled us to throw object with any initial velocity and observe its trajectory by placing a simulated pinhole camera at any place within 3D vicinity of internal logistics. We also employed multi-view geometry among simulated cameras in order to observe trajectories more accurately. Hence, it provided us an ample opportunity of precise experimentation in order to create enormous dataset of thrown object trajectories to train an encoder-decoder bidirectional LSTM deep neural network. The trained neural network has given the best results for accurately predicting trajectory of thrown objects in real time.

show abstract

Section: Related Workmentioning

confidence: 99%

Intelligent Tracking of Mechanically Thrown Objects by Industrial Catching Robot for Automated In-Plant Logistics 4.0

Qadeer

Shah

Sharif

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…In the event that lateral forces do not act in flight on the body (they can be associated, for example, with the action of the wind or with the Magnus effect), the flight path lies in such a plane. Experiments conducted in [20] showed that the influence of lateral forces on the flight of an object can be neglected. Since the real direction of the throw in each case will be different from the others, then the transition matrix between the systems will be different for each case.…”

Section: Extraction Of the Spatial Coordinates From Video Signalmentioning

confidence: 99%

“…Moreover, the results of prediction by neural networks are difficult to interpret; therefore, it was later proposed to apply a more transparent method of k nearest neighbors [19]. The development of individual details of this method is described in the articles of [20,21].…”

Section: Predicting the Trajectory Of A Thrown Objectmentioning

confidence: 99%

“…Robotic transfer as a method of such transportation was proposed in 2006 by Frank [12]. This application was developed in [13][14][15][16][17][18][19][20][21]. Transportation of an object from some point of departure A to some destination B is as follows: the robot thrower located in A throws the object in direction B and notifies about it via the communication line, and the robot catcher located in B, having received the notification, carries out object capture on the fly.…”

Section: Introductionmentioning

confidence: 99%

“…They do it only based on the previous experience. This circumstance motivated the development of learning-based forecasting algorithms, such as neural network trajectory predictor [18] and k nearest neighbor's trajectory predictor [19][20][21]. Learning-based techniques require collecting the sampling of trajectories in order to train the predictor.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Observing and Forecasting the Trajectory of the Thrown Body with use of Genetic Programming

Mironov¹,

Gayanov²,

Kurennov³

2019

Adv. sci. technol. eng. syst. j.

Self Cite

View full text Add to dashboard Cite

Robotic catching of thrown objects is one of the common robotic tasks, which is explored in a number of papers. This task includes subtask of tracking and forecasting the trajectory of the thrown object. Here we propose an algorithm for estimating future trajectory based on video signal from two cameras. Most of existing implementations use deterministic trajectory prediction and several are based on machine learning. We propose a combined forecasting algorithm where the deterministic motion model for each trajectory is generated via the genetic programming algorithm. Object trajectory is extracted from video sequence by the image processing algorithm, which include Canny edge detection, Random Sample Consensus circle recognition and stereo triangulation. After that rajectory is forecasted using proposed method. Numerical experiments with real trajectories of the thrown tennis ball show that the algorithm is able to forecast the trajectory accurately.

show abstract

Ball Path Prediction for Humanoid Robots: Combination of k-NN Regression and Autoregression Methods

Mirmohammad

Khorsandi

Shahsavari

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Processing and Forecasting the Trajectory of a Thrown Object Measured by the Stereo Vision System

Cited by 12 publications

References 12 publications

Intelligent Tracking of Mechanically Thrown Objects by Industrial Catching Robot for Automated In-Plant Logistics 4.0

Intelligent Tracking of Mechanically Thrown Objects by Industrial Catching Robot for Automated In-Plant Logistics 4.0

Observing and Forecasting the Trajectory of the Thrown Body with use of Genetic Programming

Ball Path Prediction for Humanoid Robots: Combination of k-NN Regression and Autoregression Methods

Contact Info

Product

Resources

About