Random Cropping Ensemble Neural Network for Image Classification in a Robotic Arm Grasping System

Cheng, Bo; Wu, Wanyin; Tao, Dapeng; Mei, Shibo; Mei, Ting; Cheng, Jun

doi:10.1109/tim.2020.2976420

Cited by 36 publications

(12 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This can be beneficial, as different models may perform better in different aspects of the problem [53]. In our work, the ensemble was comprised of networks with the same architecture, similar to [54]. We chose not to use different networks in our ensemble due to the very small size of our dataset.…”

Section: Discussionmentioning

confidence: 99%

Predicting the Timing of Camera Movements From the Kinematics of Instruments in Robotic-Assisted Surgery Using Artificial Neural Networks

Kossowsky¹,

Nisky²

2021

Preprint

View full text Add to dashboard Cite

Robotic-assisted surgeries benefit both surgeons and patients, however, surgeons frequently need to adjust the endoscopic camera to achieve good viewpoints. Simultaneously controlling the camera and the surgical instruments is impossible, and consequentially, these camera adjustments repeatedly interrupt the surgery. Autonomous camera control could help overcome this challenge, but most existing systems are reactive, e.g., by having the camera follow the surgical instruments. We propose a predictive approach for anticipating when camera movements will occur using artificial neural networks. We used the kinematic data of the surgical instruments, which were recorded during robotic-assisted surgical training on porcine models. We split the data into segments, and labeled each either as a segment that immediately precedes a camera movement, or one that does not. Due to the large class imbalance, we trained an ensemble of networks, each on a balanced sub-set of the training data. We found that the instruments' kinematic data can be used to predict when camera movements will occur, and evaluated the performance on different segment durations and ensemble sizes. We also studied how much in advance an upcoming camera movement can be predicted, and found that predicting a camera movement 0.25, 0.5, and 1 second before they occurred achieved 98%, 94%, and 84% accuracy relative to the prediction of an imminent camera movement. This indicates that camera movement events can be predicted early enough to leave time for computing and executing an autonomous camera movement and suggests that an autonomous camera controller for RAMIS may one day be feasible.

show abstract

Section: Discussionmentioning

confidence: 99%

Predicting the Timing of Camera Movements From the Kinematics of Instruments in Robotic-Assisted Surgery Using Artificial Neural Networks

Kossowsky¹,

Nisky²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Additionally, the use of CNNs can substantially enhance grasping accuracy; however, the annotation of the grasping point is costly (e.g., [ 60 ]). Contrary to conventional image classification algorithms (e.g., CNNs), the weighted ensemble neural network [ 119 ] can effectively overcome the difficulty of uneven placement and illumination across an image, which can affect the grasp point estimation during learning. D’Avella et al [ 120 ] created a collaborative robotic system that combined a conventional two-finger gripper with a low-cost custom universal jamming gripper (UJG).…”

Section: Challenges and Future Directionsmentioning

confidence: 99%

Review of Learning-Based Robotic Manipulation in Cluttered Environments

Mohammed

Kwek

Chua

et al. 2022

Sensors

View full text Add to dashboard Cite

Robotic manipulation refers to how robots intelligently interact with the objects in their surroundings, such as grasping and carrying an object from one place to another. Dexterous manipulating skills enable robots to assist humans in accomplishing various tasks that might be too dangerous or difficult to do. This requires robots to intelligently plan and control the actions of their hands and arms. Object manipulation is a vital skill in several robotic tasks. However, it poses a challenge to robotics. The motivation behind this review paper is to review and analyze the most relevant studies on learning-based object manipulation in clutter. Unlike other reviews, this review paper provides valuable insights into the manipulation of objects using deep reinforcement learning (deep RL) in dense clutter. Various studies are examined by surveying existing literature and investigating various aspects, namely, the intended applications, the techniques applied, the challenges faced by researchers, and the recommendations adopted to overcome these obstacles. In this review, we divide deep RL-based robotic manipulation tasks in cluttered environments into three categories, namely, object removal, assembly and rearrangement, and object retrieval and singulation tasks. We then discuss the challenges and potential prospects of object manipulation in clutter. The findings of this review are intended to assist in establishing important guidelines and directions for academics and researchers in the future.

show abstract

“…Data augmentation is designed based on domain knowledge. For example, in CV datasets, operations such as color jittering [42], random cropping [4], applying Gaussian blur [9], Mixup [24,21,25] are proven useful. In biology and some easy datasets, linear combinations τ lc (•) in k-nearest neighbor data is a simple and effective way.…”

Section: Dlme Frameworkmentioning

confidence: 99%

DLME: Deep Local-flatness Manifold Embedding

Zang¹,

Liu²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Manifold learning (ML) aims to seek low-dimensional embedding from high-dimensional data. The problem is challenging on realworld datasets, especially with under-sampling data, and we find that previous methods perform poorly in this case. Generally, ML methods first transform input data into a low-dimensional embedding space to maintain the data's geometric structure and subsequently perform downstream tasks therein. The poor local connectivity of under-sampling data in the former step and inappropriate optimization objectives in the latter step leads to two problems: structural distortion and underconstrained embedding. This paper proposes a novel ML framework named Deep Local-flatness Manifold Embedding (DLME) to solve these problems. The proposed DLME constructs semantic manifolds by data augmentation and overcomes the structural distortion problem using a smoothness constrained based on a local flatness assumption about the manifold. To overcome the underconstrained embedding problem, we design a loss and theoretically demonstrate that it leads to a more suitable embedding based on the local flatness. Experiments on three types of datasets (toy, biological, and image) for various downstream tasks (classification, clustering, and visualization) show that our proposed DLME outperforms state-of-the-art ML and contrastive learning methods.

show abstract

Random Cropping Ensemble Neural Network for Image Classification in a Robotic Arm Grasping System

Cited by 36 publications

References 36 publications

Predicting the Timing of Camera Movements From the Kinematics of Instruments in Robotic-Assisted Surgery Using Artificial Neural Networks

Predicting the Timing of Camera Movements From the Kinematics of Instruments in Robotic-Assisted Surgery Using Artificial Neural Networks

Review of Learning-Based Robotic Manipulation in Cluttered Environments

DLME: Deep Local-flatness Manifold Embedding

Contact Info

Product

Resources

About