Human–Robot Collaborative Assembly Based on Eye-Hand and a Finite State Machine in a Virtual Environment

Zhao, Xue; He, Ye; Chen, Xiaoan; Li, Zhi

doi:10.3390/app11125754

Cited by 14 publications

(22 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of computer VR technology in the design of digital media system can enable the cross-border integration and development of information expression, which is conducive to further optimizing the design of digital media system and further strengthening the sense of user experience. In the design of digital media system, computer VR technology can generate computer simulation according to the diversified needs of different users and bring users into a virtual environment [7]. This communication method can make users feel immersive.…”

Section: Literature Reviewmentioning

confidence: 99%

Application of Computer Technology in VR Digital Media

Zhang

Wang

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

In order to make up for the defects of traditional digital media system design, continuously strengthen the user’s sense of experience, and further improve the digital media system design through diversified digital media forms, this paper proposes an application method of computer technology in VR digital media. Through the introduction and research of human-computer interaction interface, the usability of human-computer interaction interface evaluation and testing is analyzed, and combined with GOMS model to optimize and improve the model, a new optimized GOMS layered quantitative model is proposed. The experimental results show that when the system is running, the trend of video cache rate in WiFi and 4G environments is basically the same. The playback start rate is fast and then tends to be stable. During the playback process, the video frame rate is stable at about 60 FPS, which meets the needs of human eyes to watch without Caton. After the playback starts, the GPU occupancy rate remains about 20% and stable. Computer technology can realize the effective combination of multiscreen interaction and digital media in VR digital media, creating a better prospect interactive experience for the future development of digital media mobile terminal.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Application of Computer Technology in VR Digital Media

Zhang

Wang

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…Therefore, it implements SBF (6). The additional variables b j ∈ B enter LUTerZT which implements SBFs ( 9) and (10). The IMFs D r ∈ Φ enter the state code register SCR hidden inside of LUTerZT.…”

Section: Related Workmentioning

confidence: 99%

“…(2) serial communication and display protocols [7]; (3) various software tools of embedded systems [8]; (4) control-dominated systems [9]; (5) different systems in robotics [10] (6) hardware-software interfaces of embedded systems [3]; (7) the activation functions for deep neutral networks [11,12] and so on. Currently, research related to finite state machines is actively developing [9,13,14].…”

Section: Introductionmentioning

confidence: 99%

Using Codes of Output Collections for Hardware Reduction in Circuits of LUT-Based Finite State Machines

et al. 2022

View full text Add to dashboard Cite

A method is proposed which aims to reduce the hardware in FPGA-based circuits of Mealy finite state machines (FSMs). The proposed method is a type of structural decomposition method. Its main goal is the reducing the number of look-up table (LUT) elements in FSM circuits compared to the three-block FSM circuit. The main idea of the proposed method is the using codes of collections of FSM outputs for replacing the FSM inputs and state variables. The interstate transitions are defined using collections of outputs generated in two adjacent cycles of synchronization. One, of output collection codes, is kept into a register. To optimize block-generating FSM outputs, a new type of state codes is proposed. A state is encoded as an element of some class of states. This approach allows both the number of logic levels and inter-level interconnections in LUT-based FSM circuit to be diminished. An example of an LUT-based Mealy FSM circuit with the proposed method applied is shown. Moreover, the results of our research are represented. The research was conducted using the CAD tool Vivado by Xilinx. The experiments prove that the proposed approach allows the reduction of hardware compared with such known methods as Auto and One-hot of Vivado, and JEDI. Moreover, the proposed approach gives better results than a method based on the simultaneous replacement of inputs and encoding collections of outputs. Compared to circuits of the three-block FSMs, the LUT counts are reduced by an average of 10.07% without significant reduction in the value of operating frequency. The gain in LUT counts increases with the increasing the numbers of FSM states and inputs.

show abstract

“…There has been a remarkable emergence of works related to the utilization of 3D sensor-based or depth sensor-based hand gesture recognition as an interaction interface between the robot system and the user. 15–27 Studies concerning HRI via hand gesture communication can be categorized into various different applications, imitation robots, 15,16 show robots, 17 sport or rehabilitation instructor expert robots, 18,19 surgery robots, 20 and industrial robots. 21–34…”

Section: Introductionmentioning

confidence: 99%

Designs of human–robot interaction using depth sensor-based hand gesture communication for smart material-handling robot operations

Ding

2022

Proceedings of the Institution of Mechanical Engineers, Part B:

View full text Add to dashboard Cite

With rapid developments in biometric recognition, a great deal of attention is being paid to robots which interact smartly with humans and communicate certain types of biometrical information. Such human–machine interaction (HMI), also well-known as human–robot interaction (HRI), will, in the future, prove an important development when it comes to automotive manufacturing applications. Currently, hand gesture recognition-based HRI designs are being practically used in various areas of automotive manufacturing, assembly lines, supply chains, and collaborative inspection. However, very few studies are focused on material-handling robot interactions combined with hand gesture communication of the operator. The current work develops a depth sensor-based dynamic hand gesture recognition scheme for continuous-time operations with material-handling robots. The proposed approach properly employs the Kinect depth sensor to extract features of Hu moment invariants from depth data, through which feature-based template match hand gesture recognition is developed. In order to construct continuous-time robot operations using dynamic hand gestures with concatenations of a series of hand gesture actions, the wake-up reminder scheme using fingertip detection calculations is established to accurately denote the starting, ending, and switching timestamps of a series of gesture actions. To be able to perform typical template match on continuous-time dynamic hand gesture recognition with the ability of real-time recognition, representative frame estimates using centroid, middle, and middle-region voting approaches are also presented and combined with template match computations. Experimental results show that, in certain continuous-time periods, the proposed complete hand gesture recognition framework can provide a smooth operation for the material-handling robot when compared with robots controlled using only extractions of full frames; presented representative frames estimated by middle-region voting will maintain fast computations and still reach the competitive recognition accuracy of 90.8%. The method proposed in this study can facilitate the smart assembly line and human–robot collaborations in automotive manufacturing.

show abstract

Human–Robot Collaborative Assembly Based on Eye-Hand and a Finite State Machine in a Virtual Environment

Cited by 14 publications

References 52 publications

Application of Computer Technology in VR Digital Media

Application of Computer Technology in VR Digital Media

Using Codes of Output Collections for Hardware Reduction in Circuits of LUT-Based Finite State Machines

Designs of human–robot interaction using depth sensor-based hand gesture communication for smart material-handling robot operations

Contact Info

Product

Resources

About