Adaptive multimodal interaction in mobile augmented reality: A conceptual framework

Abidin, Rimaniza Zainal; Arshad, Haslina; Shukri, S.

doi:10.1063/1.5005483

Cited by 5 publications

(3 citation statements)

References 14 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To name a few, the input can be an image target [2], [3] an object target [4] or location [5], [6]. The interaction in the AR environment can also be added with speech [7], [8] gesture [9] or a combination of both [10].…”

Section: Introductionmentioning

confidence: 99%

Augmented Reality to Promote Guided Discovery Learning for STEM Learning

Majid¹,

Majid²

2018

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

Augmented reality is one of the technologies that will impact teaching and learning. When augmented reality is applied in an educational setting, it can provide a superior learning environment because educators can add specific virtual information to enrich the existing learning materials or to create new learning materials that integrate the environment. Therefore, discussions on augmented reality applications for learning can be vastly found in the literature. Each application utilizes the resources available by using approaches appropriate for the learning outcome in question. The key problem with achieving learning outcome in analyzing group one elements in a periodic table is that some students have difficulties in visualizing an atom and what more to relate them with reactivity. The objective of this paper aims to design and develop a learning application that supports science, technology, engineering, and mathematics (STEM) education based on guided discovery learning to assist the students through questions based on exposure to various evidence via augmented reality. This application incorporates the use of a 3D model of atoms and video of real experiments. Action research is used to solve an existing problem in a selected school. The application was evaluated by 25 students aged 16 years in a secondary school in Malaysia and has proven to be useful to improve the students' marks in tests conducted at the end of the application usage. The students can get many benefits from using the augmented reality based application because it supports Education 4.0 like flexibility in teaching and learning where learning is freed for the limitation of time, place, and pace of study.

show abstract

Section: Introductionmentioning

confidence: 99%

Augmented Reality to Promote Guided Discovery Learning for STEM Learning

Majid¹,

Majid²

2018

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

show abstract

“…This section will discuss a comparative analysis between the proposed adaptive framework between three multimodal frameworks [43], [44], and [45] as shown in Table .3. This comparison relies on the of the differences between multimodal framework's properties of the modality data type & modality number, data fusion level, interpreted context considerable, experimental dataset, and weaknesses.…”

Section: A Comparative Analysis Between Proposed Adaptive Multimodal ...mentioning

confidence: 99%

“…The proposed adaptive framework can solve many previous drawbacks in [43], [44], and [45]. Tracing: based on data modality input (the tracing may be an image or an interval of features that can convert into numerical vectors and check classifier with proposed output vector of all features of all objects).…”

Section: A Comparative Analysis Between Proposed Adaptive Multimodal ...mentioning

confidence: 99%

An Adaptive and Late Multimodal Fusion Framework in Contextual Representation based Evidential Deep Learning Dempster-Shafer Theory

Eldin,

Hassanein,

Hassanien

2023

Preprint

View full text Add to dashboard Cite

There is a growing interest in multidisciplinary research in multimodal synthesis technology to stimulate diversity of modal interpretation in different application contexts. The current literature review focuses on modality-based systems in a specific known context and leaves a gap in fusing multiple modality data types in different contexts. Therefore, there seems to be a real requirement for an analytical review of recent developments in the field of data fusion. The real requirement for modality diversity across multiple contextual representation fields is due to the conflicting nature of data in multi-target sensors, which introduces other obstacles including ambiguous, uncertain data, imbalance and redundancy in multi object classification. Additionally, there is lack of frameworks that can analyze offline stream data to identify hidden relationships between different modal data types and different modal counts. Additionally, the lack of a multimodal fusion model capable of determining the extraction conditions of the extracted fusion data has led to low accuracy rates in classifying objects across modalities and systems. This paper proposes a new adaptive and late multimodal fusion framework to interpret multiple modalities and contextual representations using evidence-enhanced deep learning-based Dempster-Shafer theory. The proposed multimodal fusion framework is a MultiFusion learning model solution to solve the Modality-and context-based fusion to improve remote management, intelligent systems, and decision making. The proposed multimodal fusion framework can address the contradictory nature of data uncertainty, diversity of methods, factors, conditions, and relationships for multimodal explanation in multi-context systems to improve decision making and control in diverse contextual representations. Furthermore, this research provides a comparative analysis of the current fusion and prior multimodal data fusion models, explaining the differences of the construction analysis, mathematical analysis of fusion models, pros, and cons of them. In addition, this research presents a comparative analysis between the proposed framework with previous published fusion frameworks, exploring their concepts, advantages and problems, drivers, and current techniques. The experimental accuracy results in multimodalities experiments and multi-context using the proposed multimodal fusion framework is 98.45%. Additionally, some future research directions are discussed.

show abstract