AwezaMed: A Multilingual, Multimodal Speech-To-Speech Translation Application for Maternal Health Care

Marais, Laurette; Louw, Johannes A.; Badenhorst, Jaco; Calteaux, Karen; Wilken, Ilana; Niekerk, Nina van; Stein, Glenn

doi:10.23919/fusion45008.2020.9190240

Cited by 8 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It distinguishes between abstract syntax, which models the compositional relations between concepts, and concrete syntax, which defines how concepts are expressed in different natural languages. Multilingual domainspecific grammars for the South African languages have been used to facilitate speech-to-speech translation in the health domain (Marais et al 2020), and they are currently employed in the Ngiyaqonda! project to support literacy development.…”

Section: Grammar Developmentmentioning

confidence: 99%

Bringing Children’s Dictionaries to Digital Life

Wilken,

Marais

2024

JDHASA

View full text Add to dashboard Cite

South Africa is facing a literacy crisis, with the latest PIRLS results showing that 8 out of 10 learners cannot read for basic comprehension by the time they leave the foundation phase. In this climate, the development of strategies to assist educators in harnessing the available resources to maximum effect is needed. However, most teaching resources are not digitally available, and even fewer are available in formats that make them readily available for use in natural language applications.The Ngiyaqonda! project aims to provide an interactive, multimodal digital environment within which learners can practise their reading and writing skills. Computational grammars and speech technology are combined in a mobile application to facilitate the transition from oral competency in a language to written competency. In this paper, we show how words from a multilingual dictionary for foundation phase learners can be brought to digital life within the Ngiyaqonda! application to enhance the learning experience of core concepts and vocabulary.We use the official foundation phase CAPS English-isiZulu dictionary (Mbatha et al. 2018) to ensure that the content of the computational grammars is aligned with relevant learning outcomes. The result is a fully parallel, multilingual computational grammar that is aligned at the semantic level, ready to be included in the Ngiyaqonda! application.

show abstract

Section: Grammar Developmentmentioning

confidence: 99%

Bringing Children’s Dictionaries to Digital Life

Wilken,

Marais

2024

JDHASA

View full text Add to dashboard Cite

show abstract

“…Video modality was found in a total of 45 articles, with 11 papers [165], [218], [269], [330], [378], [380]- [382], [387], [388], [390] separately and 34 combined with data of different modalities. The use of audio modality was found in 72 articles; 5 were used individually [129], [243], [276], [284], [332], and 67 were used with others. The occupancy of text modality was found in 93 papers, and from them, it was individually used in 11 articles [48], [68], [82], [159], [168], [169], [200], [245], [316], [354], [419] and combinedly used in the rest of 82 articles.…”

Section: B Taskmentioning

confidence: 99%

“…Grammar-based, encoderdecoder, and continuous are three subtypes of generative. Of 19 articles, one was grammar-based [372], twelve were encoder-decoder based [110], [155], [194], [222], [291], [339], [341], [342], [371], [373], [374], [417], and six were continuous-based [129], [177], [243], [294], [344], [346].…”

Section: Translationmentioning

confidence: 99%

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

2023

View full text Add to dashboard Cite

Multimodal machine learning (MML) is a tempting multidisciplinary research area where heterogeneous data from multiple modalities and machine learning (ML) are combined to solve critical problems. Usually, research works use data from a single modality, such as images, audio, text, and signals. However, real-world issues have become critical now, and handling them using multiple modalities of data instead of a single modality can significantly impact finding solutions. ML algorithms play an essential role by tuning parameters in developing MML models. This paper reviews recent advancements in the challenges of MML, namely: representation, translation, alignment, fusion and co-learning, and presents the gaps and challenges. A systematic literature review (SLR) applied to define the progress and trends on those challenges in the MML domain. In total, 1032 articles were examined in this review to extract features like source, domain, application, modality, etc. This research article will help researchers understand the constant state of MML and navigate the selection of future research directions.

show abstract

“…These earlier publications are mostly confined to reporting literature on the use of MT for real-time bilingual person-to-person communication. The technology is mainly studied as an on-premise solution to support medical service provision in spoken interactions between specific groups of patients (eg, tourists, refugees, or expatriates) and health care staff (eg, general practitioners, caregivers, or paramedics) [24][25][26][27][28]. Only a few of the articles explore the use of multilingual translation tools for disseminating PH information to specific target audiences [29] or for population-wide health initiatives [30].…”

Section: Prior Workmentioning

confidence: 99%

The Use of Machine Translation for Outreach and Health Communication in Epidemiology and Public Health: Scoping Review

Herrera-Espejel,

Rach

2023

JMIR Public Health Surveill

View full text Add to dashboard Cite

Background Culturally and linguistically diverse groups are often underrepresented in population-based research and surveillance efforts, leading to biased study results and limited generalizability. These groups, often termed “hard-to-reach,” commonly encounter language barriers in the public health (PH) outreach material and information campaigns, reducing their involvement with the information. As a result, these groups are challenged by 2 effects: the medical and health knowledge is less tailored to their needs, and at the same time, it is less accessible for to them. Modern machine translation (MT) tools might offer a cost-effective solution to PH material language accessibility problems. Objective This scoping review aims to systematically investigate current use cases of MT specific to the fields of PH and epidemiology, with a particular interest in its use for population-based recruitment methods. Methods PubMed, PubMed Central, Scopus, ACM Digital Library, and IEEE Xplore were searched to identify articles reporting on the use of MT in PH and epidemiological research for this PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews)–compliant scoping review. Information on communication scenarios, study designs and the principal findings of each article were mapped according to a settings approach, the World Health Organization monitoring and evaluation framework and the service readiness level framework, respectively. Results Of the 7186 articles identified, 46 (0.64%) were included in this review, with the earliest study dating from 2009. Most of the studies (17/46, 37%) discussed the application of MT to existing PH materials, limited to one-way communication between PH officials and addressed audiences. No specific article investigated the use of MT for recruiting linguistically diverse participants to population-based studies. Regarding study designs, nearly three-quarters (34/46, 74%) of the articles provided technical assessments of MT from 1 language (mainly English) to a few others (eg, Spanish, Chinese, or French). Only a few (12/46, 26%) explored end-user attitudes (mainly of PH employees), whereas none examined the legal or ethical implications of using MT. The experiments primarily involved PH experts with language proficiencies. Overall, more than half (38/70, 54% statements) of the summarizing results presented mixed and inconclusive views on the technical readiness of MT for PH information. Conclusions Using MT in epidemiology and PH can enhance outreach to linguistically diverse populations. The translation quality of current commercial MT solutions (eg, Google Translate and DeepL Translator) is sufficient if postediting is a mandatory step in the translation workflow. Postediting of legally or ethically sensitive material requires staff with adequate content knowledge in addition to sufficient language skills. Unsupervised MT is generally not recommended. Research on whether machine-translated texts are received differently by addressees is lacking, as well as research on MT in communication scenarios that warrant a response from the addressees.

show abstract

AwezaMed: A Multilingual, Multimodal Speech-To-Speech Translation Application for Maternal Health Care

Cited by 8 publications

References 11 publications

Bringing Children’s Dictionaries to Digital Life

Bringing Children’s Dictionaries to Digital Life

A Systematic Literature Review on Multimodal Machine Learning: Applications, Challenges, Gaps and Future Directions

The Use of Machine Translation for Outreach and Health Communication in Epidemiology and Public Health: Scoping Review

Contact Info

Product

Resources

About