Humanoid Robots and Conversational Recommender Systems: a Preliminary Study

Iovine, Andrea; Narducci, Fedelucio; Gemmis, Marco de; Semeraro, Giovanni

doi:10.1109/eais48028.2020.9122705

Cited by 5 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These experiments measured usability characteristics referred to effectiveness, efficiency and satisfaction as shown in Table 1. Some of the primary studies ( [19], [26], [27], [28], [29], [30], [31], [32], [33]) consider all three of these aspects, others ( [24], [25], [34], [35], [36], [37], [38], [39], [40]) investigate efficiency and satisfaction, others again ( [41], [42], [43], [44], [45]) consider only satisfaction, whereas only one ( [46]) evaluated both effectiveness and satisfaction. Satisfaction is again the usability characteristic of most concern to researchers since it is evaluated most often.…”

Section: Related Workmentioning

confidence: 99%

“…Within the primary studies, chatbots are dedicated to various domains. Most chatbots are used as personal assistants [25], [26], [29], [30], [34], [37], [38], [39], [40], [42], [43], [44], [45], [46], [47], especially in the healthcare domain [32], [35], [38], [44], some act as recommenders [27], [28], [31], emotion-aware conversational agents [41] and e-commerce chatbots [33], [36]. Nevertheless, none of the above chatbots is applied as a modelling tool like the SOCIO chatbot.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Using the SOCIO Chatbot for UML Modelling: A Family of Experiments

Ren

Castro

Santos

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Context: Recent developments in natural language processing have facilitated the adoption of chatbots in typically collaborative software engineering tasks (such as diagram modelling). Families of experiments can assess the performance of tools and processes and, at the same time, alleviate some of the typical shortcomings of individual experiments (e.g., inaccurate and potentially biased results due to a small number of participants). Objective: Compare the usability of a chatbot for collaborative modelling (i.e., SOCIO) and an online web tool (i.e., Creately). Method: We conduct a family of three experiments to evaluate the usability of SOCIO against the Creately online collaborative tool in academic settings. Results: The student participants were faster at building class diagrams using the chatbot than with the online collaborative tool and more satisfied with SOCIO. Besides, the class diagrams built using the chatbot tended to be more concise -albeit slightly less complete. Conclusion: Chatbots appear to be helpful for building class diagrams. In fact, our study has helped us to shed light on the future direction for experimentation in this field and lays the groundwork for researching the applicability of chatbots in diagramming.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Using the SOCIO Chatbot for UML Modelling: A Family of Experiments

Ren

Castro

Santos

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…Real Human Dialogue for CRSs. Plenty of studies [11,26,40] on CRSs are built on real human dialogues. In general, they initially create specific dialogue tasks and then recruit actual users to engage in natural language conversations.…”

Section: Related Workmentioning

confidence: 99%

“…With the text-based conversation generated, we proceed to generate the voice-based conversation by adopting the SOTA end-to-end text-to-speech (TTS) model VITS 10 . In particular, for our agent, we utilize VITS trained by the LJ Speech dataset 11 containing short audio clips of a single speaker; whereas for users, we exploit VITS trained by the Vctk dataset 12 , which includes speech data uttered by 109 English speakers with various accents. The statistics of the speakers regarding gender and age are summarized in Table 3, where we divide their ages into three groups, i.e., < 20, [20,30], > 30.…”

Section: Voice-based Conversation Generationmentioning

confidence: 99%

Towards Building Voice-based Conversational Recommender Systems: Datasets, Potential Solutions and Prospects

Qu¹,

Li²,

Sun

et al. 2023

Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Conversational recommender systems (CRSs) have become crucial emerging research topics in the field of RSs, thanks to their natural advantages of explicitly acquiring user preferences via interactive conversations and revealing the reasons behind recommendations. However, the majority of current CRSs are text-based, which is less user-friendly and may pose challenges for certain users, such as those with visual impairments or limited writing and reading abilities. Therefore, for the first time, this paper investigates the potential of voice-based CRS (VCRSs) to revolutionize the way users interact with RSs in a natural, intuitive, convenient, and accessible fashion.To support such studies, we create two VCRSs benchmark datasets in the e-commerce and movie domains, after realizing the lack of such datasets through an exhaustive literature review. Specifically, we first empirically verify the benefits and necessity of creating such datasets. Thereafter, we convert the user-item interactions to text-based conversations through the ChatGPT-driven prompts for generating diverse and natural templates, and then synthesize the corresponding audios via the text-to-speech model. Meanwhile, a number of strategies are delicately designed to ensure the naturalness and high quality of voice conversations. On this basis, we further explore the potential solutions and point out possible directions to build end-to-end VCRSs by seamlessly extracting and integrating voice-based inputs, thus delivering performance-enhanced,

show abstract

“…To calculate the numerical value of each participant's satisfaction score, we used Brooke's equation [23] below to calculate the quantitative SUS result. The team score was calculated using the median of the scores of the three team members for each question:…”

Section: A Objectives Hypotheses and Variablesmentioning

confidence: 99%

Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings

Ren¹,

Pérez-Soler²,

Castro

et al. 2022

IEEE Access

View full text Add to dashboard Cite

After improving the SOCIO chatbot prototype model, we wanted to know how/if its usability has changed. An evidence-based empirical evaluation of the usability of SOCIO V1 (updated version) requires an extensive verification of the experimental results. A family of experiments is a method of verification whereby we can check if the experimental results are reproducible. Through comparison with the updated control tool Creately, we aimed to gain a better understanding of the usability of the collaborative modeling chatbot and how it could be improved based on experimental evidence of changes in terms of efficiency, effectiveness, satisfaction, and quality. A total of 87 students from three countries were recruited. We conducted a family of three experiments to compare the usability of SOCIO V1 and updated Creately in academic settings. Students appeared to be more satisfied with SOCIO V1, and SOCIO V1 scored better on completeness. There were no significant differences between the two tools regarding efficiency and quality. This study provides evidence on how to employ a family of experiments to improve chatbot usability and enrich knowledge on chatbot usability experimentation.INDEX TERMS Chatbot, usability, family of experiments.

show abstract

Humanoid Robots and Conversational Recommender Systems: a Preliminary Study

Cited by 5 publications

References 13 publications

Using the SOCIO Chatbot for UML Modelling: A Family of Experiments

Using the SOCIO Chatbot for UML Modelling: A Family of Experiments

Towards Building Voice-based Conversational Recommender Systems: Datasets, Potential Solutions and Prospects

Using the SOCIO Chatbot for UML Modeling: A Second Family of Experiments on Usability in Academic Settings

Contact Info

Product

Resources

About