Determination of the satisfaction attribute in usability tests using sentiment analysis and fuzzy logic

Abstract

With the growth in the number of applications deployed in cloud app stores, usability has become a fundamental attribute to ensure end-user productivity and enterprise competitiveness. According to ISO 9241-11, the three attributes that determine the usability of a product are effectiveness, efficiency and satisfaction, the latter being the subjective attribute of usability. In conventional user tests, the satisfaction attribute is determined using perception questionnaires, being a challenge to determine this attribute more objectively, given the limitations of surveys in terms of veracity and subjectivity. Based on the above, in this article we propose as a contribution, the construction of a system based on fuzzy logic for the estimation of the usability satisfaction attribute in user tests, which has as inputs both the level of satisfaction obtained from the answers to the post-test questionnaire, and the level of satisfaction obtained from the polarities of the opinions of the test users, determined by means of sentiment analysis techniques. The proposed fuzzy system is intended to serve as a reference to be replicated at the academic and enterprise level in the conduct of usability tests and specifically in the more objective determination of the satisfaction attribute based on the advantages of fuzzy logic and sentiment analysis techniques.

Herramienta de estimación de la confiabilidad de clientes potenciales del sector financiero mediante el uso de técnicas de análisis de sentimientos y lógica difusa

“…Al comparar este trabajo investigativo con similares [30], el cual es un trabajo que se enfoca en la determinación de los niveles de satisfacción en los test de usabilidad, se encuentra que la utilización de la lógica difusa y el análisis de sentimientos hacen buena combinación al momento de evaluar datos subjetivos y en la determinación de niveles de percepción. Ambas investigaciones usan un patrón metodológico similar, aunque se enfocan en contextos distintos, pero obteniendo resultados igualmente satisfactorios, por lo que se podría esperar que se realicen más investigaciones utilizando este patrón de análisis de sentimientos más lógica difusa para la determinación de niveles de percepción en otras áreas como investigación social, opinión pública, educación, entre otros.…”

Fuzzy System for Perception Level Estimation in E-Commerce Web Sites

“…While the rating variable can be directly obtained through web scraping techniques from the reviews made by users on the products and services, the sent_perc variable is calculated based on the polarities (negative, neutral and positive) belonging to the user's opinion regarding a specific product or service. Thus, for the calculation of the sent_perc variable, Equation (1) [21] is used.…”