While Quality of Service (QoS) is a service quality indicator that is used as objective evaluation by service providers, Quality of Experience (QoE) is a quality indicator subjectively evaluated by users. However, subjective evaluation of QoE for every service imposes the users a great load. Therefore, it is needed to estimate QoE from a physiological index without every subjective evaluation. In this research, we focus on skin conductance activity (SCA) as a physiological index and develop an experimental scheme to clarify the relationship between QoE and SCA when abrupt playback pause degrades the video quality. As the results of experiment and data analysis, QoE and SCA are correlated differently according to the video categories such as sport and news.