Just-in-time learning (JITL) has recently been used for online soft sensor modeling. Different from traditional global manners, the JITL-based method exhibits a local model built from historical samples similar to a query sample so that both nonlinearity and changes of the process characteristics can be well coped with. A key issue in JITL is to establish a suitable similarity criterion to select relevant samples.Conventional JITL methods, which use distance-based similarity measure for local modeling, may be inappropriate for many industrial processes exhibiting time-varying and non-Gaussian behaviors. In this paper, a GMM-based similarity measure is proposed to improve the prediction accuracy of the JITL soft sensor. By taking the non-Gaussianity of process data and the characteristics of the query sample into account, a more suitable similarity criterion is defined for sample selection of JITL soft sensor and better modeling performance can be achieved. Case studies on a numerical example as well as an industrial process are demonstrated to evaluate the feasibility and effectiveness of the proposed method.