This paper describes a recently created image database, TID2013, intended for evaluation of full-reference visual quality assessment metrics. With respect to TID2008, the new database contains a larger number (3000) of test images obtained from 25 reference images, 24 types of distortions for each reference image, and 5 levels for each type of distortion. Motivations for introducing 7 new types of distortions and one additional level of distortions are given; examples of distorted images are presented. Mean Opinion Scores (MOS) for the new database have been collected by performing 985 subjective experiments with volunteers (observers) from five countries (Finland, France, Italy, Ukraine, and USA). The availability of MOS allows the use of the designed database as a fundamental tool for assessing the effectiveness of visual quality. Furthermore, existing visual quality metrics have been tested with the proposed database and the collected results have been analyzed using rank order correlation coefficients between MOS and considered metrics. These correlation indices have been obtained both considering the full set of distorted images and specific image subsets, for highlighting advantages and drawbacks of existing, state of the art, quality metrics. Approaches to thorough performance analysis for a given metric are presented to detect practical situations or distortion types for which this metric is not adequate enough to human perception. The created image database and the collected MOS values are freely available for downloading and utilization for scientific purposes

International audienc

Abstract-BACKGROUND -predicting defect-prone software components is an economically important activity and so has received a good deal of attention. However, making sense of the many, and sometimes seemingly inconsistent, results is difficult. OBJECTIVE -we propose and evaluate a general framework for software defect prediction that supports (i) unbiased and (ii) comprehensive comparison between competing prediction systems. METHOD -the framework comprises (i) scheme evaluation and (ii) defect prediction components. The scheme evaluation analyzes the prediction performance of competing learning schemes for given historical data sets. The defect predictor builds models according to the evaluated learning scheme and predicts software defects with new data according to the constructed model. In order to demonstrate the performance of the proposed framework, we use both simulation and publicly available software defect data sets. RESULTS -the results show that we should choose different learning schemes for different data sets (i.e. no scheme dominates), that small details in conducting how evaluations are conducted can completely reverse findings and lastly that our proposed framework is more effective, and less prone to bias than previous approaches. CONCLUSIONS -failure to properly or fully evaluate a learning scheme can be misleading, however, these problems may be overcome by our proposed framework.

No abstract

The subjective test for compressed visual content is typically conducted by few experts called golden eyes. Here, we attempt to characterize the visual experience on JPEG-coded images of ordinary people statistically. To achieve this goal, a new image quality database, MCL-JCI, is constructed and introduced in this work. We explain the test procedure and conduct a preliminary analysis on test results. It is observed that people can only differentiate a finite number of quality levels and the perceived quality plot is a stair function of the coding bit rate. The relationship between the perceived quality plot and image content is discussed.