An online benchmark system for image processing algorithms

Abstract

When we have to solve a specific problem, we can find various approaches and recommendations provided by individual partners. A fair competition among the partners can be reached only if there are possibilities to compare their results in objective ways. When the task relates to data processing, such an objective evaluation can be reached using ground truth through annotated publicly available datasets. Moreover, in case of available ground truth, efficient learning-based algorithms could enter the specific field. However, in several application fields such ground truth data are missing or incomplete making the participants report their results on own datasets. In this paper, we present a platform that will aid researchers in developing functional components (image processing algorithms) for specific (detection and classification) tasks by providing training data sets with ground truth/gold standard and with providing evaluation of the outputs of the algorithms in an objective manner under identical conditions on separate test data sets using standardized measures. This quantitative evaluation will allow the comparison and ranking of the algorithms with the same functionalities. Different participants will contribute datasets and data validation to the platform.

Data analysis applied to diabetic retinopathy screening: Performance evaluation

“…User registration is also supported, so the uploaded datasets and the corresponding screening results can be safely checked via a web browser if the uploader prefers this option. Though our focus is on mass screening, it is also possible to visualize the features (e.g., lesions found) involved in the decision within the system [12].…”

An Online Integrated Fingerprint Image System

“…In order to interact with the dataset and the database, the TensorFlow should be still run at the application server in our proposal. Toth et al [6], presented a platform that can help researchers to develop functional components for image processing algorithms for specific tasks by providing training data sets. Senthilkumar et al [7] proposed a new system that can be achieved by adopting web service based image processing system.…”