<title>Public automated web-based evaluation service for watermarking schemes: StirMark benchmark</title>

Abstract

One of the main problems, which darkens the future of digital watermarking technologies, is the lack of detailed evaluation of existing marking schemes. This lack of benchmarking of current algorithms is blatant and confuses rights holders as well as software and hardware manufacturers and prevents them from using the solution appropriate to their needs. Indeed basing longlived protection schemes on badly tested watermarking technology does not make sense.In this paper we will present the architecture of a public automated evaluation service we have developed for still images, sound and video. We will detail and justify our choice of evaluation profiles, that is the series of tests applied to different types of watermarking schemes. These evaluation profiles allow us to measure the reliability of a marking scheme to different levels from low to very high.Beside the known StirMark transformations, we will also detail new tests that will be included in this platform. One of them is intended to measure the real size of the key space. Indeed, if one is not careful, two different watermarking keys may produce interfering watermarks and as a consequence the actual space of keys is much smaller than it appears. Another set of tests is related to audio data and addresses the usual equalisation and normalisation but also time stretching, pitch shifting. Finally we propose a set of tests for fingerprinting applications. This includes: averaging of copies with different fingerprint, random exchange of part between different copies and comparison between copies with selection of most/less frequently used position differences. NEED FOR EVALUATIONThe growing number of attacks against watermarking systems (e.g., 1, 2, 3 ) has shown that far more research is required to improve the quality of existing watermarking methods so that, for instance, the coming JPEG 2000 (and new multimedia standards) can be more widely used within electronic commerce applications.We already pointed out that most papers have used their own limited series of tests, their own pictures and their own methodology and that consequently comparison was impossible without re-implementing the method and trying to test them separately 4 . But then, the implementation might be very different and probably weaker than the one of the original authors. This led to suggest that methodologies for evaluating existing watermarking algorithms were urgently required and we proposed a simple benchmark for still image marking algorithms.With a well-defined benchmark, researchers and watermarking software manufacturers would just need to provide a table of results, which would give a good and reliable summary of the performances of the proposed scheme. So end users can check whether their basic requirements are satisfied. Researchers can compare different algorithms and see how a method can be improved or whether a newly added feature actually improves the reliability of the whole method. As far as the industry is concerned, risks can be properly associated with the ...

Fast Robust Template Matching for Affine Resistant Image Watermarks

“…The algorithm fails for random geometric distortions since the FFT is severely distorted. However, to our knowledge, at this time no algorithm systematically decodes watermarks successfully after being attacked by the random geometric distortions implemented in the Stirmark3 [14] package.…”

“…We use the stirmark [14] program to evaluate the algorithm. The tests are divided into the following 8 sections: signal enhancement, compression, scaling, cropping, shearing, rotation, row/column removal, and random geometric distortions.…”

“…Since an exhaustive search leads to an intractable problem, we demonstrate how a careful pruning of the search space yiels to robust detection of transformations reasonable quickly. The proposed method is evaluated relative to the benchmark series of tests proposed by Kutter and Petitcolas [9] and implemented in the software package Stirmark3 [14]. The algorithm performs very well relative to the extensive series of tests implemented in the benchmark.…”

Data Hiding for Binary Documents Robust to Print-Scan, Photocopy and Geometric Distortions

“…Stirmark is a software used to test the robustness of watermarking techniques [21,22]. We carried on more exhaustive tests using documents distorted by Stirmark 4.0, release 129.…”

Evaluation of watermark robustness to JPEG2000 based content adaptation attacks

“…There have been some efforts on on the evaluation of watermarking technologies. For example with a given watermarked image, Stirmark [9] applies different attacks including cropping, filtering, rotation, JPEG compression to generate a number of modified images which are used to verify the existence of the watermark. Checkmark [8] performs the same job as Stirmark does and also evaluate and rate the watermarking schemes using different attacks including wavelet based compression.…”