Evaluation of an alert system for selective dissemination of broadcast news

Abstract

This paper describes the evaluation of the system for selective dissemination of Broadcast News that we developed in the context of the European project ALERT. Each component of the main processing block of our system was evaluated separately, using the ALERT corpus. Likewise, the user interface was also evaluated separately. Besides this modular evaluation which will be briefly mentioned here, as a reference, the system can also be evaluated as a whole, in a field trial from the point of view of a potential user. This is the main topic of this paper. The analysis of the main sources of problems hinted at a large number of issues that must be dealt with in order to improve the performance. In spite of these pending problems, we believe that having a fully operational system is a must for being able to address user needs in the future in this type of service.