While a number of algorithms for multiobjective reinforcement learning have been proposed, and a small number of applications developed, there has been very little rigorous empirical evaluation of the performance and limitations of these algorithms. This paper proposes standard methods for such empirical evaluation, to act as a foundation for future comparative studies. Two classes of multiobjective reinforcement learning algorithms are identified, and appropriate evaluation metrics and methodologies are proposed for each class. A suite of benchmark problems with known Pareto fronts is described, and future extensions and implementations of this benchmark suite are discussed. The utility of the proposed evaluation methods are demonstrated via an empirical comparison of two example learning algorithms.

Expert sport performers cope with a multitude of visual information to achieve precise skill goals under time stress and pressure. For example, a major league baseball or cricket batter must read opponent variations in actions and ball flight paths to strike the ball in less than a second. Crowded playing schedules and training load restrictions to minimise injury have limited opportunity for field-based practice in sports. As a result, many sports organisations are exploring the use of virtual reality (VR) simulators. Whilst VR synthetic experiences can allow greater control of visual stimuli, immersion to create presence in an environment, and interaction with stimuli, compared to traditional video simulation, the underpinning mechanisms of how experts use visual information for anticipation have not been properly incorporated into its content design. In themes, this opinion article briefly explains the mechanisms underpinning expert visual anticipation, as well as its learning and transfer, with a view that this knowledge can better inform VR simulator content design. In each theme, examples are discussed for improved content design of VR simulators taking into consideration its advantages and limitations relative to video simulation techniques. Whilst sport is used as the exemplar, the points discussed have implications for skill learning in other domains, such as military and law enforcement. It is hoped that our paper will stimulate improved content design of VR simulators for future research and skill enhancement across several domains.