Recognizing Human-Object Interaction in Multi-Camera Environments

Abstract

This work introduces Multi-Fusion Network for human-object interaction detection with multiple cameras. We present a concept and implementation of the architecture for a beverage refrigerator with multiple cameras as proof-of-concept. We also introduce an effective approach for minimizing the required amount of training data for the network as well as reducing the risk of overfitting, especially when dealing with a small data set that is commonly recorded by a person or small organization. The model achieved high test accuracy and comparable results in a real-world scenario at the Event Solutions in Hamburg 2019. Multi-Fusion Network is easy to scale due to shared learnable parameters. It is also lightweight, hence suitable to run on small devices with average computation capability. Furthermore, it can be used for smart home applications, gaming experiences, or mixed reality applications.