We present a deep metadata fusion approach that connects image data and heterogeneous metadata inside a Convolutional Neural Network (CNN). This approach enables us to assign all relevant traffic lights to their associated lanes. To achieve this, a common CNN topology is trained by down-sampled and transformed input images to predict an indication vector. The indication vector contains the column positions of all the relevant traffic lights that are associated with lanes. In parallel, we fuse prepared and adaptively weighted Metadata Feature Maps (MFM) with the convolutional feature map input of a selected convolutional layer. The results are compared to rule-based, only-metadata, and only-vision approaches. In addition, human performance of the traffic light to ego-vehicle lane assignment has been measured by a subjective test. The proposed approach outperforms all other approaches. It achieves about 93.0% average precision for a real-world dataset. In a more complex dataset, 87.1% average precision is achieved. In particular, the new approach reaches significantly higher results with 93.7% to 91.0% average accuracy for a real-world dataset in contrast to lower human performance.
erzielen die gemessene menschliche Leistung und der tiefe Metadatenfusionsansatz eine nahezu identische Richtig-Positive Rate von 92,9 ±1,3 %. Außerdem wird festgestellt, dass eine frühe Fusion der Metadaten am effektivsten ist und alle fusionierten Metadatenmerkmalkarten sich positiv auf die Ergebnisse auswirken. Ferner hat sich gezeigt, dass der beste Fusionsoperator die elementweise Multiplikation ist. In Anlehnung an die menschliche Wahrnehmung ist festzustellen, dass sich die Ergebnisse mit sinkendem Abstand zur Haltelinie steigern.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.