MOLTR: Multiple Object Localisation, Tracking, and Reconstruction from Monocular RGB Videos

ODAM: Object Detection, Association, and Mapping using Posed RGB Video

“…We use the Adam optimizer in Pytorch to optimize the logarithm of the posterior in Eq. ( 4) for 20 iterations for every 50 associated 2D obser- [24] and Vid2CAD [27] in four classes at IoU> 0.25 and all classes at IoU> 0.5 respectively.…”

“…The first category mostly extends existing 2D detectors to also output 3D bounding box from single images [23,26,28,33,54,58]. If a video sequence is available, the single-view 3D estimations can be fused using a filter or a LSTM to create a consistent mapping of the scene [5,20,24]. Yet, the fused 3D detections might not satisfy multi-view geometry constraints.…”

“…front and back of a chair) in a frameto-model fashion. Prior methods working for indoor environments resort to handcrafted matching by IoU [24] or SLAM feature point matching [18,60], whereas we learn the matching using the GNN. 3D detection from 2.5D and 3D input.…”

“…Detections from the single-view detector are matched to existing object instances in the map using an attentionbased GNN as opposed to handcrafted data association algorithms used in prior art [24,27]. The benefit of using a GNN for data association is twofold.…”

“…Nevertheless, this task of localizing objects and estimating their extents in 3D using RGB-only videos presents a number of challenges. First, despite the impressive success of deep learning methods for 2D object detectors [7,16,43], recent efforts that formulate 3D object mapping as a singleview 3D detection problem [5,24,33] suffer from accuracy due to the depth-scale ambiguity in the perspective projection (as demonstrated empirically in Sec. 4.2).…”