Learning Material-Aware Local Descriptors for 3D Shapes

Lin, Hubert; Averkiou, Melinos; Kalogerakis, Evangelos; Kovács, Balázs; Ranade, Siddhant; Kim, Vladimir G.; Chaudhuri, Siddhartha; Bala, Kavita

doi:10.1109/3dv.2018.00027

Cited by 8 publications

(2 citation statements)

References 43 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although geometric information is generated, appearance properties are not accounted for. This needs to be addressed since existing structural and geometric attributes provide strong cues to material and texture appearance, something that has been under‐explored at the object level [JTRS12, LAK*18, CXY*15]. Learning to model a coupling of this with the scene layouts is an interesting approach to bypass the typical object retrieval step and directly generate objects with novel appearances.…”

Section: Conclusion and Open Problemsmentioning

confidence: 99%

Advances in Data‐Driven Analysis and Synthesis of 3D Indoor Scenes

Patil,

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

This report surveys advances in deep learning‐based modelling techniques that address four different 3D indoor scene analysis tasks, as well as synthesis of 3D indoor scenes. We describe different kinds of representations for indoor scenes, various indoor scene datasets available for research in the aforementioned areas, and discuss notable works employing machine learning models for such scene modelling tasks based on these representations. Specifically, we focus on the analysis and synthesis of 3D indoor scenes. With respect to analysis, we focus on four basic scene understanding tasks – 3D object detection, 3D scene segmentation, 3D scene reconstruction and 3D scene similarity. And for synthesis, we mainly discuss neural scene synthesis works, though also highlighting model‐driven methods that allow for human‐centric, progressive scene synthesis. We identify the challenges involved in modelling scenes for these tasks and the kind of machinery that needs to be developed to adapt to the data representation, and the task setting in general. For each of these tasks, we provide a comprehensive summary of the state‐of‐the‐art works across different axes such as the choice of data representation, backbone, evaluation metric, input, output and so on, providing an organized review of the literature. Towards the end, we discuss some interesting research directions that have the potential to make a direct impact on the way users interact and engage with these virtual scene models, making them an integral part of the metaverse.

show abstract

Section: Conclusion and Open Problemsmentioning

confidence: 99%

Advances in Data‐Driven Analysis and Synthesis of 3D Indoor Scenes

Patil,

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Additionally, the ability to infer physical properties e.g. material [35,18], mass [35,36] etc. can further make this process accurate.…”

Section: Related Workmentioning

confidence: 99%

Where2Act: From Pixels to Actions for Articulated 3D Objects

Mo¹,

Guibas²,

Mukadam³

et al. 2021

Preprint

View full text Add to dashboard Cite

One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment. In this paper, we take a step towards that long-term goalwe extract highly localized actionable information related to elementary actions such as pushing or pulling for articulated objects with movable parts. For example, given a drawer, our network predicts that applying a pulling force on the handle opens the drawer. We propose, discuss, and evaluate novel network architectures that given image and depth data, predict the set of actions possible at each pixel, and the regions over articulated parts that are likely to move under the force. We propose a learning-from-interaction framework with an online data sampling strategy that allows us to train the network in simulation (SAPIEN) and generalizes across categories. But more importantly, our learned models even transfer to real-world data. Check the project website for the code and data release.

show abstract