Furnishing Your Room by What You See: An End-to-End Furniture Set Retrieval Framework with Rich Annotated Benchmark Dataset

Liu, Bingyuan; Zhang, Jiantao; Zhang, Xiaoting; Zhang, Wei; Yu, Cong; Zhou, Yuan

doi:10.48550/arxiv.1911.09299

Cited by 4 publications

(3 citation statements)

References 29 publications

(41 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these datasets are lack realism and semantic diversity. Recent synthetic datasets only release a few 3D data [18,20,28,42,54] or imagery data [32,33,35,36,55]. Most recently, Open-Rooms [34] release an interior synthetic dataset built upon publicly available real scanned datasets, but the diversity of scenes and objects are not as good as artist-created datasets such as SUNCG.…”

Section: Related Workmentioning

confidence: 99%

MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

Ren¹,

Zhang²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

With the rapid development of data-driven techniques, data has played an essential role in various computer vision tasks. Many realistic and synthetic datasets have been proposed to address different problems. However, there are lots of unresolved challenges: (1) the creation of dataset is usually a tedious process with manual annotations, (2) most datasets are only designed for a single specific task, (3) the modification or randomization of the 3D scene is difficult, and (4) the release of commercial 3D data may encounter copyright issue. This paper presents MINERVAS, a Massive INterior EnviRonments VirtuAl Synthesis system, to facilitate the 3D scene modification and the 2D image synthesis for various vision tasks. In particular, we design a programmable pipeline with Domain-Specific Language, allowing users to (1) select scenes from the commercial indoor scene database, (2) synthesize scenes for different tasks with customized rules, and (3) render various imagery data, such as visual color, geometric structures, semantic label. Our system eases the difficulty of customizing massive numbers of scenes for different tasks and relieves users from manipulating fine-grained scene configurations by providing user-controllable randomness using multi-level samplers. Most importantly, it empowers users to access commercial scene databases with millions of indoor scenes and protects the copyright of core data assets, e.g., 3D CAD models. We demonstrate the validity and flexibility of our system by using our synthesized data to improve the performance on different kinds of computer vision tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

Ren¹,

Zhang²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, MF tends to achieve higher recommendation accuracy than collaborative filtering and SVD because it is a learning-efficient method that refers only to data evaluated by a large number of users. Further, research focusing on furniture style compatibility uses the embedding of furniture images in a Euclidean space to classify the complex styles of each piece based on the outputs of multiple networks [8,9]. In this field, the Siamese network, a type of deep metric learning, optimizes the Euclidean distance between the embedding of each furniture image, keeping furniture of similar styles close and those of dissimilar styles further away in Euclidean space.…”

Section: Introductionmentioning

confidence: 99%

Furniture Recommendations Based on User Propensity and Furniture Style Compatibility

Takeda,

Ono,

Taisho

2024

IEEE Access

View full text Add to dashboard Cite

As digital information becomes more voluminous and e-commerce becomes more widespread, there is a growing demand for item recommendations that match the users' sensibilities. However, learning users' propensities is a difficult problem, especially in the field of furniture, which requires the consideration of many factors, such as color and shape. In addition, pieces of furniture should not be recommended only as stand-alone items, but must also be considered in terms of their affinity with other pieces, making the compatibility of styles among them an important factor. However, a consumer's furniture style is an ambiguous concept that is difficult to define. To reduce this ambiguity, Siamese networks are often used to estimate style compatibility by adding various features that represent styles, but even when they make use of alternative features associated with images, they are difficult to represent accurately. This paper proposes a method for recommending multiple pieces of furniture by learning style compatibility properties with a high degree of accuracy, taking users' preferences and styles' compatibility into account. To this end, we engaged in two tasks: (1) extracting users' preferences and (2) improving the accuracy of style suitability estimation. For (1), we applied matrix factorization to identify users whose sensitivities were close to those of the users who will receive recommendations. For (2), we used the Siamese network we have already proposed, which can learn from multiple furniture images simultaneously. Specifically, we propose a one-to-many input ratio to maintain high performance even when the input is ambiguous. Validation experiments were conducted for each task, and the results showed that the performance was improved; the actual recommendation results also showed a high performance.

show abstract

“…Meanwhile, style congruence is essential for furniture. A few studies have focused on furniture style compatibility [13][14][15], utilizing visual embeddings in Euclidian space to classify complex boundaries by outputs from multiple networks. In these studies, the Siamese network, a Deep Metric Learning method, was used to evaluate the compatibility of furniture styles.…”

Section: Introductionmentioning

confidence: 99%

Furniture Style Compatibility Estimation by Multi-Branch Deep Siamese Network

Taisho

Ono

Makihara

et al. 2022

MCA

View full text Add to dashboard Cite

As demands for understanding visual style among interior scenes increase, estimating style compatibility is becoming challenging. In particular, furniture styles are difficult to define due to their various elements, such as color and shape. As a result, furniture style is an ambiguous concept. To reduce ambiguity, Siamese networks have frequently been used to estimate style compatibility by adding various features that represent the style. However, it is still difficult to accurately represent a furniture’s style, even when using alternate features associated with the images. In this paper, we propose a new Siamese model that can learn from several furniture images simultaneously. Specifically, we propose a one-to-many ratio input method to maintain high performance even when inputs are ambiguous. We also propose a new metric for evaluating Siamese networks. The conventional metric, the area under the ROC curve (AUC), does not reveal the actual distance between styles. Therefore, the proposed metric quantitatively evaluates the distance between styles by using the distance between the embedding of each furniture image. Experiments show that the proposed model improved the AUC from 0.672 to 0.721 and outperformed the conventional Siamese model in terms of the proposed metric.

show abstract

Furnishing Your Room by What You See: An End-to-End Furniture Set Retrieval Framework with Rich Annotated Benchmark Dataset

Cited by 4 publications

References 29 publications

MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

Furniture Recommendations Based on User Propensity and Furniture Style Compatibility

Furniture Style Compatibility Estimation by Multi-Branch Deep Siamese Network

Contact Info

Product

Resources

About