Hierarchical Attention Network for Visually-Aware Food Recommendation

Gao, Xiaoyan; Feng, Fuli; He, Xiangnan; Huang, Heyan; Guan, Xinyu; Feng, Chong; Ming, Zhaoyan; Chua, Tat-Seng

doi:10.1109/tmm.2019.2945180

Cited by 66 publications

(19 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A few of the approaches proposed include user information into the recommendation procedures (i.e., collaborative filtering). Still, they only considered similar users based on the overlapping rated recipes, ignoring the relational information among users, recipes, or ingredients (Freyne and Berkovsky, 2010 ; Forbes and Zhu, 2011 ; Ge et al, 2015 ; Vivek et al, 2018 ; Khan et al, 2019 ; Gao et al, 2020 ). For example, Yang et al ( 2017 ) developed a framework to learn food preference based on the item-wise and pairwise recipe image comparisons.…”

Section: Related Workmentioning

confidence: 99%

“…Existing recipe recommendation approaches are mostly based on the similarity between recipes (Yang et al, 2017 ; Chen et al, 2020 ). A few of the approaches tried to take the user information into account (Freyne and Berkovsky, 2010 ; Forbes and Zhu, 2011 ; Ge et al, 2015 ; Vivek et al, 2018 ; Khan et al, 2019 ; Gao et al, 2020 ), but they only defined similar users based on the overlapping rated recipes between users, while ignoring the relational information between users, recipes, or ingredients. Nevertheless, user preference toward food is complex.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Recipe Recommendation With Hierarchical Graph Attention Network

et al. 2022

View full text Add to dashboard Cite

Recipe recommendation systems play an important role in helping people find recipes that are of their interest and fit their eating habits. Unlike what has been developed for recommending recipes using content-based or collaborative filtering approaches, the relational information among users, recipes, and food items is less explored. In this paper, we leverage the relational information into recipe recommendation and propose a graph learning approach to solve it. In particular, we propose HGAT, a novel hierarchical graph attention network for recipe recommendation. The proposed model can capture user history behavior, recipe content, and relational information through several neural network modules, including type-specific transformation, node-level attention, and relation-level attention. We further introduce a ranking-based objective function to optimize the model. Thorough experiments demonstrate that HGAT outperforms numerous baseline methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Recipe Recommendation With Hierarchical Graph Attention Network

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Table 1 summarizes the statistics of the datasets. One, called Allrecipes, was crawled from Allrecipes.com by Gao et al in [4]. Each of the interactions in Allrecipes represents that the user has tried this recipe.…”

Section: Experiments 41 Experimental Setupmentioning

confidence: 99%

Comparative Study of Adversarial Training Methods for Cold-Start Recommendation

Meng

et al. 2021

Proceedings of the 1st International Workshop on Adversarial Learning for Multimedia

View full text Add to dashboard Cite

Adversarial training in recommendation is originated to improve the robustness of recommenders to attack signals and has recently shown promising results to alleviate cold-start recommendation. However, existing methods usually should make a trade-off between model robustness and performance, and the underlying reasons why using adversarial samples for training works has not been sufficiently verified. To address this issue, this paper identifies the key components of existing adversarial training methods and presents a taxonomy that defines these methods using three levels of components for perturbation generation, perturbation incorporation, and model optimization. Based on this taxonomy, different variants of existing methods are created, and a comparative study is conducted to verify the influence of each component in cold-start recommendation. Experimental results on two benchmarking datasets show that existing state-of-the-art algorithms can be further improved by a proper pairing of the key components as listed in the taxonomy. Moreover, using case studies and visualization, the influence of the content information of items on cold-start recommendation has been analyzed, and the explanations for the working mechanism of different components as proposed in the taxonomy have been offered. These verify the effectiveness of the proposed taxonomy as a design paradigm for adversarial training.

show abstract

“…We evaluate our proposed method on the same dataset as proposed in [30]. We choose this public dataset because it has various types of non-stationary noise and we can compare our results with other published work.…”

Section: A Datasetmentioning

confidence: 99%

“…To further preserve linguistic information and capture contextual relationship in enhancing process, a multi-attention mechanism is employed in U-shape generators, which consist of encoding layers, transformation blocks, and decoding layers. Attention mechanism can compute the long-range relative dependencies among elements in sequences [28], which has been widely used both in the computer vision field and speech area [29], [30]. The proposed multi-attention uses attention mechanism in two different ways: attention gates in U-net [31] encoding-decoding layers (AU gate) and self-attention [32] in dilated residual networks [33] (DRN-SA block).…”

Section: Introductionmentioning

confidence: 99%

Improved Relativistic Cycle-Consistent GAN With Dilated Residual Network and Multi-Attention for Speech Enhancement

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Generative adversarial networks (GANs) have been increasingly used as feature mapping functions in speech enhancement, in which the noisy speech features are transformed to the clean ones through the generators. This paper proposes a novel speech enhancement model based on a cycle-consistent relativistic GAN with Dilated Residual Networks and a Multi-attention mechanism. Using the adversarial loss, improved cycle-consistency losses, and an identity-mapping loss, a noisy-to-clean generator G and an inverse clean-to-noisy generator F simultaneously learn the forward and backward mappings between the source and target domains. To guarantee the stability of the training process, we replace vanilla GAN loss with relativistic average GAN loss and use spectral normalization in discriminators so that they conform to Lipschitz continuity. Furthermore, we employ two attention-based components as multi-attention mechanism to reduce importing signal distortion: attention U-net gates and dilated residual self-attention blocks. By employing these components, our proposed generators can capture long-term inner dependencies between elements of speech features and further preserve linguistic information. Experimental results on a public dataset indicate that the proposed model achieves state-of-the-art speech enhancement performance, especially in reducing speech distortion and improving signal overall quality. Compared with the representative GAN-based approaches, the proposed method significantly achieves the best performance in terms of STOI, CSIG, COVL, and CBAK objective metrics. Moreover, we demonstrate the contribution of each proposed component including relativistic average loss, attention U-net gate, self-attention layers, spectral normalization, and dilation operation by ten comparison systems. INDEX TERMS speech enhancement, cycle-consistent GAN, relativistic average loss, multi-attention, dilated residual network, U-net.

show abstract

Hierarchical Attention Network for Visually-Aware Food Recommendation

Cited by 66 publications

References 47 publications

Recipe Recommendation With Hierarchical Graph Attention Network

Recipe Recommendation With Hierarchical Graph Attention Network

Comparative Study of Adversarial Training Methods for Cold-Start Recommendation

Improved Relativistic Cycle-Consistent GAN With Dilated Residual Network and Multi-Attention for Speech Enhancement

Contact Info

Product

Resources

About