Knowledge Condensation Distillation

Li, Chenxin; Lin, Mingbao; Ding, Zhiyong; Lin, Na; Zhuang, Yihong; Huang, Yue; Ding, Xinghao; Cao, Liujuan

doi:10.1007/978-3-031-20083-0_2

Cited by 10 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the evaluation of the final rendered images by GS, PSNR, SSIM, and LPIPS are employed. All metrics are calculated on the test set and averaged across all scenarios and embedded images [7,26,34]. For subjective evaluation, 360-degree rotational videos of 3D Gaussians generated by different methods are rendered for a collection of 30 images.…”

Section: Methodsmentioning

confidence: 99%

“…In this paper, we introduce EndoSparse, a framework enabling efficient reconstruction and rendering of endoscopic scenes from sparse observations. EndoSparse enhances 3D-GS scene reconstruction by distilling [15,19] geometric and appearance priors from pretrained foundation models. Specifically, the optimization of 3D-GS is designed to obey the data distribution with large-scale pre-trained generative models.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Identity of university Chinese heritage language learners in Hong Kong

Li¹,

李蓁²

View full text Add to dashboard Cite

The core of out-of-distribution (OOD) detection is to learn the in-distribution (ID) representation, which is distinguishable from OOD samples. Previous work applied recognition-based methods to learn the ID features, which tend to learn shortcuts instead of comprehensive representations. In this work, we find surprisingly that simply using reconstruction-based methods could boost the performance of OOD detection significantly. We deeply explore the main contributors of OOD detection and find that reconstructionbased pretext tasks have the potential to provide a generally applicable and efficacious prior, which benefits the model in learning intrinsic data distributions of the ID dataset. Specifically, we take Masked Image Modeling as a pretext task for our OOD detection framework (MOOD). Without bells and whistles, MOOD outperforms previous SOTA of one-class OOD detection by 5.7%, multi-class OOD detection by 3.0%, and near-distribution OOD detection by 2.1%. It even defeats the 10-shot-per-class outlier exposure OOD detection, although we do not include any OOD samples for our detection.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%