Understanding the Generalization Power of Overfitted NTK Models: 3-layer vs. 2-layer (Extended Abstract)

Ju, Peizhong; Lin, Xiaojun; Shroff, Ness B.

doi:10.1109/allerton49937.2022.9929374

Cited by 1 publication

(1 citation statement)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A major challenge here is that a single pre-trained model is unable to generalize well. We approached such a problem by leveraging a popular meta-learning framework of model-agnostic meta-learning (MAML), for which we first made several innovative contributions, including characterizing the optimization performance (Ji, Yang, and Liang 2021), understanding the statistical convergence guarantee (Collins et al 2022), and characterizing the generalization performance in the overparameterized regime (Ju, Liang, and Shroff 2023). Equipped with these powerful tools, we further demonstrated the advantages of MAML over traditional deep-learning techniques wherein a model retrained in the unseen test environment (i) uses a fraction of the data compared to classical retraining, which, in turn, simplifies data collection and storage, and (ii) results in equal or higher accuracy in optimal beam selection compared to the case when the new environment dataset is fully available during initial training.…”

Section: Meta-learning For Autonomous Transportationmentioning

confidence: 99%

AI‐EDGE: An NSF AI institute for future edge networks and distributed intelligence

Ju,

Li,

Liang

et al. 2024

AI Magazine

View full text Add to dashboard Cite

This paper highlights the overall endeavors of the NSF AI Institute for Future Edge Networks and Distributed Intelligence (AI‐EDGE) to create a research, education, knowledge transfer, and workforce development environment for developing technological leadership in next‐generation edge networks (6G and beyond) and artificial intelligence (AI). The research objectives of AI‐EDGE are twofold: “AI for Networks” and “Networks for AI.” The former develops new foundational AI techniques to revolutionize technologies for next‐generation edge networks, while the latter develops advanced networking techniques to enhance distributed and interconnected AI capabilities at edge devices. These research investigations are conducted across eight symbiotic thrust areas that work together to address the main challenges towards those goals. Such a synergistic approach ensures a virtuous research cycle so that advances in one area will accelerate advances in the other, thereby paving the way for a new generation of networks that are not only intelligent but also efficient, secure, self‐healing, and capable of solving large‐scale distributed AI challenges. This paper also outlines the institute's endeavors in education and workforce development, as well as broadening participation and enforcing collaboration.

show abstract

Section: Meta-learning For Autonomous Transportationmentioning

confidence: 99%