Dramatic improvements in measuring genetic variation across agriculturally relevant populations (genomics) must be matched by improvements in identifying and measuring relevant trait variation in such populations across many environments (phenomics). Identifying the most critical opportunities and challenges in the area of genome to phenome (G2P) research is the focus of this paper. Previously (Tuggle et al. 2022), we laid out how Agricultural Genome to Phenome Initiative (AG2PI) will coordinate activities with US federal government agencies, including USDA, expand public–private partnerships, and engage with external stakeholders to achieve a shared vision of future the AG2PI. Acting on this latter step, AG2PI organized the “Thinking Big: Visualizing the Future of AG2PI” two-day workshop held September 9-10, 2022, in Ames, Iowa, co-hosted with the United State Department of Agriculture’s National Institute of Food and Agriculture (USDA NIFA). During the meeting, attendees were asked to use their experience and curiosity to review the current status of agricultural genome-to-phenome (AG2P) work and envision the future of the AG2P field. To tackle some of these big challenges, we designed this workshop to include diverse voices with the intention of hearing the different perspectives important to solving these challenges. The topic summaries composing this paper are distilled from two 1.5-hour small group discussions that included 6-8 participants, each representing a mix of crop researchers and livestock researchers, data scientists, social scientists, engineers, and other diverse stakeholders. Subsequent to the conference, additional writing, idea exchanges, and e-mails were shared among participants and the output of those conversations is summarized in this report. Challenges and solutions were identified across multiple topics at the workshop and are reported here. We end our discussion with a vision for the future of agricultural progress, identifying two areas of innovation needed: 1) innovate in genetic improvement methods development and evaluation and 2) innovate in agricultural research processes to solve societal problems. To address these needs, we then provide six specific goals that we recommend be implemented immediately in support of advancing AG2P research.