Four types of experiential learning approaches used for predoctoral graduate students and postdoctoral scholars in the biomedical sciences are described and associated learning objectives and evaluation strategies are compared. This framework will help other institutions design and deliver experiential learning programs for career training.

Background: There has been a groundswell of national support for transparent tracking and dissemination of PhD career outcomes. In 2017, individuals from multiple institutions and professional organizations met to create the Unified Career Outcomes Taxonomy (UCOT 2017), a three-tiered taxonomy to help institutions uniformly classify career outcomes of PhD graduates. Early adopters of UCOT 2017, noted ambiguity in some categories of the career taxonomy, raising questions about its consistent application within and across institutions. Methods: To test and evaluate the consistency of UCOT 2017, we calculated inter-rater reliability across two rounds of iterative refinement of the career taxonomy, classifying over 800 PhD alumni records via nine coders. Results: We identified areas of discordance in the taxonomy, and progressively refined UCOT 2017 and an accompanying Guidance Document to improve inter-rater reliability across all three tiers of the career taxonomy. However, differing interpretations of the classifications, especially for faculty classifications in the third tier, resulted in continued discordance among the coders. We addressed this discordance with clarifying language in the Guidance Document, and proposed the addition of a flag system for identification of the title, rank, and prefix of faculty members. This labeling system provides the additional benefit of highlighting the granularity and the intersectionality of faculty job functions, while maintaining the ability to sort by - and report data on - faculty and postdoctoral trainee roles, as is required by some national and federal reporting guidelines. We provide specific crosswalk guidance for how a user may choose to incorporate our suggestions while maintaining the ability to report in accordance with UCOT 2017. Conclusions: Our findings underscore the importance of detailed guidance documents, coder training, and periodic collaborative review of career outcomes taxonomies as PhD careers evolve in the global workforce. Implications for coder-training and use of novice coders are also discussed.

Several reports have shown that doctoral and postdoctoral trainees in biomedical research pursue diverse careers that advance science meaningful to society. Several groups have proposed a three-tier career taxonomy to showcase these outcomes. This three-tier taxonomy will be a valuable resource for institutions committed to greater transparency in reporting outcomes, to not only be transparent in reporting their own institutional data but also to lend greater power to a central repository.

PhD-trained scientists are essential contributors to the workforce in diverse employment sectors that include academia, industry, government, and nonprofit organizations. Hence, best practices for training the future biomedical workforce are of national concern. Complementing coursework and laboratory research training, many institutions now offer professional training that enables career exploration and develops a broad set of skills critical to various career paths. The National Institutes of Health (NIH) funded academic institutions to design innovative programming to enable this professional development through a mechanism known as Broadening Experiences in Scientific Training (BEST). Programming at the NIH BEST awardee institutions included career panels, skill-building workshops, job search workshops, site visits, and internships. Because doctoral training is lengthy and requires focused attention on dissertation research, an initial concern was that students participating in additional complementary training activities might exhibit an increased time to degree or diminished research productivity. Metrics were analyzed from 10 NIH BEST awardee institutions to address this concern, using time to degree and publication records as measures of efficiency and productivity. Comparing doctoral students who participated to those who did not, results revealed that across these diverse academic institutions, there were no differences in time to degree or manuscript output. Our findings support the policy that doctoral students should participate in career and professional development opportunities that are intended to prepare them for a variety of diverse and important careers in the workforce.

Background: There has been a groundswell of national support for transparent tracking and dissemination of PhD career outcomes. In 2017, individuals from multiple institutions and professional organizations met to create the Unified Career Outcomes Taxonomy (UCOT 2017), a three-tiered taxonomy to help institutions uniformly classify career outcomes of PhD graduates. Early adopters of UCOT 2017, noted ambiguity in some categories of the career taxonomy, raising questions about its consistent application within and across institutions. Methods: To test and evaluate the consistency of UCOT 2017, we calculated inter-rater reliability across two rounds of iterative refinement of the career taxonomy, classifying over 800 PhD alumni records via nine coders. Results: We identified areas of discordance in the taxonomy, and progressively refined UCOT 2017 and an accompanying Guidance Document to improve inter-rater reliability across all three tiers of the career taxonomy. However, differing interpretations of the classifications, especially for faculty classifications in the third tier, resulted in continued discordance among the coders. We addressed this discordance with clarifying language in the Guidance Document, and proposed the addition of a flag system for identification of the title, rank, and prefix of faculty members. This labeling system provides the additional benefit of highlighting the granularity and the intersectionality of faculty job functions, while maintaining the ability to sort by - and report data on - faculty and postdoctoral trainee roles, as is required by some national and federal reporting guidelines. We provide specific crosswalk guidance for how a user may choose to incorporate our suggestions while maintaining the ability to report in accordance with UCOT 2017. Conclusions: Our findings underscore the importance of detailed guidance documents, coder training, and periodic collaborative review of career outcomes taxonomies as PhD careers evolve in the global workforce. Implications for coder-training and use of novice coders are also discussed.

PhD-trained scientists are essential contributors to the workforce in diverse employment sectors that include academia, industry, government, and non-profit organizations. Hence, best practices for training the future biomedical workforce are of national concern. Complementing coursework and laboratory research training, many institutions now offer professional training that enables career exploration and develops a broad set of skills critical to various career paths. The National Institutes of Health funded academic institutions to design innovative programming to enable this professional development through a mechanism known as Broadening Experiences in Scientific Training (BEST). Programming at the BEST awardee institutions included career panels, skill-building workshops, job-searching workshops, site visits, and internships. An initial concern was since doctoral training is lengthy and requires focused attention on dissertation research, having students participate in additional complementary training activities might lengthen time to degree and hamper student research productivity. To address this concern, using time to degree and publication records as measures of efficiency and productivity, metrics were analyzed from ten BEST awardee institutions. Comparing doctoral students who participated to those who did not, results revealed that across these diverse academic institutions, there were no differences in time to degree or manuscript output. Furthermore, a few institutions even demonstrated a positive correlation between participation in career and professional development activities and productivity. Our findings suggest that doctoral students should be encouraged to participate in career and professional development opportunities to ensure their preparedness for a variety of diverse and important careers in the workforce.Significance StatementOur study is unique in that it compiled doctoral degree durations at ten different universities, recorded individual participation in career and professional development activities in terms of dosage, and tracked individual engagement in real-time rather than relying on surveys sent to trainees after graduation. Participation in career and professional development activities, including internships, did not decrease efficiency or productivity. Our findings suggest that doctoral students should be encouraged to participate in career and professional development opportunities to ensure their preparedness for a variety of diverse and important careers in the workforce.

In the past year, there has been an exciting groundswell of national efforts to integrate multiple taxonomies for the transparent dissemination and analysis of PhD career outcomes. In this study, we leveraged the unique resources of the Broadening Experiences in Scientific Training Consortium to examine the reliability of the three-tiered Unified Career Outcomes Taxonomy (UCOT v.2017) that was collaboratively developed at a meeting convened by Rescuing Biomedical Research in August 2017. Using an amended version of the UCOT v.2017 (UCOT v.2017-rev1) and a new Supplementary Guidance document, we categorized over 570 PhD alumni records from three different universities. Utilizing Krippendorff's alpha to measure the interrater reliability from nine different individuals, we determined moderate to robust reproducibility within the first two tiers of the taxonomy (Workforce Sector and Career Type); however, the reliability for the third tier (Job Function) did not meet established standards. The team identified significant sources of error, revised category definitions, improved coder training materials and processes, and tested for improved reliability through coding 219 PhD alumni records using the revised taxonomy, UCOT v.2017-rev2. Our results revealed that the changes introduced in UCOT v.2017-rev2 improved inter-rater reliability in all three tiers, and either met or exceeded the acceptable standards for reliability. A final set of clarifications were made to UCOT v.2017-rev2, resulting in UCOT v.2018 and a Finalized Guidance document. Our findings underscore the importance of carefully developing guidance documents to aid coders in the reliable and consistent categorization of alumni career outcomes. We propose periodic assessment of the UCOT v.2018 to address the natural evolution of PhD careers in the global workforce. Ultimately, we hope that UCOT v.2018 will aid in the classification and dissemination of alumni career outcomes that is essential to educating trainees, institutions, and agencies about the diversity of career options for PhDs, and therein empower all PhDs to pursue the careers of their choice.

There is increasing awareness of the need for pre- and post-doctoral professional development and career guidance, however many academic institutions are only beginning to build out these functional roles. As a graduate career educator, accessing vast silos and resources at a university and with industry-partners can be daunting, yet collaboration and network development are crucial to the success of any career and professional development office. To better inform and direct these efforts, forty-five stakeholders external and internal to academic institutions were identified and interviewed to gather perspectives on topics critical to career development offices. Using a stakeholder engagement visualization tool developed by the authors, strengths and weaknesses can be assessed. General themes from interviews with internal and external stakeholders are discussed to provide various stakeholder subgroup perspectives to help prepare for successful interactions. Benefits include increased engagement and opportunities to collaborate, and to build or expand graduate career development offices.