SUMMARYScience gateways are digital interfaces to advanced technologies that support science/engineering research/ education. Frequently implemented as Web and mobile applications, they provide access to community resources such as software, data, collaboration tools, instrumentation, and high-performance computing. We anticipate opportunities for growth within a fragmented community. Through a large-scale survey, we measured the extent and characteristics of the gateway community (reliance on gateways and nature of existing resources) to understand useful services and support for builders and users. We administered an online survey to nearly 29,000 principal investigators, senior administrators, and people with gateway affiliations. Nearly 5000 respondents represented diverse expertise and geography. The majority of researchers/educators indicated that specialized online resources were important to their work. They choose technologies by asking colleagues and looking for documentation, demonstrated reliability, and technical support; adaptability via customizing or open-source standards was another priority. Research groups commonly provide their own resources, but public/academic institutions and commercial services also provide substantial offerings. Application creators and administrators welcome external services providing guidance such as technology selection, sustainability planning, evaluation, and specialized expertise (e.g., quality assurance and design). Technologies are diverse, so flexibility and ongoing community input are essential, as is offering specific, easy-to-access training, community support, and professional development.

Under normal circumstances, when students invest more effort in their schoolwork, they generally show evidence of improved academic achievement.  But when universities abruptly transitioned to remote instruction in Spring 2020, instructors assigned rapidly-prepared online learning activities, disrupting the normal relationship between effort and outcomes.  In this study, we examine this relationship using data observed from a large-scale survey of undergraduate students, from logs of student activity in the online learning management system, and from students’ estimated cumulative performance in their courses (n = 4,636).  We find that there was a general increase in the number of assignments that students were expected to complete following the transition to remote instruction, and that students who spent more time and reported more effort carrying out this coursework generally had lower course performance and reported feeling less successful.  We infer that instructors, under pressure to rapidly put their course materials online, modified their courses to include online busywork that did not constitute meaningful learning activities, which had a detrimental effect on student outcomes at scale.  These findings are discussed in contrast with other situations when increased engagement does not necessarily lead to improved learning outcomes, and in comparison with the broader relationship between effort and academic achievement.

What is cyberinfrastructure? • Term popularized by US National Science Foundation, defined by IU as: "Cyberinfrastructure consists of computing systems, data storage systems, advanced instruments and data repositories, visualization environments, and people, all linked together by software and high performance networks to improve research productivity and enable breakthroughs not otherwise possible." [1]* • Very similar to the European concept of e-Science but explicitly includes people and human expertise as part of the infrastructure • Cloud and utility computing are specific forms of computing and storage resources included in the general term "cyberinfrastructure." 3 *References at end of slides

With the rise of science gateway use in recent years, we anticipate there are additional opportunities for growth, but the field is currently fragmented. We describe our efforts to measure the extent and characteristics of the gateway community through a large-scale survey. Our goal was to understand what type of support services might be provided to the gateway community.

In recent years, considerable attention has been given to assessing the value of investments in cyberinfrastructure (CI). This paper includes a survey of current methods for the assessment of financial returns on investment (ROI) in CI. Applying the financial concept of ROI proves challenging with regard to a service that, in most academic environments, does not generate a "sold amount" such as one would find in the buying and selling of stocks. The paper concludes with a discussion of future research directions and challenges in the assessment of financial ROI in CI. This work is intended less as a definitive guide than as a starting point for further exploration in the assessment of CI's value for scientific research.