Tying benchmarks and metrics to evaluations and organizational performance

Abstract

Mission, vision, and objectives statements are standard items created for most information technology units. Alignment of these with both the overall University mission and individual staff performance goals is often weak or lacking. Building upon the work of Kohrman and Trinkle [1], objectives for Indiana State University's Instructional and Research Technology Services (IRTS) were written as facilitating activities and built to be SMART (Specific, Measurable, Aggressive but attainable, Rewarding, and Time-bound). In addition, these activities were mapped to the University's mission as well as specific individual performance goals.In the spring of 2004 the staff of IRTS held a one day retreat to develop activities that included benchmarks and metrics to determine organizational and individual success. These facilitating activities were captured into a document which included who was doing the work, who was responsible for work completion and success, how the activity was to be measured, what types of data would be collected, the benchmarks for success, and a timeline for completion.Each quarter of the academic year, the unit reflected on the progress toward year-end benchmarks and made adjustments of resources to ensure timely completion of facilitating activities. As IRTS moves into its second full year, the initial process of developing facilitating activities has helped guide the direction and growth of the unit. It has also aided in quantification of the unit's work and staff accountability which is documented in the Office of Information Technology's 2005 Technology Profile.

Instructional technology outreach and metrics

“…In conducting a literature review on metrics in instructional technology, the author found few papers on the topic, but one paper in particular had a nice quote: "Setting benchmarks and metrics requires time and commitment, but in the end, the fruits of that labor are obvious and help each unit prepare their story of their contributions to the University [2]. "…”

The EU AI Act: A pioneering effort to regulate frontier AI?

“…In this context, computational resources are often considered a useful node for AI governance. Since there is a strong correlation between the amount of compute used for training and the capabilities of the resulting model [26], detecting where large amounts of compute are being used may enable governments to develop early awareness of which actors are likely to be developing and deploying highly capable systems [27]. This could be possible if developers report their training resources or if regulators monitor training runs or even the semiconductor supply chain [28].…”