Usability Results from Human Performance Feedback and Development (HPFD) and ePerformance System Users: Technical Report #1

Abstract

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing this collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Department of Defense, Washington Headquarters Services, Directorate for Information , 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. REPORT DATE (DD-MM-YYYY) 02-15-2006 REPORT TYPE Technical Report DATES COVERED (From -To SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S)HPC-N5 SPONSOR/MONITOR'S REPORT NUMBER(S)Navy Human Performance Center Dam Neck Annex 2025 Tartar Ave, Suite 100 Virginia Beach, VA, 23461-1924 DISTRIBUTION / AVAILABILITY STATEMENTA -Approved for public release; distribution is unlimited SUPPLEMENTARY NOTES ABSTRACTThe Navy Human Performance Center (HPC) and Navy Personnel Command (NPC) have reached the final phase of development for the Navy's Human Performance Feedback and Development (HPFD) and ePerformance system. This second and final iteration of usability testing was designed to verify the functionality of system revisions made as a result of recommendations offered in Schwerin et al. (2004) and gather user perceptions of system usability and cultural issues that may affect HPFD and ePerformance system implementation. This report documents the results of usability test and user survey data collection. Usability timing and errors analyses suggest that system enhancements as a result of the first usability study improved the overall usability of the HPFD and ePerformance system, but users still have significant difficulty finding, opening and creating HPFD and ePerformance documents. SUBJECT TERMSHPFD, ePerformance, Navy, usability testing, Human Systems Integration (HSI) • HPFD tasks were completed more quickly and with fewer errors than had been observed in previous research.• Users of HPFD and ePerformance systems required the Quick Reference Guide (QRG) to locate, open, and create documents.• Tasks were completed much more efficiently and with fewer errors when using a shipboard NSIPS-dedicated NMCI server.• Supervisors and nonsupervisors experienced similar user problems.• Overall, users were more satisfied with the professionalism, efficiency, and overall effectiveness of the revised HPFD and ePerformance systems compared to the previous versions.These positive findings represent a dramatic departure from previous usability research, reflecting system enhancements made prior to the current usability study and th...

Multi-Iteration Usability Testing of the U.S. Navy's Performance Management System

“…The QRG was made available to HPFD and ePerformance usability testing participants at test sites 2 and 3 in Schwerin et al (2006) and all three sites of the current study. At test site 3 of Dean et al (2006), the QRG was further enhanced covering more features of the HPFD and ePerformance systems. …”

“…Results from analyses comparing data from Iteration 1 (reported in Schwerin, Dean, Robbins, Bourne, & Reed, 2006) and Iteration 2 (reported in Dean, Aspinwall, Schwerin, & Kendrick, 2006) show an overall reduction in user burden (e.g., fewer errors and less time to complete usability tasks) and increased user satisfaction (e.g., more satisfied with the professionalism, efficiency, and overall effectiveness). These findings indicate that the Navy HPFD and ePerformance systems are easier to use but, more generally, results support the value and effectiveness of usability in human systems integration (HSI) and usability testing.…”