A close examination of the literature on including positively and negatively worded items in structured survey questionnaires revealed that contrary to the traditional wisdom, it is better not to use a mix of positively and negatively worded items because doing so can create threats to validity and reliability of the survey instrument. If mixing is done, it is recommended to use strategies derived from research to improve the quality of data and the instrument's validity and reliability.

In this article, the authors describe a 10-step procedure for conducting program evaluation in the human performance improvement (HPI) context, followed by a case study illustrating the procedure taken and outcomes produced. A team of graduate students at Boise State University completed an evaluation study of an administrative services qualification card program in a not-for-profit organization by following the 10-step evaluation procedure. The end result was a set of evidence-based recommendations focused on improving the quality of the program. 6