Validation Guidelines for IS Positivist Research

Abstract

The issue of whether IS positivist researchers were validating their instruments sufficiently was initially raised fifteen years ago. Rigor in IS research is still one of the critical scientific issues facing the field. Without solid validation of the instruments that are used to gather data on which findings and interpretations are based, the very scientific basis of the profession is threatened. This study builds on four prior retrospectives of IS research that conclude that IS positivist researchers continue to face major barriers in instrument, statistical, and other forms of validation. It goes beyond these studies by offering analyses of the state-of-the-art of research validities and deriving specific heuristics for research practice in the validities. Some of these heuristics will, no doubt, be controversial. But we believe that it is time for the IS academic profession to bring such issues into the open for community debate. This article is a first step in that direction. Based on our interpretation of the importance of a long list of validities, this paper suggests heuristics for reinvigorating the quest for validation in IS research via content/construct validity, reliability, manipulation validity, and statistical conclusion validity. New guidelines for validation and new research directions are offered.

The mediating role of the dimensions of the perceived risk in the effect of customers’ awareness on the adoption of Internet banking in Iran

“…That is to say, the indicators selected for a given construct are considered together to identify the erroneous indicators of the construct. In fact, reliability is used to investigate the coordination of indicators with the constructs with which they must be measured [75].…”

The relationship between website quality, trust and price premiums at online auctions

“…Two survey pretests were conducted before the main empirical study to enhance the content validity of the proposed survey items [35,83]. The first survey pretest was Loiacono et.…”

Students’ communicative behavior adaptability in CSCL environments

“…Straub et al (2004) have discussed that convergent validity is evidenced when items highly correlate with one another. According to Gefen and Straub (2005) the convergent validity is shown when at Alpha level of 0.05, t-value of all items of a factor is significant (t-value greater than 1.96).…”

“…The discriminant validity examines whether the items that represent a factor differ from those that are not believed to make up the factor (Straub et al 2004). To assess existence of discriminant validity, Gefen and Straub (2005) have mentioned that two criteria must be tested:…”

“…The content validity deals with the question: "does the instrumentation (e.g. questionnaire items) pull in a representative manner from all of the ways that could be used to measure the content of given construct" (Straub et al 2004). Straub and colleagues have pointed out that the content validity is established through the opinion of experts as well as the literature review.…”