Instantiation Validity in IS Design Research

Artifact Sampling: Using Multiple Information Technology Artifacts to Increase Research Rigor

“…However, as argued in [19], [38], many ITDR research questions deal with situations where it is unclear how to design an artifact and many (and sometimes potentially an unlimited number of) design choices exist. Indeed, the notion of a potentially vast space of possible operationalizations is recognized in other disciplines [51], and we believe it should at least be considered in ITDR, especially during the process of designing and evaluating artifacts.…”

“…For example, perhaps in the initial operationalization of the design, providing a definitive theoretical justification for design choices is impractical, and, further studies can help refine and confirm the validity of the design choices e.g., [54]. We believe this approach is also sound, but note that much of ITDR research has been criticized for the lack of extensive replication and some question whether a cumulative research tradition is even possible when dealing with ever changing IT artifacts [19]. Future studies should explore in greater detail when artifact sampling is more effective and epistemically appropriate, and when other strategies should be pursued.…”

“…These concerns have resulted in a proposal for a new kind of research validity [18] validity (IV) -defined as the extent to which inferences and conclusions are warranted from observations of features of IT as instantiations of theoretical constructs or design principles [19]. Instantiation validity is made of inner instantiation validity and outer instantiation validity.…”

Guidelines for Establishing Instantiation Validity in IT Artifacts: A Survey of IS Research

“…While Lukyanenko et al [2] discuss IV in the context of IS theory testing, we believe this concept also applies to other types of knowledge contribution in IS research. Instantiation validity plays a role in evaluating design knowledge when this knowledge is used to instantiate properties of an IT artifact.…”

“…They conducted an experiment demonstrating that different levels of perceived personalization and familiarity engender different levels of intention to adopt. According to Lukyanenko et al [2], the validity of this conclusion (i.e., that personalization of an IS leads to its increased adoption) "depends critically on whether the chosen artifacts faithfully instantiated the underlying theoretical construct of personalization and levels thereof" (p. 322). Lukyanenko et al introduce the notion of instantiation validity (IV) to denote "validity of IT artifacts as instantiations of theoretical constructs" (ibid).…”