Guidelines for improving the contextual relevance of field surveys: the case of information security policy violations

Abstract

The information systems (IS) field continues to debate the relative importance of rigor and relevance in its research. While the pursuit of rigor in research is important, we argue that further effort is needed to improve practical relevance, not only in terms of topics, but also by ensuring contextual relevance. While content validity is often performed rigorously, validated survey instruments may still lack contextual relevance and be out of touch with practice. We argue that IS behavioral research can improve its practical relevance without loss of rigor by carefully addressing a number of contextual issues in instrumentation design. In this opinion article, we outline five guidelines -relating to both rigor and relevance -designed to increase the contextual relevance of field survey research, using case examples from the area of IS security. They are: (1) inform study respondents that a behavior is an ISP violation, (2) measure specific examples of ISP violations, (3) ensure that ISP violations are important ISP problems in practice, (4) ensure the applicability of IS security violations to the organizational context, and (5) consider the appropriate level of specificity and generalizability for instrumentation. We review previous behavioral research on IS security and show that no existing study meets more than three of these five guidelines. By applying these guidelines where applicable, IS scholars can increase the contextual relevance of their instrumentation, yielding results more likely to address important problems in practice.

Intervention Effect Rates as a Path to Research Relevance: Information Systems Security Example

“…One of the earliest definitions in the area of insecure behaviors is Parker's (1976) definition of the term computer abuse. Later, Straub (1990) introduced it to ISS, and this paper's first author might be to blame for introducing the terms information security policy violations and employees' noncompliance with information security procedures (Siponen & Vance, 2014). …”

“…As a result, previous research put a premium on models in which the same reasons, such as fear of threat or sanctions, explain users' ISS behavior across all insecure behaviors (e.g., selecting a weak password, not locking a computer, etc.). Generic measures represent the most common method for measuring insecure behaviors in previous research (Siponen & Vance, 2014). These include evaluating responses to prompts, such as "I comply with the information security policies of my organization" (ibid).…”

Perceived Control and Privacy in a Professional Cloud Environment

“…To enable ease of design without sacrificing rigor, we implemented our research design within a professional cloud environment to match our target population [43]. We empirically tested our research hypotheses using the data collected through a survey that included items for the constructs specified in the model.…”

Seeing the forest and the trees: A meta-analysis of information security policy compliance literature

“…Specifically, we know little of the relative importance of the various predictors of security policy compliance, as the results differ across studies and research contexts. Some authors have attributed these differences to the inconsistent measurement of the policy compliance construct (i.e., actual vs. intended compliance; general vs. behavior-specific compliance [39]) and we investigate this issue through moderation tests within our meta-analysis.…”