The Impact of Surface Features on Choice of (in)Secure Answers by Stackoverflow Readers

Abstract

Existing research has shown that developers will use StackOverflow to answer programming questions: but what draws them to one particular answer over any other? The choice of answer they select can mean the difference between a secure application and insecure one, as the quality of supposedly secure answers can vary. Prior work has studied people posting on Stack Overflow-a two-way communication between the original poster and the Stack Overflow community. Instead, we study the situation of one-way communication, where people only read a Stack Overflow thread without being actively involved in it, sometimes long after a thread has closed. We report on a mixed-method study including a controlled between-groups experiment and qualitative analysis of participants' rationale (N=1188), investigating whether explanation detail, answer scoring, accepted answer marks, as well as the security of the code snippet itself affect the answers participants accept. Our findings indicate that explanation detail affects what answers participants reading a thread select (p<0.01), while answer score and acceptance do not (p>0.05)-the inverse of what research has shown for those asking and answering questions. The qualitative analysis of participants' rationale further explains how several cognitive biases underpin these findings. Correspondence bias, in particular, plays an important role in instilling readers with a false sense of confidence in an answer through the way it looks, regardless of whether it works, is secure, or if the community agrees with it. As a result, we argue that StackOverflow's use as a knowledge base by people not actively involved in threads-when there is only one-way-communication-may inadvertently contribute to the spread of insecure code, as the community's voting mechanisms hold little power to deter them from answers.

Developers Are Neither Enemies Nor Users: They Are Collaborators

“…If a developer struggles to understand how to use an API correctly (the miscommunication challenge in Table I), they may search Stack Overflow for solutions. The answers to Stack Overflow questions are voted and this may create herding behaviors (Table II), regardless of whether the answer they follow is correct or not [42]. The intangibility challenge represents a challenge with security as a whole, rather than being specific to any particular development practice.…”

“…Van der Linden et al conduct an observation of developer's use of Stack Overflow with 1,188 participants. They found developers go by surface features of Stack Overflow posts (such as answer length) over correctness [42]. Hidden information has a bearing on our characterization of shifting responsibility.…”

“…• Prior Beliefs -Developers exhibit a tendency to bring their own privacy and security beliefs into their development process. [2], [4], [10], [27], [28], [42], [43], [45], [46], [49], [52], [53] Herding…”

“…The incentives for inducing developers to spend the effort to program securely are also not aligned with expected returns from doing so [28], [48]. The challenges result in bias and group behavior [42], [52]. The improvisations to which developers resort are aided by online sources which can be insecure [43], [52], [59].…”

Crypto Experts Advise What They Adopt

“…For instance, profiling developer expertise contributes to heightening the members' awareness about the reliability of responses [5] [6]. In particular, platforms such as Stack Overflow contain insecure code snippets and inexperienced developers blindly use such snippets [7]. Due to the lack of secure code examples in cryptography, we hypothesize that mapping the activity of top crypto developers cross-platform can provide an interesting path to find and evaluate their practices from the security perspective, and present such results for developers who are looking for reliable, secure crypto examples.…”