Assessing the Credibility of Grey Literature: A Study with Brazilian Software Engineering Researchers

Abstract

In recent years, the use and investigations about Grey Literature (GL) increased, in particular, in Software Engineering (SE) research. However, its understanding is still scarce and sometimes controversial, such as interpreting GL types and assessing their credibility. This study aimed to understand the credibility aspects that SE researchers consider in assessing GL and its types. To achieve this goal, we surveyed 53 SE researchers (who answered that they have used GL in our previous investigation), receiving a total of 34 valid responses. Our main findings show that: 1) GL source produced or cited by a renowned source is the main credibility criteria used to assess GL, 2) most of the GL types tend to have a Low to Moderate level of Control and Expertise, 3) there is a positive statistical correlation between the level of Control and Expertise for most GL types, and 4) the different respondent profiles shared similar opinions about the credibility criteria. Our investigation contributes to helping future SE researchers that intend to use GL with more credibility. Additionally, shows the need for future studies to better understand the GL types in SE research.

Generative Artificial Intelligence, Python, and Gathering Grey Literature for a Systematic Literature Review with Google’s Programmable Search Engine.

“…Introduced by [11], grey information serves as a classi cation encompassing various forms of 'grey'; including data, literature, and information. In this context, 'grey data' pertains to user-generated web content like tweets or blogs, 'grey literature' encompasses policy documents, standards, and regulations, while 'grey information' involves informal data such as meeting notes or emails [12,13]. This is not the rst instance of creating ways to systematically gather grey literature for use in a SLR.…”