Modeling and corpus methods in experimental philosophy

Abstract

Research in experimental philosophy has increasingly been turning to corpus methods to produce evidence for empirical claims, as they open up new possibilities for testing linguistic claims or studying concepts across time and culture. The present article reviews the quasi‐experimental studies that have been done using textual data from corpora in philosophy, with an eye for the modeling and experimental design that enable statistical inference. I find that most studies forego comparisons that could control for confounds, and that only a little less than half employ statistical testing methods to control for chance results. Furthermore, at least some researchers make modeling decisions that either do not take into account the nature of corpora and of the word‐concept relationship, or undermine the experiment's capacity to answer research questions. I suggest that corpus methods could both provide more powerful evidence and gain more mainstream acceptance by improving their modeling practices.

Twenty years of experimental philosophy research

“…On the other hand, it would be beneficial to amass evidence for theories from multiple perspectives, such as intuitions, considered judgments, usage, corpus data, and so on (Devitt 2012). In this regard, we think that besides adopting research methods commonly used in psychology and cognitive science to pump out intuitions from ordinary people or philosophers, researchers could turn their attention to various linguistic corpora that are readily available and to the employment of techniques in corpus linguistics as well as advanced natural language processing techniques in computer science, as these methods have great potential to generate evidence for various philosophical arguments and claims and to facilitate studies of philosophical concepts across time and cultures (Chartrand 2022). Certainly, whatever research design experimental philosophers adopt, greater efforts should be made to improve the reliability and validity of their experimental endeavors, including both the experimental designs and the statistical analyses.…”

Introduction: Whither (Experimental) Philosophy of Medicine?

“…Th e culture-boundedness and historicity of many diseases and disorders may also require a somewhat diff erent approach from experimental philosophers. Corpus analysis, for instance, is already oft en used in experimental philosophy (Chartrand 2022) because it allows studying concepts across time and cultures and thus seems especially suited for XPhiMed. Also, more qualitative methods that investigate experiential aspects could prove fruitful.…”

Knowledge, the concept know, and the word know: considerations from polysemy and pragmatics

“…We can study this mental object using experimental methods (including psycholinguistic and neurolinguistic approaches in addition to armchair investigations) where variables are manipulated. We can also use observational or pseudo-experimental methods such as corpus analyses (Chartrand 2022) to support hypotheses about the properties of expressions like know. But as we'll see, such analyses prove more useful for the study of the linguistic expression know than the concept KNOW, given practical considerations pertinent to the range of different behaviors that each one generates.…”

“…The contemporary incarnation of this methodological camp contends that philosophical inquiry is better served by attending to the ordinary use of language. This has manifested in a number of ways, many of which take cues from methods (and findings) in the social sciences (see for example Hansen & Chemla 2015, Semler & Henne 2019, Ventura 2019, Chartrand 2022). Because the guiding principle of ordinary language philosophy insists that ordinary language usage is more reflective of our linguistic and conceptual competence, these interdisciplinary sources invoke methods that utilize language.…”