Finding the Minimum Document Length for Reliable Clustering of Multi-Document Natural Language Corpora

Why Molière most likely did write his plays

“…( 34 ) and the difficulty of working on short texts ( 35 ), length criteria deemed necessary to obtain reliable results in authorship attribution can vary, some authors seeming to achieve good results with texts under 1000 words ( 36 , 37 ), while recent systematic studies seem to advocate the study of more substantial texts ( 38 ).…”

“…To increase the reliability of the analyses, in a corpus with texts of varying length, we decided to select features based on the confidence level and margin of error that we could attain even for the smallest available sample in our corpus. The minimum sample size n was calculated using the following formula ( 38 )n=truep¯false(1−truep¯false)true(zetrue)2where truep¯ is the feature mean probability in our corpus, used as an estimate of the population probability π, z is the confidence level, and e is the margin of error of the probability estimate. We set z to obtain a confidence level above 90% and e = 2σ, where σ is the feature standard deviation in the corpus.…”

“…For this equation to be valid, the features need to be normally distributed among samples. Because this is not always the case, we correct for normality by generating a mirror variable ( 38 )vmirrorji=false(maxv+minvfalse)−vjiwhere v j is the vector of the feature j , max v and min v are the maximum and minimum values in the feature vector v j , and v ji is the probability estimate of the feature j in a sample i . We then computed the arithmetic mean between this mirror value and the original value: This way, any overestimation by one value is compensated by an equivalent underestimation by its mirror value.…”

Stylometry for Noisy Medieval Data: Evaluating Paul Meyer's Hagiographic Hypothesis

“…6). Texts that are too short create a problem of reliability, as the observed frequencies may not accurately represent the actual probability of a given variable's appearance (Moisl, 2011). To limit this issue, we removed texts below 1,000 words, a relatively low limit when compared to existing benchmarks (Eder, 2015(Eder, , 2017, but motivated by the necessity to not exclude too many texts.…”

“…Given the short length of the texts and the sparsity caused by noise, we implement a procedure to select for analysis only those features that satisfy a criterion of statistical reliability. In this, we follow the procedure suggested by Moisl (2011), in the implementation already used by Cafiero and Camps (2019). To summarize it, features are only retained if they match the desired confidence level and margin of error even for the smallest text in the corpus.…”