Why readability formulas fail

Bruce, Bertram C.; Rubin, Andee; Starr, Kathleen

doi:10.1109/tpc.1981.6447826

Cited by 78 publications

(62 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In fact, readability formulas have long been discredited as a way to guide revision or to evaluate writing. 2,[9][10][11][12] Consider the fact that a text and the same text written backwards, making it meaningless, have the same reading grade level.…”

Section: Why the Rule Won't Necessarily Improve Writingmentioning

confidence: 99%

Why scientific writers should study writing scientifically: the value of research into written communication

2017

ESE

View full text Add to dashboard Cite

In the November 2016 EASE Newsletter, Julian Venables proposed "Five golden rules for writing a good scientific paper. "1 Rule #5 is "Be clear -sentences should be less than 25 words long. " He continues: "The 20 + 10 rule says that for each 10 words above 20 a sentence becomes twice as hard to read. So for a 70 word sentence, which is not uncommon for francophone scientists, there is an 'excess of 50 words' ie 5 extra 10s, or a 2x2x2x2x2 = a 32-fold increase in difficulty!" [Spelling and punctuation are as published.] Unfortunately, Rule #5 won't necessarily improve writing.2-6 It's actually a myth, and one that harms us professionally when we perpetuate it.7 I don't mean to be critical of Dr Venables. Most of my fellow medical writers and clients who are nonnative English speakers are also unaware of the research base and advanced skills that are a part of our profession. I learned them in the 1970s, when I was trained to be a technical writer by technical writers. My sense is that people entering the profession of medical writing in mid career do not get the same orientation or training. This lack of awareness is curious, if not disturbing, for another reason however. If physicians study medicine, psychologists study psychology, and historians study history, why don't writers study writing? Why the rule won't necessarily improve writingThe Rule is puzzling. What does it mean if a text is "twice as hard to read?" What is a "32-fold increase in difficulty?" I suspect that "reading difficulty" is the concept of "reading grade level, " which is the outcome of some readability formulas. The usual understanding of reading grade level is that a text written at, say, an 8th-grade level can be understood by a reader with at least an 8th-grade education. Not so. It means that half the 8th graders tested for their comprehension of a standardised text can answer correctly half of the questions about it.8 So, if we test 10 adolescents and only 5 answer 5 of 10 questions correctly-the circumstance that defines reading grade level-the "comprehension success rate" is only 25%. In fact, readability formulas have long been discredited as a way to guide revision or to evaluate writing.2,9-12 Consider the fact that a text and the same text written backwards, making it meaningless, have the same reading grade level.Back to Rule #5. Shortening sentences, by itself, does not necessarily improve comprehension, 4,5 and may even reduce it.6 Complexity, not length, is what reduces comprehension, and longer sentences have more opportunities to be complex. 13 Sentence length is a factor in many readability formulas, however, which penalise longer sentences as surrogates for increased complexity. The advice to use shorter sentences is not necessarily bad, it's just simplistic; based on correlation, not causation; and not supported by the research. Why the rule harms scientific writersConventional notions of writing are founded on what we learned in college composition classes and practiced in writing term papers in college; that is, on ...

show abstract

Section: Why the Rule Won't Necessarily Improve Writingmentioning

confidence: 99%

Why scientific writers should study writing scientifically: the value of research into written communication

2017

ESE

View full text Add to dashboard Cite

show abstract

“…If some frequent proposed formulas seem intuitively relevant (such as sentence or word length), others are more questionable, particularly when they concern not only the form but also the content of the texts. The first studies on readability have been criticized for their violation of knowledge about "reading and the reading process" (Bruce et al 1981) or for their statistical bases; plenty of studies have addressed this issue (Klare 1976;Bruce et al 1981;Gilliland 1972).…”

Section: Normalisation Cnls and Readability Formulasmentioning

confidence: 99%

Towards the creation of a CNL adapted to requirements writing by combining writing recommendations and spontaneous regularities: example in a space project

Condamines

Warnier

2016

Lang Resources & Evaluation

View full text Add to dashboard Cite

Abstract:The Quality Department of the French National Space Agency (CNES, Centre National d'Études Spatiales) wishes to design a writing guide based on the real and regular writing of requirements. As a first step in this project, the present article proposes a linguistic analysis of requirements written in French by CNES engineers. One of our goals is to determine to what extent they conform to several rules laid down in two existing Controlled Natural Languages (CNLs), namely the Simplified Technical English developed by the AeroSpace and Defense Industries Association of Europe and the Guide for Writing Requirements proposed by the International Council on Systems Engineering. Indeed, although CNES engineers are not obliged to follow any controlled language in their writing of requirements, we believe that language regularities are likely to emerge from this task, mainly due to the writers' experience. We are seeking to identify these regularities in order to use them as a basis for a new CNL for the writing of requirements. The issue is approached using natural language processing tools to identify sentences that do not comply with the rules or contain specific linguistic phenomena. We further review these sentences to understand why the recommendations cannot (or should not) always be applied when specifying large-scale projects.

show abstract

“…Some popular formulas include Flesch Reading Ease Score, Flesch-Kincaid Grade Level, Dale-Chall Grade Level, Gunning Fog Index, SMOG Index, Coleman-Liau Index, and Automated Readability Index. The readability formulas have come under criticism because of their purported low validity from the perspective of psycholinguistic theories [7] and there have been efforts to develop new approaches for predicting reading difficulty, for example, by using statistical language modeling techniques and linguistic features [10,22] and by devising domain-specific readability measures [42]. While this body of readability research can be used to assess the reading complexity of a piece of text in isolation, our focus is on studying the organization and presentation of concepts in the entire textbook.…”

Section: Related Workmentioning

confidence: 99%

Empowering authors to diagnose comprehension burden in textbooks

Chakraborty

Gollapudi

Kannan

et al. 2012

Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Good textbooks are organized in a systematically progressive fashion so that students acquire new knowledge and learn new concepts based on known items of information. We provide a diagnostic tool for quantitatively assessing the comprehension burden that a textbook imposes on the reader due to non-sequential presentation of concepts. We present a formal definition of comprehension burden and propose an algorithmic approach for computing it. We apply the tool to a corpus of high school textbooks from India and empirically examine its effectiveness in helping authors identify sections of textbooks that can benefit from reorganizing the material presented.

show abstract

Why readability formulas fail

Cited by 78 publications

References 4 publications

Why scientific writers should study writing scientifically: the value of research into written communication

Why scientific writers should study writing scientifically: the value of research into written communication

Towards the creation of a CNL adapted to requirements writing by combining writing recommendations and spontaneous regularities: example in a space project

Empowering authors to diagnose comprehension burden in textbooks

Contact Info

Product

Resources

About