Identification of effective predictive variables for document qualities

Ng, Kwong Bor; Tang, Rong; Small, Sharon; Strzalkowski, Tomek; Kantor, Paul B.; Rittman, Robert; Song, Peng; Sun, Ying; Wacholder, Nina

doi:10.1002/meet.1450400128

Cited by 5 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of the early findings of the first stage of the study were reported in Ng et al, 2003 and Tang, Ng, Strzalkowski, and Kantor, 2003. In this article, we report all our findings from both stages.…”

Section: Overviewsupporting

confidence: 57%

Automated judgment of document qualities

Kantor

Strzalkowski

et al. 2006

J. Am. Soc. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

The authors report on a series of experiments to automate the assessment of document qualities such as depth and objectivity. The primary purpose is to develop a qualitysensitive functionality, orthogonal to relevance, to select documents for an interactive question-answering system. The study consisted of two stages. In the classifier construction stage, nine document qualities deemed important by information professionals were identified and classifiers were developed to predict their values. In the confirmative evaluation stage, the performance of the developed methods was checked using a different document collection. The quality prediction methods worked well in the second stage. The results strongly suggest that the best way to predict document qualities automatically is to construct classifiers on a person-by-person basis.

show abstract

Section: Overviewsupporting

confidence: 57%

Automated judgment of document qualities

Kantor

Strzalkowski

et al. 2006

J. Am. Soc. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Early results of our analysis (Ng et al, 2003) show some success in predicting document qualities for 1000 documents using just a few grammatical categories. For instance, using stepwise discriminant analysis (Huberty 1994), we are able to correctly classify objectivity in documents in 73.3% of training cases, and 71.3% of testing cases overall (800 training cases; 200 testing cases).…”

Section: Human Judgment Of Document Qualitiesmentioning

confidence: 99%

Adjectives as indicators of subjectivity in documents

Rittman

Wacholder

Kantor

et al. 2004

Proc of Assoc for Info

Self Cite

View full text Add to dashboard Cite

The goal of this research is to automatically predict human judgments of document qualities such as subjectivity, verbosity and depth. In this paper, we explore the behavior of adjectives as indicators of subjectivity in documents. Specifically, we test whether a subset of automatically derived subjective adjectives (Wiebe, 2000b), selected a priori, behaves differently than other adjectives. 3,200 documents were ranked by 100 subjects as being high or low in nine document qualities (Tang, Ng, Strzalkowski, & Kantor, 2003). We report a statistically significant correlation between the occurrence of adjectives in documents and human judgments of subjectivity. More importantly, we find that the subset of subjective adjectives is more strongly correlated with subjectivity than adjectives in general. These results can be used to identify document qualities for use in information retrieval and question‐answering systems.

show abstract

“…In our previous studies (Ng et al 2003, Tang et al 2003), we have identified nine document qualities deemed important by information professionals. They are shown in Table 1 (see Appendix A for the definitions):…”

Section: Overviewmentioning

confidence: 99%

“…In the work described elsewhere (Ng et al 2003), we used the fact that two judges had assessed every document to produce a combined quality score for the document, and a measure of confidence in that score, which was based on the agreement between the judges. This formed the basis for a detailed analysis of the possibility of computational estimation of these quality scores, based on linguistic features of the texts.…”

Section: Introductionmentioning

confidence: 99%

The institutional dimension of document quality judgments

Bai

Sun

et al. 2004

Proc of Assoc for Info

Self Cite

View full text Add to dashboard Cite

In addition to relevance, there are other factors that contribute to the utility of a document. For examples, content properties like depth of analysis and multiplicity of viewpoints, and presentational properties like readability and verbosity, all will affect the usefulness of a document. These kinds of relevance‐independent properties are difficult to determine, as their estimations are more likely to be affected by personal aspects of one's knowledge structure. Reliability of judgments on those properties may decrease when one moves from the personal level to the global level. In this paper, we report several experiments on document qualities. In our experiments, we explore the correlation of judgments on nine different document qualities to see if we can generate fewer dimensions underlying the judgments. We also investigate the issue of reliability of the judgments and discuss its theoretical implications. We find that, between the global level of agreement of judgment and the inter‐personal level of agreement of judgment, there is an intermediate level of agreement, which can be characterized as an inter‐institutional level.

show abstract

Identification of effective predictive variables for document qualities

Cited by 5 publications

References 6 publications

Automated judgment of document qualities

Automated judgment of document qualities

Adjectives as indicators of subjectivity in documents

The institutional dimension of document quality judgments

Contact Info

Product

Resources

About