Visualization and scaling of TREC topic document sets

Rorvig, Mark E.; Fitzpatrick, Steven J.

doi:10.1016/s0306-4573(97)00058-7

Cited by 12 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The cosine similarity coefficient has been consistently found to reflect interdocument relationships well since the days of the Cranfield tests and the Sparck‐Jones series of experiments over 30 years ago (Sparck‐Jones, 1971): Orthogonal documents are totally dissimilar (Sij = 0), whereas overlapping documents are totally similar (Sij = 1). Recently, Mark Rorvig (Rorvig, 2000; Rorvig & Hemmje, 1996; Rorvig, Sullivan, & Oyarce, 1998) also reached the same conclusion in a series of experiments. Rorvig and Fitzpatrick (2000) reported that under certain conditions, i.e., when interdocument similarities are treated as interval data and assumed to have a log‐normal error distribution, the statistical treatments produce more visually cohesive structures.…”

Section: Introductionmentioning

confidence: 69%

Using the shape recovery method to evaluate indexing techniques

Oyarce

2008

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

Text representation, central to information processing, must be descriptive and discriminative. Although some of the many techniques to construct document representations may outperform others for certain tasks, no one is consistently better than others. Representations are still problematic. Evaluation techniques are needed to penetrate foundational questions about term behavior in representation. A study that applies the shape recovery analysis method is reported here as an evaluative tool to compare different indexing schemes. Three weight coefficients are used to rank indexing terms and are compared to the documents' full text. Two of the weight coefficients are novel and the third relies on the chi-squared distribution. Multidimensional scaling reduces the dimensional space of the document surrogates into a two-dimensional Cartesian space. Ten concentric circles evenly separated at 10% intervals of relevant data points starting at the centroid are used to construct a precision-recall curve. ANOVA is used for a straightforward computation of the 4 ؋ 11 matrix of test data to see whether the four treatments yield the same P-R result. A post hoc HSD Tukey multiple comparisons test among pairwise treatments is also used to discover homogeneous groups. The findings show the value of the methodology to study term weighting schemes, and their descriptiveness and discriminative power, as well as the potential strength of the novel coefficients introduced.

show abstract

Section: Introductionmentioning

confidence: 69%

Using the shape recovery method to evaluate indexing techniques

Oyarce

2008

J. Am. Soc. Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…Detailed procedures for the scaling of textual document sets are available in Rorvig and Fitzpatrick (1998), Rorvig (1999). In this procedure, documents are placed in a separate directory and measured for their token correspondence.…”

Section: Methodsmentioning

confidence: 99%

“…SRA as an idea arose from the post ACM SIGIR96 Workshop on Visual Information Retrieval Interfaces (Rorvig and Hemmje, 1999) in which various members of the European and North American research community asked for development of some analytical tools to better understand document visualization interfaces. The work resulted in a series of papers (for example, see Rorvig and Fitzpatrick, 1998;Rorvig, 1999) demonstrating common recurring patterns in TREC and other test collection data. First among these discoveries was the observation that the analysis performed directly "recovered" the method by which topics and queries were constructed for the TREC data.…”

Section: Introductionmentioning

confidence: 99%

Shape recovery: A visual method for evaluation of information retrieval experiments

Rorvig

Fitzpatrick

2000

J. Am. Soc. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Detailed procedures for the scaling of textual document sets are available in Rorvig and Fitzpatrick (1998), Rorvig et al (1998), and Rorvig (1999). In this procedure, documents are placed in a separate directory and measured for their token correspondence.…”

Section: Methodsmentioning

confidence: 99%