Detecting sequential patterns and determining their reliability with fallible observers.

Bakeman, Roger; McArthur, Duncan; Quera, Vincenç

doi:10.1037//1082-989x.2.4.357

Psychological Methods

1997

DOI: 10.1037//1082-989x.2.4.357

|View full text |Cite

Detecting sequential patterns and determining their reliability with fallible observers.

Roger Bakeman¹,

Duncan McArthur²,

Vincenç Quera³

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2000

2024

Publication Types

Select...

Article3

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Interobserver Agreement, Intraobserver Reliability, and the Rorschach Comprehensive System

Acklin¹,

McDowell²,

Verschell³

et al. 2000

Journal of Personality Assessment

View full text Add to dashboard Cite

Interrater agreement and reliability for the Rorschach have recently come under increasing scrutiny. This is the second report examining methods of Comprehensive System reliability using principles derived from observational methodology and applied behavioral analysis. This study examined a previous nonpatient sample of 20 protocols (N = 412 responses) and also examined a new clinical sample of 20 protocols (N = 374 responses) diagnosed with Research Diagnostic Criteria. Reliability was analyzed at multiple levels of Comprehensive System data, including response-level individual codes and coding decisions and ratios, percentages, and derivations from the Structural Summary. With a number of exceptions, most Comprehensive System codes, coding decisions, and summary scores yield acceptable, and in many instances excellent, levels of reliability. Limitations arising from the nature of Rorschach data and Comprehensive System coding criteria are discussed.

show abstract

Interobserver Agreement, Intraobserver Reliability, and the Rorschach Comprehensive System

Acklin¹,

McDowell²,

Verschell³

et al. 2000

Journal of Personality Assessment

View full text Add to dashboard Cite

show abstract

The Determinants and Consequences of Big 4 Lobbying Positions on Proposed Financial Accounting Standards

Monsen

2021

The Accounting Review

View full text Add to dashboard Cite

Despite the considerable participation of Big 4 accounting firms in accounting standard setting, there is no systematic evidence on what factors shape Big 4 support or opposition toward proposed accounting standards or whether their lobbying positions materially influence standards. Using textual features of Big 4 comment letters on FASB proposals, I find that Big 4 firms' lobbying positions reflect profit motives through support for standards that will generate more fees or are supported by their clients. Big 4 lobbying support is concentrated in proposals exhibiting both characteristics, with some evidence suggesting client agreement dominates fee-generating incentives. Big 4 lobbying positions are significantly associated with standard setting outcomes, both in isolation and relative to other FASB constituents, including financial statement users. Although I primarily focus on Big 4 accounting firms, results indicate the tone of comment letters submitted by users is unassociated with the standard setting outcomes measured in this study.

show abstract

Inter-rater agreement in annotating text world elements in the TextWorlds corpus

Mikhalkova

2024

Philology. Theory & Practice

View full text Add to dashboard Cite

From the perspective of Text World Theory, narratives contain elements (indications of time, place, characters, etc.) that can be automatically identified and compared to establish versions of events and similar plots based on these elements. We have annotated a corpus of fairy tales and short stories, TextWorlds, and discovered that raters do not always agree on whether a particular word refers to a character, time, or place of action. The aim of the research is to determine the degree of inter-rater agreement regarding the position of these narrative categories in the text. The practical task of the research is to assess the reliability of the annotation that will be used to train algorithms for automatically identifying text worlds. The scientific novelty lies in the fact that we are specifically studying the degree of agreement, whereas in other works, agreement is taken for granted, and if raters disagree with each other, it is perceived as an error by one of the raters or the annotation procedure. In this paper, we present the results of two expert agreement metrics: percent agreement and Krippendorff’s alpha. The obtained results for these metrics show that agreement regarding different elements varies depending on the work and sometimes reaches an average level, sufficient to speak of the reliability of the annotation.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Detecting sequential patterns and determining their reliability with fallible observers.

Cited by 3 publications

References 0 publications

Interobserver Agreement, Intraobserver Reliability, and the Rorschach Comprehensive System

Interobserver Agreement, Intraobserver Reliability, and the Rorschach Comprehensive System

The Determinants and Consequences of Big 4 Lobbying Positions on Proposed Financial Accounting Standards

Inter-rater agreement in annotating text world elements in the TextWorlds corpus

Contact Info

Product

Resources

About