Paul R. Yarnold
Optimal Data Analysis, LLC
Foundational to the ODA algorithm when used with an ordered attribute is the identification of the optimal threshold—the specific cutpoint that yields the most accurate (weighted) classification solution for a sample of observations. ODA models involving a single optimal threshold will henceforth be called “fixed-threshold” models. This note proposes a new “relative-threshold” ODA model for an inter-examiner reliability study in which four examiners independently rate teeth condition for a sample of ten patients: “An important inferential question is whether the rater effects differ significantly from one another” (p. 19). In the original study, analysis of variance showed rater C assigned the greatest mean rating across patients: “The inference is therefore drawn that differential measurement bias exists (i.e., the k examiners differ systematically from one another in their mean levels of measurement)” (pp. 20-21). ODA was used to compare the entire response distribution (not only means) between raters. A fixed-threshold model identified no effects. A relative-threshold model tested the hypothesis that, for each observation in the sample considered separately, the rating by rater X will be less than (or equal to) the rating by rater Y. Analysis showed that the distribution of ratings made by rater C was nearly perfectly greater than corresponding (non-discriminable) ratings made by raters A, B, and D. This finding hints of possible development of optimal analogues of multidimensional scaling and facet theory methodologies.
odajournal.com 