UniODA vs. Kendall’s Coefficient of Concordance (W): Multiple Rankings of Multiple Movies

Paul R. Yarnold

Optimal Data Analysis, LLC

Kendall’s coefficient of concordance W is a non-parametric statistic used to assess agreement in rankings of multiple stimuli made by multiple raters. A normalization of the test statistic for Friedman’s non-parametric alternative to ANOVA with repeated measures, W ranges from 0 (no agreement) to 1 (complete agreement). Problems with W include complications arising from tied rankings, invalid Type I error estimates for small samples, the absence of an effect strength index normed versus chance, and the absence of an explicit model relating ranks to stimuli. The use of UniODA to assess inter-rater agreement is demonstrated as an alternative approach that overcomes these issues.

View journal article

UniODA vs. ROC Analysis: Computing the “Optimal” Cut-Point

Paul R. Yarnold

Optimal Data Analysis, LLC

Receiver operator characteristic (ROC) analysis is sometimes used to assess the classification accuracy achieved using an ordered attribute to discriminate a dichotomous class variable, and in this context to identify an “optimal” discriminant cutpoint. In ROC analysis the optimal cutpoint corresponds to the threshold value at which distance from the ROC curve to the point representing perfect classification accuracy is minimized. This note discusses the difference between an ROC-defined optimal discriminant threshold, and the optimal cutpoint identified by UniODA that maximizes ESS for the sample. ROC and UniODA methods are illustrated and compared for an application involving prediction of Cesarean delivery.

View journal article

UniODA vs. Bray-Curtis Dissimilarity Index for Count Data

Paul R. Yarnold

Optimal Data Analysis, LLC

The Bray-Curtis dissimilarity index is widely-used as an index of the magnitude of compositional dissimilarity in the count of different categories between two samples, yet it fails to address the statistical reliability of the dissimilarity, the precise nature of the dissimilarity, and the potential cross-generalizability of the findings. All of these shortcomings are remedied by the use of UniODA in this application.

View journal article