Novometric Statistical Analysis and the Pearson-Yule Debate

Paul R. Yarnold

Optimal Data Analysis, LLC

Karl Pearson and George Yule debated the validity of the assumptions made when estimating the association between cross-classified vaccine (or antitoxin) administration and mortality, if both variables are assessed on binary measurement scales. Pearson assumed these measures reflect an inherently continuous distribution and used tetrachoric correlation to estimate their association, and Yule assumed these measures reflect an inherently discrete distribution and used the phi coefficient to estimate their association. Novometric statistical analysis (NSA) may be used in either circumstance: measurement scales may be treated as categorical or ordered, motivated by a priori substantive theoretical or empirical consideration; exploratory or confirmatory hypotheses may be tested; and observations may be analytically weighted.

View journal article

ODA vs. π and κ: Paradoxes of Kappa

Paul R. Yarnold

Optimal Data Analysis, LLC

Widely-used indexes of inter-rater or inter-method agreement, π and κ sometimes produce unexpected results called the paradoxes of kappa. For example, prior research obtained four legacy agreement statistics (κ, Scott’s π, G-index, Fleiss’s generalized π) for a 2×2 table in which two independent raters failed to jointly classify any observations into the “negative” rating-class category: two indexes reported > 88.8% overall agreement and the other two reported < -2.3% overall agreement. ODA sheds new light on this paradox by testing confirmatory and exploratory hypotheses for these data, separately modeling the ratings made by each rater, and separately maximizing model predictive accuracy normed for chance (ESS; 0=inter-rater agreement expected by chance, 100=perfect agreement) as well as model overall accuracy that is not normed for chance (PAC; 0=no inter-rater agreement, 100=perfect agreement).

View journal article

Novometric Analysis vs. EO-CTA: Disentangling Sets of Sign-Test-Based Multiple-Comparison Findings

Paul R. Yarnold

Optimal Data Analysis, LLC

Prior empirical comparison of the timeline follow-back (TLFB, dummy-coded as 1) vs. Drinker Profile (DP, coded as 2) methods of quantifying alcohol consumption in treatment research reported pairwise sign tests comparing these methods separately on four categorical ordinal outcomes: abstinent=1; light=2; moderate=3; heavy=4. It was concluded: “The direction of differences for the abstinent and medium categories approached significance (with unprotected alpha criterion at .05) with the DP more often yielding higher estimates of abstinent days and lower estimates of medium days. The DP significantly more often yielded lower estimates of light days”. This example is used to illustrate and compare the use of CTA vs. novometric analysis to disentangle sets of pairwise comparison outcomes.

View journal article

CTA vs. Non-Disentangled Omnibus Chi-Square: Comparing Samples (Not) Selected for Study Participation

Paul R. Yarnold

Optimal Data Analysis, LLC

An empirical comparison of timeline follow-back vs. averaging methods for quantifying alcohol consumption in treatment research reported non-disentangled, non-interpreted chi-square-based comparisons of factors differentiating subjects selected vs. not selected for participation in the study. CTA easily identifies underlying inter-study differences.

View journal article