UniODA vs. Kendall’s Coefficient of Concordance (W): Multiple Rankings of Multiple Movies

Paul R. Yarnold

Optimal Data Analysis, LLC

Kendall’s coefficient of concordance W is a non-parametric statistic used to assess agreement in rankings of multiple stimuli made by multiple raters. A normalization of the test statistic for Friedman’s non-parametric alternative to ANOVA with repeated measures, W ranges from 0 (no agreement) to 1 (complete agreement). Problems with W include complications arising from tied rankings, invalid Type I error estimates for small samples, the absence of an effect strength index normed versus chance, and the absence of an explicit model relating ranks to stimuli. The use of UniODA to assess inter-rater agreement is demonstrated as an alternative approach that overcomes these issues.

View journal article

UniODA vs. ROC Analysis: Computing the “Optimal” Cut-Point

Paul R. Yarnold

Optimal Data Analysis, LLC

Receiver operator characteristic (ROC) analysis is sometimes used to assess the classification accuracy achieved using an ordered attribute to discriminate a dichotomous class variable, and in this context to identify an “optimal” discriminant cutpoint. In ROC analysis the optimal cutpoint corresponds to the threshold value at which distance from the ROC curve to the point representing perfect classification accuracy is minimized. This note discusses the difference between an ROC-defined optimal discriminant threshold, and the optimal cutpoint identified by UniODA that maximizes ESS for the sample. ROC and UniODA methods are illustrated and compared for an application involving prediction of Cesarean delivery.

View journal article

UniODA vs. Bray-Curtis Dissimilarity Index for Count Data

Paul R. Yarnold

Optimal Data Analysis, LLC

The Bray-Curtis dissimilarity index is widely-used as an index of the magnitude of compositional dissimilarity in the count of different categories between two samples, yet it fails to address the statistical reliability of the dissimilarity, the precise nature of the dissimilarity, and the potential cross-generalizability of the findings. All of these shortcomings are remedied by the use of UniODA in this application.

View journal article

UniODA vs. Polychoric Correlation: Number of Lambs Born Over Two Years

Paul R. Yarnold

Optimal Data Analysis, LLC

This study assesses agreement between number of lambs born to 227 ewes over two consecutive years. Polychoric correlation could not be validly used to assess agreement because underlying distributional assumptions were violated. Requiring no distributional assumptions, UniODA identified moderate, statistically significant agreement.

View journal article

Triage Algorithm for Chest Radiography for Community-Acquired Pneumonia of Emergency Department Patients: Missing Data Cripples Research

Paul R. Yarnold

Optimal Data Analysis, LLC

Novometric analysis is used to develop a triage algorithm for rapid ordering of chest radiography for community-acquired pneumonia (CAP), for a retrospective Emergency Department-based matched case-control study providing data on attributes assessed for 100 radiographic confirmed cases of both CAP and influenza-like illness (ILI). Results for the least complex model revealed intake temperature exceeding 99.25°F was 73.9% sensitive [exact discrete 95% confidence interval (CI): 62.5-84.4%] and 67.9% specific (95% CI: 57.1-78.3%). Analysis was compromised by missing data so novometry was conducted omitting structural decomposition analysis in order to evaluate underlying potential of the attributes. Findings illustrate how missing data limit the fruitfulness of empirical research.

View journal article

What Most Satisfies Emergency Department Patients?

Paul R. Yarnold

Optimal Data Analysis, LLC

Novometric analysis is used to determine aspects of care which induce greatest satisfaction among Emergency Department (ED) patients. Data were obtained from a satisfaction survey with responses obtained using five-point Likert-type scales. The first analysis discriminated 1,045 strongly satisfied and 671 moderately satisfied patients, and the second analysis discriminated 1,012 patients very likely and 584 patients moderately likely to recommend the ED to others. Maximum satisfaction was associated with amount of attention nurses paid to patients, and maximum likelihood of recommending the ED was associated with physician concern for patient comfort.

View journal article

Illustrating How 95% Confidence Intervals Indicate Model Redundancy

Paul R. Yarnold

Optimal Data Analysis, LLC

In novometric analysis exact 95% confidence intervals (CIs) are computed for overall model performance and for model endpoints. When CIs overlap for two or more endpoints a model is said to be redundant, meaning that the domain of the outcome cannot be distinguished between overlapping endpoints. This research note provides an illustration of model redundancy.

View journal article

What Most Dissatisfies Emergency Department Patients?

Paul R. Yarnold

Optimal Data Analysis, LLC

Novometric analysis is used to determine aspects of care which induce greatest dissatisfaction among Emergency Department (ED) patients. Data were obtained from a satisfaction survey on which responses were obtained using five-point Likert-type scales. The first analysis discriminated 131 strongly dissatisfied and 114 moderately dissatisfied patients, and the second analysis discriminated 182 patients who were very unlikely and 92 patients who were moderately unlikely to recommend the ED to others. Maximum dissatisfaction was associated with perceived inadequacy of physician explanation of one’s illness or injury, and minimum likelihood of recommending the ED to others was associated with extreme dissatisfaction with time spent in the treatment area waiting to see the physician.

View journal article

Increasing the Likelihood of an Ambivalent Patient Recommending the Emergency Department to Others

Paul R. Yarnold

Optimal Data Analysis, LLC

Novometric analysis is used to discriminate 239 ambivalent versus 584 discharged patients who are likely to recommend the Emergency Department (ED) to others. Findings reveal the critical factors are physician explanation of tests and treatment, and attention paid to the patient by the nurse.

View journal article

What Influences Patients to Recommend an Emergency Department to Others?

Paul R. Yarnold

Optimal Data Analysis, LLC

Recent research reported that an Emergency Department (ED) patient’s ratings of how well the physician explained one’s illness or injury is the best discriminator of extreme satisfaction versus extreme dissatisfaction ratings regarding care received in the ED. The present study uses novometric analysis to discriminate 1,012 ED patients who are highly likely, versus 182 who are highly unlikely, to recommend the ED to others. Although ratings of satisfaction and likelihood to recommend are nearly perfectly related, novometry reveals that the best discriminator of ratings of extreme likelihood to recommend versus not to recommend the ED to others is waiting time in the treatment area before being seen by one’s physician.

View journal article

Globally Optimal Statistical Models, II: Unrestricted Class Variable, Two or More Attributes

Paul R. Yarnold & Robert C. Soltysik

Optimal Data Analysis, LLC

Novometrics—meaning new (Latin: novo) measurement, connotes a newly discovered theoretically-motivated algorithm that explicitly identifies the globally-optimal (GO) statistical model underlying any random statistical sample, indicated as S. Originating from the field of operations research, “optimal” denotes explicitly maximized (weighted) classification accuracy for S: that is, predicting the class category (“dependent variable”) of (weighted) observations in S as accurately as is theoretically possible for S. Novometry identifies the nature and strength of the GO relationship between a class variable and one or more attributes (“independent variables”) for S, where nature and strength are characterized by the number and homogeneity of discrete sample strata which are identified by the GO statistical model. Models maximizing ESS (a normed index of effect strength) and efficiency (ESS/number of strata, a normed index of parsimony) prevent over-fitting and promote cross-generalizability when using the model to classify an independent random S. The present article demonstrates novometry for two applications involving multiple attributes. The first study models a binary measure of patient satisfaction with care received in the Emergency Department (ED) using survey ratings of administration, nurse, physician, lab, and care of family and friends, for 2,198 ED patients. The second study ascertains the effect of atmospheric pressure on fibromyalgia symptoms recorded by a single individual over 297 sequential days.

View journal article

Globally Optimal Statistical Classification Models, I: Binary Class Variable, One Ordered Attribute

Paul R. Yarnold & Robert C. Soltysik

Optimal Data Analysis, LLC

Imagine a random sample S consisting of a class variable (“dependent measure”), one or more attrib­utes (“independent measures”), a weight (unit-weighted obser­vations are equally-valued), and a number of observations N yielding at least minimally ade­quate statistical power for testing the a priori or post hoc hypothesis that the attributes predict the class variable. The null hypothesis is the attributes can’t predict the class varia­ble. A statistical model is identified for S that optimally (most accurately) predicts the (weighted) class variable on the basis of the attributes. This is the first axiom underlying novometrics—meaning new (Latin: novo) measure­ment, and connoting a newly dis­covered theoretically-motivated algorithm that explicitly identifies the globally-optimal (GO) statistical model(s) underlying S. Originating from operations research, “optimal” as used here denotes explicitly maximized (weighted) classification accuracy for S: that is, predicting the class category of (weighted) observations in S as accurately as is theoreti­cally possible for S. Novometry identifies the nature and strength of the GO relation­ship(s) between a class variable and one or more at­trib­utes for S, where nature and strength are char­acter­ized by the number and homo­geneity of discrete sample strata identified by GO statistical model(s). Models maximizing ESS (a normed index of effect strength) and efficiency (ESS/number of strata, a normed index of parsimony) prevent over-fitting and promote cross-generalizability when using the model to classify an independent random S. This article demonstrates novometry for elemental applications involving one binary class variable and one ordered attribute. Using data from the Surveillance, Epidemiology, and End Results (SEER) Program, cancer incidence is parsed separately by sex (male, female) and by race (white, African American) to ascertain whether these class variables identify dis­crete patient strata dif­fering in cancer in­cidence.

View journal article

How to Assess the Inter-Method (Parallel-Forms) Reliability of Ratings Made on Ordinal Scales: Emergency Severity Index (Version 3) and Canadian Triage Acuity Scale

Paul R. Yarnold

Optimal Data Analysis, LLC

An exact, optimal (“maximum-accuracy”) psychometric methodology for assessing inter-method reliability for measures involving ordinal ratings is used to evaluate and compare two emergency medicine triage algorithms—both of which classify patients into one of five ordinal categories. Ten raters independently evaluated the identical set of 200 patients, five with each algorithm. UniODA revealed moderate levels of inter-method agreement, which is theoretically consistent with recent findings of moderate inter-observer reliability for triage ratings derived using both algorithms.

View journal article

How to Assess Inter-Observer Reliability of Ratings Made on Ordinal Scales: Evaluating and Comparing the Emergency Severity Index (Version 3) and Canadian Triage Acuity Scale

Paul R. Yarnold

Optimal Data Analysis, LLC

An exact, optimal (“maximum-accuracy”) psychometric methodology for assessing inter-observer reliability for measures involving ordinal ratings is used to evaluate and compare two emergency medicine triage algorithms—both of which classify patients into one of five ordinal categories. Ten raters independently evaluated the identical set of 200 patients, five with each algorithm. Analysis revealed moderate levels of inter-observer reliability, indicating that prior estimates of almost perfect inter-observer reliability obtained for the present data using suboptimal statistical methods are untenable.

View journal article

Finding Joy in the Past, Present, and Future: The Relationship Between Type A Behavior and Savoring Beliefs Among College Undergraduates

Fred B. Bryant & Paul R. Yarnold

Optimal Data Analysis, LLC

Prior research investigating savoring behaviors and Type A behavior (TAB) found that extreme Type A undergraduates are most likely to score in the highest quintile on self-congratulation, and in the lowest three quintiles on memory-building. This study used scores on past-, present-, and future-focused savoring beliefs to discriminate 117 extreme Type A versus 131 extreme Type B college undergraduates. Univariate statistical analysis conducted via UniODA revealed that compared to extreme Type Bs, extreme Type As had significantly greater reminiscence (past focus) and anticipation (future focus) scores, and also had marginally greater savor the moment (present focus) scores. Multivariate analysis via CTA identified a single-attribute model involving a three-branch parse: extreme Type Bs are substantially more likely than extreme Type As to score at lowest levels on anticipation; extreme As and Bs are comparably likely to score at moderate levels on anticipation; and extreme Type As are modestly more likely than extreme Type Bs to score at the highest levels on anticipation.

View journal article

Type A Behavior, Pessimism and Optimism Among College Undergraduates

Fred B. Bryant & Paul R. Yarnold

Optimal Data Analysis, LLC

This study used scores on measures of dispositional optimism and pessimism to discriminate 117 extreme Type A versus 131 extreme Type B college undergraduates. Consistent with a priori hypotheses the analysis revealed that Type As were significantly less pessimistic, and significantly more optimistic, than Type Bs.

View journal article

“A Statistical Guide for the Ethically Perplexed” (Chapter 4, Panter & Sterba, Handbook of Ethics in Quantitative Methodology, Routledge, 2011): Clarifying Disorientation Regarding the Etiology and Meaning of the Term Optimal as Used in the Optimal Data Analysis (ODA) Paradigm

Paul R. Yarnold

Optimal Data Analysis, LLC

The authors of Chapter 4 complain that use of the word “optimal” in the name Optimal Data Analysis (ODA) is unethical because it implies that alternative data analysis methods are less than optimal. The present Errata note addresses this misunderstanding.

View journal article