Paul R. Yarnold, Ph.D.
Optimal Data Analysis, LLC
The initial report on concurrent Phase-3 clinical trials assessing safety and efficacy of Oxford-AstraZeneca (Ox-AZ) COVID-19 vaccine is a superb example of Simpson’s Paradox—arguably the greatest threat to validity of statistical analysis in empirical science. I discuss the confounded Ox-AZ analysis in the context of the prevention of paradoxical findings.
Michael J. Maloney
Proof School
As the number of COVID-19 deaths in the US increased, various policies were enacted in an effort to slow the spread of the pandemic. As sufficient data accumulate over time, the impact of policy on public health outcomes may be statistically evaluated. The present paper uses ODA to evaluate the hypothesized ability of Minnesota’s limited mask mandate (MM) to reduce the daily number of COVID-19 deaths. Validity sensitivity analysis showed that chance-corrected reduction in the number of daily deaths began the day after the MM, and maximum reduction occurred 20 days after the MM.
To run the MegaODA software within R, please download the ODA package for R available at GitHub: https://github.com/njrhodes/ODA created by Dr. Nathaniel J. Rhodes and Dr. Paul Yarnold.
This package serves as an interface for the MegaODA software suite. To fully utilize this package, the user must obtain a licensed copy of the MegaODA software here: https://odajournal.com/purchase/
Once MegaODA is purchased and ODA for R is installed, create the following directory within the local R installation ~/ODA/win32/bin and copy/paste MegaODA.exe within this directory.
Additional resources vignettes and user guides are provided by typing “ODAmanual()” within R. For more resources and readings please visit the ODA Journal website: https://odajournal.com/
Paul R. Yarnold, Ph.D.
Optimal Data Analysis, LLC
Statistically unmotivated exploratory parametric analysis reported that the mean number of monthly consults at an academic otolaryngology service in 2014-2015 was significantly lower than in 2017-2018, suggesting a trend involving increasing numbers of consults over time. Evaluating these data, exploratory novometric temporal analysis identified a globally optimal (GO) model achieving 100% accurate discrimination in training and leave-one-out (LOO) jackknife validity analysis: all 25 months before April 2016 had <125 consults/month, and all 33 months after March 2016 had >155 consults/month (ESS=100, p<0.0001). A discontinuity thus occurred in the series between March and April in 2016.
Michael J. Maloney
Proof School
Users running MegaODA and/or CTA software using the Windows-10 operating system may execute programs more efficiently than vis-à-vis the standard procedure of using the command prompt.
Ariel Linden
Linden Consulting Group, LLC
Boosted regression (BR) has been recommended as a machine learning alternative to logistic regression for estimating the propensity score because of its greater accuracy. Commonly known as multiple additive regression trees, BR is a general, automated, data-adaptive modelling algorithm which can estimate the non-linear relationship between treatment assignment (the outcome variable) and a large number of covariates including multiple level interaction terms. However, BR is a “black-box” approach that provides scant information as to how the estimates are derived, and recent research has shown that BR can identify erroneous relationships between outcome and covariates in fabricated random data. The present paper revisits the BR approach used by Linden, et al. (2010) for estimating the propensity score and compares it to a propensity score CTA model which is generated by using the new Stata package for implementing CTA.
Paul R. Yarnold
Optimal Data Analysis LLC
Novometric classification tree analysis was used to evaluate Surveillance, Epidemiology, and End Results (SEER) Program data to discover cancer sites moderately or relatively strongly predicted by male vs. female gender. Future research using any of the 13 cancer sites which met this criterion should account for gender using matching or propensity score weighting.
Paul R. Yarnold
Optimal Data Analysis LLC
Surveillance, Epidemiology and End Results (SEER) Program data were used to find cancer sites with at least moderately different rates for African vs. Caucasian Americans. Future research in ten cancer sites which involves subjects represented by these groups should account for associated cancer-incidence disparity in matching or via propensity score weighting methods.
Paul R. Yarnold
Optimal Data Analysis LLC
This note interprets a novometric descendant family (DF) as an optimal “cluster” analysis indicating number of discriminable groups and strength of their differences at every differentiable point identified for the sample.
Paul R. Yarnold
Optimal Data Analysis LLC
This note updates current MultiODA research activity in the ODA lab.
Paul R. Yarnold
Optimal Data Analysis LLC
Novometric (Latin: new measure) statistical theory is introduced.
Nathaniel J. Rhodes
Chicago College of Pharmacy, and the Pharmacometrics Center of Excellence, Midwestern University
I study the role of the random seed number in affecting the reliability of a statistical finding, which in turn determines upper and lower bounds of statistical confidence in expected gain or loss yielded from associated decision-making. Simulation research reveals that obtaining a bootstrap solution consistent with an effect (e.g., the lower exact, discrete 2.5th percentile bound of the model bootstrap exceeds the upper exact, discrete 97.5th percentile bound of the chance bootstrap)—more than once using any two independent seeds—supports the hypothesis that the effect is not attributable to random chance because it was replicated vis-à-vis an independent seed. Based upon this finding I suggest consistent use of a primary seed and confirmation using a secondary seed when undertaking model qualification via simulation. This procedure reduces doubt about the veracity of borderline statistically significant findings, and eliminates false positive results identified by replication failure.
Ariel Linden
Linden Consulting Group, LLC
Data from the National Supported Work (NSW) randomized experiment have been used frequently over the past 30 years to demonstrate the implementation of various non-experimental methods for drawing causal inferences about treatment effects. The present paper reassesses the approach used by Dehejia and Wahba (2002) for estimating propensity scores and compares it to a propensity score CTA model which is generated by using the new Stata package for implementing CTA.
Ariel Linden & Paul R. Yarnold
Linden Consulting Group LLC & Optimal Data Analysis LLC
Data from the National Supported Work (NSW) randomized experiment have been used frequently over the past 30 years to demonstrate implementation of various non-experimental methods for drawing causal inferences about treatment effects. In this paper we reanalyze these data using the new Stata package for implementing ODA.
Nathaniel J. Rhodes and Paul R. Yarnold
Chicago College of Pharmacy, and the Pharmacometrics Center of Excellence, Midwestern University & Optimal Data Analysis LLC
We introduce a method for evaluating the upper and lower bounds of statistical confidence [e.g., an exact discrete confidence interval (CI)] in the expected effect strength for sensitivity of a decision model. The method presented herein uses bootstrap simulations to determine if a given effect is robust and not attributable to random chance (i.e., the lower bound of the model bootstrap CI is greater than the upper bound of the chance bootstrap CI). We evaluated relatively weak, moderate, and strong effects using the novometric bootstrap method. Our approach should serve to increase confidence in the veracity of decision models.
Paul R. Yarnold & Ariel Linden
Optimal Data Analysis LLC & Linden Consulting Group LLC
This paper describes how to test a directional (confirmatory) hypothesis for a design relating a three-category class (“dependent”) variable and a four-level categorical ordinal attribute (“Likert-type independent variable”) vis-à-vis the new Stata package for implementing ODA.
Paul R. Yarnold & Ariel Linden
Optimal Data Analysis LLC & Linden Consulting Group LLC
This paper describes how to assess a confirmatory (directional) hypothesis for a design involving a multicategorical class (“dependent”) variable and an ordinal attribute (“independent variable”) using the new Stata package for implementing ODA.
Paul R. Yarnold & Ariel Linden
Optimal Data Analysis LLC & Linden Consulting Group LLC
This paper demonstrates how to evaluate a confirmatory (directional) hypothesis for a design involving a multicategorical class (“dependent”) variable and a multicategorical attribute (“independent variable”) using the new Stata package for implementing ODA.
Paul R. Yarnold & Ariel Linden
Optimal Data Analysis LLC & Linden Consulting Group LLC
This paper describes how to evaluate an exploratory (nondirectional) hypothesis for a design involving a multicategorical class (“dependent”) variable and a multicategorical attribute (“independent variable”) using the new Stata package for implementing ODA.
