Standards for Reporting UniODA Findings Expanded to Include ESP and All Possible Aggregated Confusion Tables

Paul R. Yarnold, Ph.D.

Optimal Data Analysis, LLC

UniODA models maximize Effect Strength for Sensitivity (ESS), a normed measure of classification accuracy (0=chance, 100=perfect classification) that indexes the models ability to accurately identify the members of different class categories in the sample. In a study discriminating genders, for example, the percent of each gender accurately classified by the model is indexed using ESS. Unlike ESS, the Effect Strength for Predictive Value (ESP) varies across base-rate. Measured using the identical scale as ESS, ESP indexes the models ability to produce accurate classifications. In the study discriminating genders, for example, the percent of the time the model made an accurate prediction that an observation was either male or female is indexed using ESP. While ESS is important in helping to guide the development and testing of theory, ESP is important in translating theory from laboratory to real-world applications, and is thus added to the recommended minimum standards for reporting of all UniODA findings. In addition, the evaluation of all possible aggregated confusion tables aids in interpreting UniODA findings, and evaluating the potential for increasing classification accuracy by improving measurement of ordered class variables and/or attributes, and so was also added as a recommended minimum standard. Current standards are demonstrated using three examples: (1) using income to discriminate gender in a sample of 416 general internal medicine (GIM) patients, testing the a priori hypothesis that men have higher income than women; (2) using body mass index (BMI) to discriminate income in a sample of 411 GIM patients, testing the a priori hypothesis that BMI and income are positively related; and (3) discriminating mental focus using GHA (a measure of barometric pressure) in a post hoc analysis of 297 sequential daily entries of a fibromyalgia patient using an intelligent health diary, that were separated into training and hold-out validity samples.

View journal article

Statistically Significant Increases in Crude Mortality Rate of North Dakota Counties Occurring After Massive Environmental Usage of Toxic Chemicals and Biocides Began There in 1998: An Optimal Static Statistical Map

Paul R. Yarnold, Ph.D.

Optimal Data Analysis, LLC

The use of optimal data analysis (ODA) in making a map reporting the findings of confirmatory statistical analyses is demonstrated by comparing the annual crude mortality rate in counties of North Dakota, before versus after large-scale commercial usage of toxic chemicals and biocides in the environment began there in 1998.

View journal article

Ipsative Standardization is Essential in the Analysis of Serial Data

Paul R. Yarnold & Robert C. Soltysik

Optimal Data Analysis, LLC

An omnipresent experimental method in all quantitative scientific disciplines involves what is commonly called, for example, a time-series, repeated measures, clinical trial, test-retest, longitudinal, prospective, pre-post, AB, or, more generally, a serial design. In a serial design each observation is assessed on a measure on two or more test sessions spaced by a theoretically meaningful time span. This note presents a classic serial study with n=12 observations, each measured at the same four theoretically-significant times. Scatter plots illustrating test session and raw, normative, and ipsative standardized data demonstrate that ipsatively standardized data are clearly the most appropriate to statistically address fundamental questions that motivate such research.

View journal article

Minimum Standards for Reporting UniODA Findings for Class Variables with Three or More Response Categories

Paul R. Yarnold

Optimal Data Analysis, LLC

An incontrovertible advantage of establishing a minimum set of standards for reporting findings obtained using any method—is researchers from all fields will be able to easily and clearly understand fundamental statistical results of any study reporting findings using that method. This note extends minimum standards proposed for reporting of UniODA findings with binary class variables to applications involving class variables with more than two response categories, and testing confirmatory or exploratory hypotheses.

View journal article

Assessing Technician, Nurse, and Doctor Ratings as Predictors of Overall Satisfaction of Emergency Room Patients: A Maximum-Accuracy Multiple Regression Analysis

Paul R. Yarnold

Optimal Data Analysis, LLC

This study extends recent quality-of-care research undertaken to enhance understanding of ratings of overall satisfaction with care received as an Emergency Room (ER) patient. Multiple regression analysis, optimized by UniODA to maximize predictive accuracy, was used to separately evaluate the ability of ratings of technicians (n=535), nurses (n=1,800) and doctors (n=1,806) seen in the ER to predict overall satisfaction. Results showed that the classification accuracy, as assessed using ESS, which was achieved by optimized MRA models of overall satisfaction was moderate for ratings of technicians (39.6) and nurses (42.0), and relatively strong for ratings of doctors (50.9). Illustrations show how the method works.

View journal article

Maximum-Accuracy Multiple Regression Analysis: Influence of Registration on Overall Satisfaction Ratings of Emergency Room Patients

Paul R. Yarnold

Optimal Data Analysis, LLC

Quality improvement research conducted in an effort to obtain and maintain the maximum possible level of patient satisfaction is an on-going process at many hospitals. This study analyzes survey data from 773 patients receiving care at a private Midwestern hospital’s Emergency Room (ER). Multiple regression analysis, optimized by UniODA to maximize predictive accuracy, was used to assess the extent to which ratings concerning the registration process are predictive of overall satisfaction ratings. Results indicated that ratings of helpfulness of the triage nurse, ease of the process to obtain insurance/billing information, waiting time before going to the treatment area, and convenience of parking together explained 50.1% of the variance in overall satisfaction ratings, corresponding to moderate accuracy (ESS=32.8). The maximum accuracy was obtained using UniODA to adjust model decision thresholds, and improved by 20.1% but remained moderately strong (ESS=39.4).

View journal article

UniODA and Small Samples

Paul R. Yarnold

Optimal Data Analysis, LLC

Statistical hypotheses investigated by researchers representing a vast domain of empirical disciplines often involve discrimination and prediction of discrete outcomes: life versus death in medicine, innocent versus guilty in law, profit versus loss in finance, victory versus defeat in military science, etcetera. Although some studies feature large samples of a thousand or more observations, most of the literature employs much smaller samples. Preliminary research and examination of theoretical distributions indicates that UniODA is capable of identifying statistically significant effects in applications offering very small samples.

View journal article

Analysis Involving Categorical Attributes Having Many Response Categories

Paul R. Yarnold & Fred B. Bryant

Optimal Data Analysis, LLC & Loyola University Chicago

Attributes measured on a categorical response scale are common in the literature. Categorical scales for attributes such as, for example, political affiliation, ethnic origin, marital status, state of residence, or diagnosis may consist of many qualitative response categories. Such disorganized variables rarely appear in multivariable models: some effects are missed in analysis due to inadequate statistical power for the many categories, and some findings are dismissed due to inability of the investigator to recognize the dimension(s) underlying segmented categories. This research note recommends that such multicategorical attributes are replaced by a new set of attributes created via content analysis. In this approach observations are scored on new dimensions all theoretically motivated to predict the class variable. The methodology is illustrated using a hypothetical example in the field of investment realty.

View journal article

Minimum Standards for Reporting UniODA Findings

Paul R. Yarnold

Optimal Data Analysis, LLC

As the number of researchers using any statistical method and the domain of disciplines they represent increases, the opportunity for and likelihood of the development of disparate traditions for the reporting of analytic findings also increases. The great advantage of establishing a minimum set of standards for reporting findings obtained using any method—is that researchers from all fields will be able to easily and clearly understand the fundamental statistical results of any study reporting findings using that method. This note discusses a tabular presentation of the minimum information which is required in order to understand a UniODA analysis.

View journal article

How to Create an ASCII Input Data File for UniODA and CTA Software

How to Create an ASCII Input Data File for UniODA and CTA Software

Fred B. Bryant & Patrick R. Harrison

Loyola University Chicago

UniODA and CTA software require an ASCII (unformatted text) file as input data. Arguably the most difficult task an operator faces in conducting analyses is converting the original data file from (a) whatever software package was used to enter the data, into (b) an ASCII file for analysis. This article first highlights critical issues concerning missing data, variable labels, and variable types that users must address in order to convert their data into an ASCII file for analysis using ODA software. Specific steps needed to convert a data set from its original file-type into a space-delimited ASCII file are then discussed. The process of converting data into ASCII files for use as input data is illustrated for three leading statistical software packages: SPSS, SAS, and STATISTICA.

View journal article

Maximizing the Accuracy of Multiple Regression Models using UniODA: Regression Away From the Mean

Maximizing the Accuracy of Multiple Regression Models using UniODA: Regression Away From the Mean

Paul R. Yarnold, Ph.D., Fred B. Bryant, Ph.D., and Robert C. Soltysik, M.S.

Optimal Data Analysis, LLC and Loyola University Chicago

Standard regression models best predict values that lie near the mean. Three examples illustrate how optimization of the regression model using an established UniODA methodology greatly improves accurate prediction of extreme values.

View journal article

Statistical Power of Optimal Discrimination with a Normal Attribute and Two Classes: One-Tailed Hypotheses

Statistical Power of Optimal  Discrimination with a Normal Attribute and Two Classes: One-Tailed Hypotheses

Robert C. Soltysik, M.S., and Paul R. Yarnold, Ph.D.

Optimal Data Analysis, LLC

This note reports statistical power (1-β) obtained by ODA when used with a normally-distributed attribute, as a function of alpha and effect size.

View journal article

Modeling Individual Reactivity in Serial Designs: Changes in Weather and Physical Symptoms in Fibromyalgia

Modeling Individual Reactivity in Serial Designs: Changes in Weather and Physical Symptoms in Fibromyalgia

Paul R. Yarnold, Ph.D., Robert C. Soltysik, M.S., and William Collinge, Ph.D.

Optimal Data Analysis, LLC and Collinge and Associates

This note criticizes current statistical convention, and discusses and illustrates appropriate statistical methodology for investigating the relationship between weather and individual symptoms.

View journal article

Reverse CTA Versus Multiple Regression Analysis

Reverse CTA Versus Multiple Regression Analysis 

Paul R. Yarnold, Ph.D. and Robert C. Soltysik, M.S.

Optimal Data Analysis, LLC

This paper illustrates how to reverse CTA for applications having an ordered class variable and categorical attributes. Whereas a regression model is used to make point predictions for the dependent measure based on values of the independent variables, reverse CTA is used to find domains on the dependent measure which are explained by the independent variables.

View journal article

Manual vs. Automated CTA: Predicting Freshman Attrition

Manual vs. Automated CTA: Predicting Freshman Attrition

Paul R. Yarnold, Ph.D., Fred B. Bryant, Ph.D., and Jennifer Howard Smith, Ph.D.

Optimal Data Analysis, LLC, Loyola University Chicago, Applied Research Solutions, Inc.

The enumerated model was 20% more accurate, but 43% less parsimonious and 31% less efficient than the manually-derived model. Granularity afforded by the enumerated model enabled prediction of seven of eight incoming freshmen who left college. Substantive, policy, and methodological implications are considered.

View journal article

Comparing Knot Strength Using UniODA

Comparing Knot Strength Using UniODA

Paul R. Yarnold, Ph.D. and Gordon C. Brofft, BS

Optimal Data Analysis, LLC  and Marine and Water Consultant

This study assessed comparative strength of three versatile knots widely used in big-game fishing. Experiment One compared Uni and San Diego knots tied in 30-, 40- and 50-pound-test monofilament line (the modal strengths), finding no statistically significant differences in knot strength. Experiment Two attached 40-pound-test monofilament line to 50- and 65-pound-test solid spectra, and to 60-pound-test hollow spectra line using a
Double Uni knot, and found the 40-to-65 connection was strongest. High levels of variation in knot strength which were observed raises concern about the durability and consistency of monofilament line.

View journal article