Paul R. Yarnold
Optimal Data Analysis, LLC
Immediately after graduating from one of four alternative sales training programs, graduates were randomly assigned to sales areas putatively having comparable sales opportunities: number of sales made by each of N=23 graduates at the end of their first week was recorded. Analysis by one-way ANOVA yielded F(3,19)=3.13, p<0.0281. It was concluded: “…evidence is sufficient to indicate a difference in mean achievement for the four training programs” (p. 383). If the omnibus effect (comparing all of the groups simultaneously) effect has p<0.05, then all-possible pairwise comparisons (or a more efficient range test procedure) are used to disentangle the omnibus effect and identify the statistically significant inter-group differences. This was not reported, but the combination of a test of a non-directional hypothesis (the anticipated relative ordering of mean sales by group was not specified a priori), in conjunction with the small sample and associated weak statistical power, limit the detectable effects to those reflecting extremely strong inter-group differences. Non-directional ODA treating group as the class variable and sales as the ordered attribute was unable to identify a statistically reliable model for discriminating all four sales groups (ESS=42.46, D=5.42, p<0.32). A single novometric model emerged: if sales<87.5 then predict group<4; otherwise predict group=4. Model performance in total sample analysis was relatively strong and statistically reliable: ESS=69.74, D=0.87, p<0.042 (sensitivity for group 4=75.00%, for groups 1-3=94.74%). Jackknife analysis suggested the effect may not cross-generalize if the model is used to classify different samples of graduates: ESS=43.42, D=2.61, p<0.015 (sensitivity for group 4=75.00%, for groups 1-3=68.42%).
odajournal.com 