Paul R. Yarnold
Optimal Data Analysis, LLC
Random samples of 200 registered voters from each of four political wards were asked if they favored a particular candidate. Seven chi-square analyses (one omnibus comparison between all four wards, six follow-up pair-wise comparisons to specify the underlying effect) were used to compare the proportion of voters favoring the candidate between wards. Evaluating results at either the generalized or the experimentwise criterion for statistical significance, chi-square found the omnibus effect, and two pairwise comparisons: Ward 1 > Ward 2, and Ward 1 > Ward 4. In contrast, a single CTA analysis was conducted predicting voter sentiment (treated as the class variable, and coded as 1 if the voter favors the candidate, or 0 otherwise) with ward (dummy-coded as 1-4) treated as a multicategorical attribute. A single model emerged: if Ward=1, then predict the voter favors the candidate; otherwise predict the voter does not favor the candidate (p<0.036). The training (total sample) effect was relatively weak (ESS=10.2), and the predictive accuracy declined to levels worse than expected by chance (ESS= -14.8) in jackknife analysis. CTA thus revealed that the most accurate model possible for this application is weak, and that there is evidence that the model may not cross-generalize if it is used to classify independent random samples.
odajournal.com 