---
title: "Back to the Future: Human-friendly Scheffé Contrasts, or, the Art of Complex Multiple Comparisons... or... Everything Old Is New Again"
subtitle: "Gordon P Brooks & Nina Adjanin"
author: "brooksg@ohio.edu & adanin@nwmissouri.edu"
date: "Presented at AERA 2023 Chicago, 14-April-2023"
output:
html_document:
toc: true
toc_level: 1
number_sections: false
pdf_document:
toc: yes
word_document:
toc: no
classoption: portrait
editor_options:
chunk_output_type: console
---
```{r KNIT_setup, include=FALSE, echo=FALSE, comment=NA}
knitr::opts_chunk$set(echo = TRUE)
```
# Program for "Barcikowski Human-Friendly" Contrasts```{r SET_UP_OPTIONS, include=FALSE, echo=FALSE, comment=NA}
myAlpha <- 0.15
onlyMax <- TRUE
printCons <- FALSE
myExample <- "Stevens" #"Stevens" "Snedecor" "effective" "TAM" "Snedecor9" "Maxwell" "MC4G" "ThreeGrps"
```
## Citation
Brooks, G. P., & Adjanin, N. (2023, April). _Back to the Future: Human-friendly Scheffé Contrasts, or, the Art of Multiple Comparisons_. Paper presented at the 2023 annual meeting of the American Educational Research Association, Chicago, IL.
## Background
* The idea for "Human-Friendly Contrasts" in One-way ANOVA was originally conceived by Robert S. Barcikowski (2000, unpublished) based on Scheffe (1956) and Hollingsworth (1978). Barcikowski's original FORTRAN code was amended and adapted by Brooks & Adjanin.
* "Human-Friendly Contrasts" are based primarily on Helmert-type contrasts, providing "reasonable" comparisons among multiple groups
* For example, in an experiment with one control group and two treatment groups, a "reasonable" comparison might be "SOMETHING" versus "NOTHING". Such a comparison would require contrasts of the form:
* Contrast Coefficients: Coef1 = 1.0, Coef2 = -0.50, Coef3 = -0.50
* Or maybe two trait/demographic groups (combined in a reasonable way, perhaps Associate and Full Tenured Professors) versus two other groups (combined reasonably, such as Assistant Professors and Instructors), a comparison might be "TENURED" versus "NOT TENURED". Such a comparison would require contrasts of the form:
* Contrast Coefficients: Coef1 = 0.5, Coef2 = 0.50, Coef3 = -0.50, Coef4 = -0.50
* Maximum number of groups allowed by the program = 8
## qqBarth R package created by the authors
* https://people.ohio.edu/brooksg/qqBarth.tar.gz
* Maximum Scheffe Contrasts and Human-Friendly Contrasts
* Version: 1.4
* Description: Functions Identify the maximum Scheffe contrast and analyzes the contrasts for an applied researcher dataset. The code also identifies the slightly different normalized maximum posttest contrast described by Hollingsworth (1978). Critically, however, the coefficient weights from complex contrasts such as these are likely to be uninterpretable. Therefore, we have also implemented a method to identify maximum human-friendly contrasts that may be meaningful to and interpretable by a human researcher. We have named these human-friendly contrasts _Barcikowski contrasts_ in memory of Robert Barcikowski, the scholar who introduced the idea to the authors.```{r LOAD_qqBarth_PACKAGE, echo=FALSE, include=FALSE}
pkg1 <- c("installr")
ins1 <- pkg1[!(pkg1 %in% installed.packages()[,"Package"])]
if(length(ins1)) install.packages(ins1, dependencies = TRUE)
invisible(lapply(pkg1, library, character.only = TRUE))
#--
pkg2 <- c("qqBarth")
ins2 <- pkg2[!(pkg2 %in% installed.packages()[,"Package"])]
installr::install.packages.zip ("https://people.ohio.edu/brooksg/qqBarth.tar.gz")
invisible(lapply(pkg2, library, character.only = TRUE))
#--
examples <- qqBarth::exampleDataFun()
```
### Data (import or create a data frame called *myData*)
* Put your data into a data frame called myData before running any more code
* Your grouping variable must be column 1 and your dependent variable must be column 2
* Replace myAlpha to some alpha value to set your desired alpha level to use for output of Human-Friendly Contrasts (you will always get at least 4 printed out)
* This code will show your first 6 cases, just to check
```{r SETUP_myDF, echo=FALSE}
XCOL <- 1
YCOL <- 2
alpha2use <- myAlpha
if (!exists("myData")) myData <- examples[[myExample]]
myDF <- as.data.frame(myData)
myDF <- data.frame(x=as.factor(myDF[,XCOL]), y=as.numeric(myDF[,YCOL]))
myDF$x <- factor(as.numeric(myDF$x))
head(myDF)
```
```{r RUN_FUNCTIONS_CREATE_OUTPUT, include=FALSE}
OUT <- qqBarth::humanContrastsFun(myDF = myDF, XCOL = 1, YCOL = 2, ALPHA = c(.10, .05, .01), alpha2use = alpha2use)
ifelse(OUT$gnum < 9, OK <- TRUE, OK <- FALSE)
```
# OUTPUT: Barcikowski's "Human-Friendly" Contrasts
```{r DATA_ERROR, include=TRUE, echo=FALSE, eval=!OK, include=!OK, comment=NA}
knitr::asis_output("# DATA ERROR\\n")
print("There is no OUTPUT because the data set contains more than 8 levels/groups")
print(paste("There were ", OUT$gnum, " levels/groups identified for the x variable in these data"))
```
```{r TOTAL_SAMPLE_DESCRIPTIVES, include=TRUE, echo=FALSE, eval=OK, include=OK, comment=NA}
knitr::asis_output("## Total Sample Descriptive Statistics\\n")
qqBarth::totalSampleDescriptivesFun(OUT = OUT)
```
```{r BY_GROUP_DESCRIPTIVES, include=TRUE, echo=FALSE, include=OK, comment=NA}
knitr::asis_output("## By-Group Descriptive Statistics\\n")
qqBarth::byGroupDescriptivesFun(OUT = OUT)
```
```{r NORMALITY_OF_RESIDUALS, include=TRUE, echo=FALSE, include=OK, comment=NA}
knitr::asis_output("## Test of Normality (of residuals)\\n")
print(OUT$nort$ShapiroResid)
```
`r if(OK) {"## Tests of Homoscedasticity \n Levene (mean)"}`
```{r LEVENE_MEAN_Homoscedasticity, include=TRUE, echo=FALSE, include=OK, comment=NA}
print(OUT$sced$leveneMean1way)
```
`r if(OK) {" Levene (median)"}`
```{r LEVENE_MEDIAN_Homoscedasticity, include=TRUE, echo=FALSE, include=OK, comment=NA}
print(OUT$sced$leveneMedian1way)
```
`r if(OK) {" Breusch-Pagan"}`
```{r BREUSCH_PAGAN_Homoscedasticity, include=TRUE, echo=FALSE, include=OK, comment=NA}
print(OUT$sced$bptest1way)
```
`r if(OK) {"## Omnibus (Overall) ANOVA \n Fisher F"}`
```{r FISHER_ANOVA_F, include=TRUE, echo=FALSE, include=OK, comment=NA}
print(summary(OUT$omni$aov1way))
```
`r if(OK) {" Robust Welch F"}`
```{r ROBUST_WELCH_F, include=TRUE, echo=FALSE, include=OK, comment=NA}
print(OUT$omni$welch1way)
```
`r if(OK) {" Robust Brown-Forsythe F"}`
```{r ROBUST_BROWN_FORSYTHE_F, include=TRUE, echo=FALSE, include=OK, comment=NA}
cat("BROWN-FORSYTHE F", "\n\n", "F =", OUT$omni$bf1way[['statistic']], ", num df =", OUT$omni$bf1way[['parameter']][1], ", den df =", OUT$omni$bf1way[['parameter']][2], ", p-value = ", OUT$omni$bf1way[['p.value']], '\n\n' )
````r if(OK) {"# Barcikowski's Most Explanatory Human-Friendly Contrasts \n * The most explanatory contrasts significant at chosen alpha (at least 4 are shown with the MAXIMUM Human Contrast listed first) \n\n * The p values reported in the section below have already been adjusted using Scheffe adjustments, so compare the p value to your desired alpha level without further adjustment \n\n * A later section provides the Brown-Forsythe adjustment for unequal variances (for the Maximum contrast only by default, but can be changed to all those significant at chosen alpha)"}`
```{r SIG_OR_TOP4_HUMAN_FRIENDLY_CONTRASTS, include=TRUE, echo=FALSE, include=OK, comment=NA}
cat(OUT$human$CONTOT, "total Human-Friendly Contrasts were tested using alpha = ", alpha2use, "\n\n")
if (printCons) print(OUT$human$ALLCON, digits=2)
qqBarth:: humanMaxFun(OUT = OUT, OK = OK)
```
`r if(OK) {"# Maximum Contrasts (Barcikowski, Scheffe, Hollingsworth) \n * These are the Maximum contrasts for all methods for comparison purposes"}`
```{r MAXIMUM_CONTRASTS_ALL_METHODS, include=TRUE, echo=FALSE, include=OK, comment=NA}
qqBarth::maximumContrastsFun(OUT = OUT, OK = OK)
```
`r if(OK) {"## Tests of Scheffe-only contrasts without Brown-Forsythe adjustment \n * These analyses performed with DescTools ScheffeTest function with the maximum contrasts as input \n\n * The p values reported in sections below have already been adjusted using Scheffe adjustments, so compare the p value to your desired alpha level without further adjustment \n\n * The sections following this one provide the Brown-Forsythe adjustments for unequal variances for these Maximum Contrasts"}`
```{r SIG_MAXIMUM_CONTRASTS_ALL_METHODS, include=TRUE, echo=FALSE, include=OK, comment=NA}
qqBarth::descToolsMaximumsFun(OUT = OUT, OK = OK)
````r if(OK) {"## Tests of Scheffe contrasts with Brown-Forsythe adjustment \n"}`
`r if(OK) {"### BARCIKOWSKI HUMAN-FRIENDLY Maximum Contrast(s) \n * The sections below use calculations of statistics and critical values for statistical significance as opposed to the p values provided by the DescTools output above \n\n * One critically important part of the results sections below is the robust Brown-Forsythe Adjustment applied to Scheffe for unequal variances, something not available many places"}`
```{r HUMAN_FRIENDLY_MAXIMUM, include=TRUE, echo=FALSE, include=OK, comment=NA}
qqBarth::top4Fun(OUT = OUT, onlyMax = onlyMax, OK = OK)
```
`r if(OK) {"### HOLLINGSWORTH Maximum Contrast \n * Hollingsworth, H. (1978). The coefficients of the normalized maximum contrast as statistics for posttest ANOVA data interpretations. Journal of Experimental Education, 46(4), 4-6.\n\n * Hollingsworth, H. (1980/1981). Maximized posttest contrasts: A clarification. Journal of Experimental Education, 49(2), 92-93."}`
```{r HOLLINGSWORTH_MAXIMUM, include=TRUE, echo=FALSE, include=OK, comment=NA}
qqBarth::hollingsworthMaxFun(OUT = OUT, OK = OK)
```
`r if(OK) {"### SCHEFFE (SCALED) Maximum Contrast \n SCHEFFE Maximum Contrast & Maximum Contrast SCALED so positive & negative coefficient sets sum to 0"}`
```{r SCHEFFE_MAXIMUM, include=TRUE, echo=FALSE, include=OK, comment=NA}
qqBarth::scheffeMaxFun(OUT = OUT, OK = OK)
```
# END