contrasts_kwdyz11.qmd

---
title: Contrast Coding of Visual Attention Effects
engine: julia
julia: 
  exeflags: ["--project", "--threads=auto"]

fig-format: png
---


```{julia}
#| code-fold: true
using AlgebraOfGraphics
using CairoMakie
using Chain
using DataFrames
using MixedModels
using SMLP2024: dataset
using StatsBase
using StatsModels

CairoMakie.activate!(; type="png")

progress = false
```

# A word of caution {#sec-caution}
Many researchers have pointed out that contrasts should be "tested instead of, rather than as a supplement to, the ordinary 'omnibus' F test" (Hays, 1973, p. 601).
 
For a (quasi-)experimental set of data, there is (or should be) a clear _a priori_ theoretical commitment to specific hypotheses about differences between factor levels and how these differences enter in interactions with other factors. This specification should be used in the first LMM and reported, irrespective of the outcome. If alternative theories lead to alternative _a priori_ contrast specifications, both analyses are justified. If the observed means render the specification completely irrelevant, the comparisons originally planned could still be reported in a Supplement).

In this script, we are working through a large number of different contrasts for the same data. The purpose is to introduce both the preprogrammed ("canned") and the general options to specify hypotheses about main effects and interactions. Obviously, we do not endorse generating a plot of the means and specifying the contrasts accordingly. This is known as the [Texas sharpshooter](https://www.bayesianspectacles.org/origin-of-the-texas-sharpshooter/) fallacy. The link leads to an illustration and brief historical account by Wagenmakers (2018).

Irrespective of how results turn out, there is nothing wrong with specifying a set of post-hoc contrasts to gain a better understanding of what the data are trying to tell us. Of course, in an article or report about the study, the _a priori_ and post-hoc nature of contrast specifications must be made clear. Some kind of alpha-level adjustment (e.g., Bonferroni) may be called for, too. And, of course, there are grey zones.

There is quite a bit of statistical literature on contrasts. Two "local" references are @Brehm2022 and @Schad2020.

For further readings see "Further Readings" in @Schad2020.

# Example data {#sec-data}

We take the `KWDYZ` dataset from @Kliegl2011.
This is an experiment looking at three effects of visual cueing under four different cue-target relations (CTRs).
Two horizontal rectangles are displayed above and below a central fixation point or they displayed in vertical orientation to the left and right of the fixation point.
Subjects react to the onset of a small visual target occurring at one of the four ends of the two rectangles.
The target is cued validly on 70% of trials by a brief flash of the corner of the rectangle at which it appears; it is cued invalidly at the three other locations 10% of the trials each.

We specify three contrasts for the four-level factor CTR that are derived from spatial, object-based, and attractor-like features of attention. They map onto sequential differences between appropriately ordered factor levels.

We also have a dataset from a replication and extension of this study @Kliegl2015
Both data sets are available in [R-package RePsychLing](https://github.com/dmbates/RePsychLing/tree/master/data/)

# Preprocessing {#sec-preprocessing}

```{julia}
dat1 = DataFrame(dataset(:kwdyz11))
cellmeans = combine(
  groupby(dat1, [:CTR]),
  :rt => mean,
  :rt => std,
  :rt => length,
  :rt => (x -> std(x) / sqrt(length(x))) => :rt_semean,
)
```

# Julia contrast options {#sec-contrasts}

We use the same formula for all analyses

```{julia}
#| output: false
form = @formula rt ~ 1 + CTR + (1 + CTR | Subj)
```

This is the default order of factor levels.

```{julia}
show(StatsModels.levels(dat1.CTR))
```

Controlling the ordering of levels for contrasts:

 1. kwarg `levels` to order the levels
 2. The first level is set as the baseline; with kwarg `base` a different level can be specified.

## SeqDiffCoding

The `SeqDiffCoding` contrast corresponds to `MASS::contr.sdif()` in R.
The assignment of random factors such as `Subj` to `Grouping()` is necessary when the sample size is very large. We recommend to include it always, but in this tutorial we do so only in the first example.

```{julia}
m1 = let levels = ["val", "sod", "dos", "dod"]
  contrasts = Dict(
    :CTR => SeqDiffCoding(; levels),
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```


What does the intercept represent?

```{julia}
mean(dat1.rt)
mean(cellmeans.rt_mean)
```

Grand Mean is mean of condition means.  

## HypothesisCoding

`HypothesisCoding` is the most general option available. We can implement all "canned" contrasts ourselves. The next example reproduces the test statistics from `SeqDiffCoding` - with a minor modification illustrating the flexibility of going beyond the default version.

```{julia}
m1b = let levels = ["val", "sod", "dos", "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
        -1  1 0  0
         0 -1 1  0
         0  0 1 -1
      ];
      levels,
      labels=["spt", "obj", "grv"],
    ),
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

The difference to the preprogrammed `SeqDiffCoding` is that for the third contrast we changed the direction of the contrast such that the sign of the effect is positive when the result is in agreement with theoretical expectation, that is we subtract the fourth level from the third, not the third level from the fourth.

## DummyCoding

This contrast corresponds to `contr.treatment()` in R

```{julia}
m2 = let
  contrasts = Dict(:CTR => DummyCoding(; base="val"))
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

The `DummyCoding` contrast has the disadvantage that the intercept returns the mean of the level specified as `base`, default is the first level, not the GM.

## YchycaeitCoding

The contrasts returned by `DummyCoding` may be exactly what we want.
Can't we have them, but also have the intercept estimate the GM, rather than the mean of the base level? Yes, we can! We call this "You can have your cake and it eat, too"-Coding (YchycaeitCoding). And we use `HypothesisCoding` to achieve this outcome.

```{julia}
m2b = let levels = ["val", "sod", "dos", "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
        -1 1 0 0
        -1 0 1 0
        -1 0 0 1
      ];
      levels,
      labels=levels[2:end],
    )
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

We can simply move the column with -1s for a different base.

```{julia}
m2c = let levels = ["val", "sod", "dos", "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
       1 -1  0  0
       0 -1  1  0
       0 -1  0  1
      ];
      levels,
      labels=["val", "dos", "dod"],
    )
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

We can simply relevel the factor with a different base.

```{julia}
m2d = let levels = ["sod", "val", "dos", "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
        -1 1 0 0
        -1 0 1 0
        -1 0 0 1
      ];
      levels,
      labels=levels[2:end],
    )
  )
  fit(MixedModel, form, dat1; contrasts)
end
```

## EffectsCoding

`EffectsCoding` estimates the difference between the Grand Mean and three of the four levels. The difference of the fourth levels can be computed from the Grand Mean and these three differences. 

```{julia}
m3 = let levels = ["val", "sod", "dos",  "dod"]
  contrasts = Dict(:CTR => EffectsCoding(; levels, base="val"))
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

This contrast corresponds almost to `contr.sum()` in R. The "almost" qualification refers to the fact that `EffectsCoding` uses the first level as default base; `contr.sum()` uses the last factor level.

```{julia}
m3b = let levels = ["val", "sod", "dos",  "dod"]
  contrasts = Dict(:CTR => EffectsCoding(; levels, base = "dod"))
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

How could we achieve the default result with HypothesisCoding?

```{julia}
m3c = let levels = ["val", "sod", "dos",  "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
         -1/4   3/4 -1/4  -1/4   # b - GM = b - (a+b+c+d)/4 =>  -1/4*a + 3/4*b - 1/4*c - 1/4*d
         -1/4  -1/4  3/4  -1/4   # c - GM = c - (a+b+c+d)/4 =>  -1/4*a - 1/4*b + 3/4*c - 1/4*d
         -1/4  -1/4 -1/4   3/4   # d - GM = d - (a+b+c+d)/4 =>  -1/4*a - 1/4*b - 1/4*c + 3/4*d
      ];
      levels,
      labels=levels[2:end],
    )
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```


## HelmertCoding

`HelmertCoding` codes each level as the difference from the average of the lower levels. With the default order of `CTR` levels we get the following test statistics. These contrasts are orthogonal.

```{julia}
m4 = let levels = ["val", "sod", "dos",  "dod"]
  contrasts = Dict(:CTR => HelmertCoding(; levels))
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

```sh
+ HeC1: (b - a)/2           # (391 - 358)/2 = 16.5
+ HeC2: (c - (b+a)/2)/3     # (405 - (391 + 358)/2)/3 = 10.17 
+ HeC3: (d - (c+b+a)/3)/4   # (402 - (405 + 391 + 358)/3)/4 = 4.33
```

We can reconstruct the estimates, but they are scaled by the number of levels involved. With `HypothesisCoding` we can estimate the "unscaled" differences. Also the labeling of the contrasts is not as informative as they could be.

```{julia}
m4b = let levels = ["val", "sod", "dos",  "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
          -1    1    0   0 
         -1/2 -1/2   1   0
         -1/3 -1/3 -1/3  1
        
      ];
      levels,
      labels= ["2-1", "3-21", "4-321"]
    )
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

## Reverse HelmertCoding

`Reverse HelmertCoding` codes each level as the difference from the average of the higher levels. To estimate these effects we simply reverse the order of factor levels. Of course, the contrasts are also orthogonal.

```{julia}
m4c = let levels = reverse(StatsModels.levels(dat1.CTR))
  contrasts = Dict(:CTR => HelmertCoding(; levels))
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

```sh
+ HeC1:(c - d)/2            # (405 - 402)/2 = 1.5
+ HeC2:(b - (c+d)/2)/3      # (391 - (405 + 402)/2)/3 = -4.17
+ HeC3:(a - (b+c+d)/3/4     # (356  -(391 + 405 + 402)/3)/4 = -10.83
```

... and the unscaled-by-number-of-levels estimates.

```{julia}
m4d = let levels = ["val", "sod", "dos",  "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
        0    0     1   -1 
        0    1   -1/2 -1/2
        1  -1/3  -1/3 -1/3
      ];
      levels,
      labels= ["3-4", "2-34", "1-234"]
    )
  )
  fit(MixedModel, form, dat1; contrasts)
end
```

# Other orthogonal contrasts

For factors with more than four levels there are many options for specifying orthogonal contrasts as long as one proceeds in a top-down strictly hierarchical fashion.

Suppose you have a factor with seven levels and let's ignore shifting columns.
In this case, you have six options for the first contrast, that is 6 vs. 1, 5 vs.2 , 4 vs. 3, 3 vs. 4, 2 vs. 5, and 1 vs. 6 levels.
Then, you specify orthogonal contrasts for partitions with more than 2 elements and so on.
That is, you don't specify a contrast that crosses an earlier partition line.

In the following example, after an initial 4 vs 3 partitioning of levels, we specify `AnovaCoding` for the left and `HelmertCoding` for the right partition.

```{julia}
contrasts = Dict(
  :CTR => HypothesisCoding(
    [
      -1/4 -1/4 -1/4 -1/4 +1/3 +1/3 +1/3
      -1/2 -1/2 +1/2 +1/2    0    0    0
      -1/2 +1/2 -1/2 +1/2    0    0    0
      +1/2 -1/2 -1/2 +1/2    0    0    0
         0    0    0    0   -1   +1    0
         0    0    0    0 -1/2 -1/2    1
    ];
    levels=["A1", "A2", "A3", "A4", "A5", "A6", "A7"],
    labels=["c567.1234", "B", "C", "BxC", "c6.5", "c6.56"],
  ),
);
```

There are two rules that hold for all orthogonal contrasts:

 1. The weights within rows sum to zero.
 2. For all pairs of rows, the sum of the products of weights in the same columns sums to zero.

## Anova Coding

Factorial designs (i.e., lab experiments) are traditionally analyzed with analysis of variance. The test statistics of main effects and interactions are based on an orthogonal set of contrasts.
We specify them with `HypothesisCoding`.

### A(2) x B(2)

An A(2) x B(2) design can be recast as an F(4) design with the levels (A1-B1, A1-B2, A2-B1, A2-B2).
The following contrast specification returns estimates for the main effect of A, the main effect of B, and the interaction of A and B.
In a figure With A on the x-axis and the levels of B shown as two lines, the interaction tests the null hypothesis that the two lines are parallel.
A positive coefficient implies overadditivity (diverging lines toward the right) and a negative coefficient underadditivity (converging lines).

```{julia}
m5 = let levels = ["val", "sod", "dos", "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
        -1 -1 +1 +1          # A
        -1 +1 -1 +1          # B
        +1 -1 -1 +1          # A x B
      ];
      levels,
      labels=["A", "B", "AxB"],
    ),
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

It is also helpful to see the corresponding layout of the four means for the interaction of A and B (i.e., the third contrast)

```
        B1     B2
   A1   +1     -1
   A2   -1     +1
```

Thus, interaction tests whether the difference between main diagonal and minor diagonal is different from zero.

### A(2) x B(2) x C(2)

Going beyond the four level factor; it is also helpful to see the corresponding layout of the eight means for the interaction of A and B and C.

```
          C1              C2
      B1     B2        B1     B2
 A1   +1     -1   A1   -1     +1
 A2   -1     +1   A2   +1     -1
```

## Nested coding

Nested contrasts are often specified as follow up as post-hoc tests for ANOVA interactions. They are orthogonal. We specify them with `HypothesisCoding`.

An A(2) x B(2) design can be recast as an F(4) design with the levels (A1-B1, A1-B2, A2-B1, A2-B2).
The following contrast specification returns an estimate for the main effect of A and the effects of B nested in the two levels of A.
In a figure With A on the x-axis and the levels of B shown as two lines, the second contrast tests whether A1-B1 is different from A1-B2 and the third contrast tests whether A2-B1 is different from A2-B2.

```{julia}

m6 = let levels = ["val", "sod", "dos", "dod"]
  contrasts = Dict(
    :CTR => HypothesisCoding(
      [
        -1 -1 +1 +1
        -1 +1  0  0
         0  0 +1 -1
      ];
      levels,
      labels=["do_so", "spt", "grv"],
    ),
  )
  fit(MixedModel, form, dat1; contrasts, progress)
end
```

The three contrasts for one main effect and two nested contrasts are orthogonal. There is no test of the interaction (parallelism).

# An Example for a complex example: Group(2) x A(2) x B(2)

Three factors:

+ G(roup): G1, G2 - between subjects
+ A: A1, A2, - within subjects
+ B: B1, B2, B3 - within subjects

2 x 3 = 6 measures / subject

```{julia}
dat2 = dataset(:exp_2x2x3)
```

We select an LMM supported by the data. 

```{julia}
cntrst2 = Dict(
    :Group => SeqDiffCoding(; levels=["G1", "G2"]),
    :A => SeqDiffCoding(; levels=["A1", "A2"]),
    :B => SeqDiffCoding(; levels=["B1", "B2", "B3"]),
  )

f6_cpx = @formula dv ~ 1 + Group*A*B   + (1 + A+B | Subj);
m6_cpx = fit(MixedModel, f6_cpx, dat2; contrasts=cntrst2)
issingular(m6_cpx)

f6_zcp = @formula dv ~ 1 + Group*A*B   + zerocorr(1 + A+B | Subj);
m6_zcp = fit(MixedModel, f6_zcp, dat2; contrasts=cntrst2)
issingular(m6_zcp)

f6_ovi = @formula dv ~ 1 + Group*A*B   +  (1 | Subj);
m6_ovi = fit(MixedModel, f6_ovi, dat2; contrasts=cntrst2)
```

```{julia}
lrtest(m6_ovi, m6_zcp, m6_cpx)
```


There is a significant interaction between A and the first contrast of B (i.e., B2 - B1). The interaction is not significant for A and the second contrast of B (i.e., B3 - B2).  This implies that the left pair of lines in the following figure is statistically not parallel and that we do not have sufficient evidence that the right pair of lines is not parallel. 

```
───────────────────────────────────────────────────────────────────
                                 Coef.  Std. Error      z  Pr(>|z|)
───────────────────────────────────────────────────────────────────
A: A2 & B: B2               3.53363       0.924789   3.82    0.0001
A: A2 & B: B3               0.523243      0.950202   0.55    0.5819
───────────────────────────────────────────────────────────────────
```

```{julia}
using Chain
tbl1 = @chain DataFrame(dat2) begin
  groupby(_, [:Subj, :A, :B])
  combine(_, nrow => :n, :dv => mean => :dv)
  groupby(_, [:A, :B])
  combine(_, 
          :dv => mean => :dv_M,
          :dv => std => :dv_SD,
          :dv => sem => :dv_SE)
end 

fig1 = data(tbl1) * mapping(:B, :dv_M; color=:A) * (visual(Lines) + visual(Scatter))
draw(fig1)
```


# Appendix: Summary (Dave Kleinschmidt)

[StatsModels](https://juliastats.org/StatsModels.jl/latest/contrasts/)

StatsModels.jl provides a few commonly used contrast coding schemes, some less-commonly used schemes, and structs that allow you to manually specify your own, custom schemes.

## Standard contrasts

The most commonly used contrasts are `DummyCoding` and `EffectsCoding` (which are similar to `contr.treatment()` and `contr.sum()` in R, respectively).

## "Exotic" contrasts (rk: well ...)

We also provide `HelmertCoding` and `SeqDiffCoding` (corresponding to base R's `contr.helmert()` and `MASS::contr.sdif()`).

## Manual contrasts

**ContrastsCoding()**

There are two ways to manually specify contrasts.
First, you can specify them **directly** via `ContrastsCoding`.
If you do, it's good practice to specify the levels corresponding to the rows of the matrix, although they can be omitted in which case they'll be inferred from the data.

**HypothesisCoding()**

A better way to specify manual contrasts is via `HypothesisCoding`, where each row of the matrix corresponds to the weights given to the cell means of the levels corresponding to each column (see @Schad2020 for more information).