Confirming my understanding of pingouin's ANOVAs for toolbox development

[mahan-hosseini]

Hey Pingouin community (thanks for your amazing work!)

I am developing an open source toolbox that uses pingouin for its ANOVAs and I would like to make sure that I am doing so correctly. In users' data there is always a repeated measurements factor, let's call it rm_fac. I want to support 1-way as well as 2-way ANOVAs, that is I want to support the following five cases. Can you please confirm that my understanding of things is correct.

Implementation & Understanding

I am denoting within- and between-subjects factors as w_fac and b_fac (_1 and _2 if needed)
The repeated measurements factor is rm_fac

Case 1

• 1-way RM ANOVA (1 within subject factor, e.g., pre / post test)
• Implemented using rm_anova with within=w_fac. rm_fac is inferred by the subjects= parameter

Case 2

• 1-way between subjects ANOVA (1 between subjects factor, e.g., experimental group)
• Implemented using mixed_anova with between=b_fac and within=rm_fac (note, not w_fac!)

Case 3

• 2-way RM ANOVA (2 within subjects factors)
• Implemented using rm_anova with within=[w_fac_1, w_fac_2] (note, not rm_fac!)
• rm_fac again inferred by subjects= parameter (unsure about this, is it true??)

Case 4

• 2-way Mixed ANOVA (1 within, 1 between subjects factor)
• Implemented using mixed_anova with between=b_fac and within=w_fac (note, not rm_fac!)
• This is equal to Case 2 but replacing rm_fac with w_fac in within=

Case 5

• 2-way between subjects ANOVA (2 between subjects factors)
• Not possible to implement

Summary
I guess in summary I would like to confirm that:

1. rm_fac is inferred automatically in Cases 1, 3, and 4
2. rm_fac is needed for Case 2
3. Case 5 is not supported at the moment

Thanks a lot for taking the time to help me with this, I really do appreciate it.
Mahan

[remrama]

Hi @mahan-hosseini,

The list of cases is kind of a lot to take in, and the pressure of confirming how things work for your package is also making me hesitatant to say much. If you are only looking for confirmation that your package is giving expected results, I would suggest simply comparing with the output of some other software you are more confident in. Of course validating the output with other software should be done regardless of the feedback in this thread, so maybe just jump ahead to that step to address this concern of yours?

I am denoting within- and between-subjects factors as w_fac and b_fac (_1 and _2 if needed) The repeated measurements factor is rm_fac

I'm confused about the difference between your w_fac and rm_fac columns. As far as I'm aware, within-subject and repeated-measures are typically referring to the same thing in ANOVA speak. So without understanding the difference between those, each case is hard for me to understand. For example, rm_fac and w_fac are the same thing in Case 1, no? The 1 within-subject factor is pre/post test, either w_fac or rm_fac could contain those values in your dataframe. Or maybe I am missing some other factor.

If you want to follow up with some feedback in this thread, could you pick just one Case example to focus on and provide a little more detail, maybe with some code? That would be more digestible and easier to provide feedback for. And again keep in mind that you should validate the Pingouin output against some other software before publishing your package.

[mahan-hosseini]

Hi @remrama

Thanks a lot for taking the time and I will certainly double-check all ANOVA results to that of another stats package. Please do not feel any pressure w.r.t. confirming that my toolbox is correct.

I'm really sorry for all the confusion. I should have kept things much simpler. Let me try again with a concrete example:
I have time series data. For example, the foot's height of mice in cm at 6 time points (0, 20, 40, 60, 80, 100). The 6 time points are my repeated measures. I called this factor rm_fac to differentiate it from factors that separate (some type of) groups. See this image as an illustration of this. We have two groups, 5mm and 25mm, and 6 time points.

So maybe just sticking to the first two cases of the 1-way ANOVAs and forgetting about my previous w_fac and b_fac. Instead I just use "group" for the df-column that differentiates groups. Would the following be a correct approach in your mind?

Case 1

• rm_anova(dv=var, within=group, subject=subject)
• here I do have to use the rm_fac column at all in the input since the repeated measures are understood via the subject column

Case 2

• mixed_anova(dv=var, between=group, within=rm_fac, subject=subject)
• here I have to use rm_fac as the within factor

I hope it's simpler now. Thanks again!

[remrama]

Okay, thanks @mahan-hosseini!

I have time series data. For example, the foot's height of mice in cm at 6 time points (0, 20, 40, 60, 80, 100). The 6 time points are my repeated measures. I called this factor rm_fac to differentiate it from factors that separate (some type of) groups. See this image as an illustration of this. We have two groups, 5mm and 25mm, and 6 time points.

In this example I only see 2 factors. From the text and the plot I think your dataframe has 4 columns:

• dv = height [I think this is y on the graph and var in your code example]
• factor1 = timepoint (0, 20, 40, 60, 80, 100) is your within-subjects factor
• factor2 = group (5mm, 20mm) is your between-subjects factor (I think...)
• subject = subject

In general I suggest standardizing your terminology. Practically speaking, within-subjects and repeated-measures are the same in this analysis, so it will be easier to track things and consider where you should be placing your arguments if you call them both within-subjects factors, because this is what Pingouin calls them in the documentation.

Case 1

rm_anova(dv=var, within=group, subject=subject)
here I do have to use the rm_fac column at all in the input since the repeated measures are understood via the subject column

I think this is wrong. Pingouin will not be understanding/inferring anything about your timepoint column if you haven't told it explicitly to do so. You do specify it correctly in Case 2 with the mixed_anova.

I think for your subsequent Cases you are asking about an additional (3rd) factor that is also within-subjects. If so, you are probably looking for a mixed-anova that can handle 2 within-subjects factors. Pingouin can handle 2 within-subjects factors in rm_anova, but not with mixed_anova (see #136 and #182). But then with rm_anova you can't account for the between-subjects factor. You might consider your options with Pingouin's ancova function and/or consider alternative packages that can handle more complex statistical models.

[raphaelvallat]

Adding to the great answer by @remrama, I would also encourage you to review this example notebook, in particular the second half on repeated measures ANOVA.

Lastly, I second @remrama's opinion of using alternative packages that can handle more complex statistical models, in particular statsmodels's Linear Mixed Effects Models module (which is often more adequate than RM ANOVA when dealing with time-series data).

[mahan-hosseini]

I appreciate this very very much @remrama @raphaelvallat
It is all clear now and I will not support Cases 3 & 4 (i.e. three factors) for now, probably using linear mixed models if users should demand it at some point.

Thanks again for your great work and quick help I hope you have lovely Christmas days.