[Eeglablist] Permutation stats and number of trials per condition

[Arnaud Delorme]

Dear Jenson,

> I’m looking at ICA-decomposed ERSP differences across 7 conditions (42 participants), each of which has a different number of usable trials based on participant accuracy and noisiness of the data.  The reviewer is concerned because there is a significant difference in the number of trials across conditions, stating that comparisons between conditions are unfair since the signal to noise ratio is higher in the condition with more trials.
> 
> Based on the use of permutation statistics, I don’t think this is a valid concern, but I am having trouble articulating why.  Can anyone help with:
> 
> 1: Confirming that an unequal number of trials across conditions is not problematic?

The surrogate data will have the same statistical properties as the original data (since if you have 2 conditions say one with 20 and one with 100 trial and testing the hypothesis that they originate from the same distribution, then all the permutated data will also be comprised of two data samples of 20 and 100 trial). From my perspective, it is not a problem although your test might be less powerful than if the data is balanced. 

If I had the book with me, I would check what Rand Wilcox says about this (Introduction to Robust Estimation and Hypothesis Testing (Statistical Modeling and Decision Science)). Other comments by statisticians welcomed.

Arno