[Eeglablist] Stat test for eeg data

[Guillaume Rousselet]

Hi Haneen,

As usual in statistics, the answer to your questions is “it depends”.

There is nothing wrong with averaging signals in regions of interest, if you are ok with loosing topographic / frequency / timing information. It could improve power in some situations. Keep in mind that the normality assumptions apply to the residuals of your model, not the marginal distributions. But in general, there is no reason to assume normality with anything we measure in psychology or neuroscience. Whether normality is an ok approximation is an empirical question. Which test to apply depends on your question. Parametric and rank-based statistics make different types of interferences – they are not interchangeable. Bootstrap tests come in many flavours, the most common being percentile bootstrap and bootstrap-t<https://urldefense.com/v3/__https://open.lnu.se/index.php/metapsychology/article/view/2058__;!!Mih3wA!BCK2WQoiHbDuDgBHPJ7TpKFxs97F1_kVNf_QRJVrj48KkDDixpyTHqk0J5_7i17Wei7MWIIUZf5RPBjPOblWDgrV5-8wTv0WJoFGHHA$ >. These methods can be used to make inferences about many estimators, and thus answer different questions – for instance to compare trimmed means, medians, variances… If instead you use a permutation test, you make an inference about the shape of the distributions, because for instance a permutation test on means will also be sensitive to differences in spread. To correct for multiple comparisons, FDR is not recommended if you want to localise effects, because of issues described here for instance:

Winkler, A. M., Taylor, P. A., Nichols, T. E., & Rorden, C. (2024). False Discovery Rate and Localizing Power (No. arXiv:2401.03554). arXiv. https://urldefense.com/v3/__https://doi.org/10.48550/arXiv.2401.03554__;!!Mih3wA!BCK2WQoiHbDuDgBHPJ7TpKFxs97F1_kVNf_QRJVrj48KkDDixpyTHqk0J5_7i17Wei7MWIIUZf5RPBjPOblWDgrV5-8wTv0WPLUMJYI$ 

Rousselet, G. A. (2025). Using cluster-based permutation tests to estimate MEG/EEG onsets: How bad is it? European Journal of Neuroscience, 61(1), e16618. https://urldefense.com/v3/__https://doi.org/10.1111/ejn.16618__;!!Mih3wA!BCK2WQoiHbDuDgBHPJ7TpKFxs97F1_kVNf_QRJVrj48KkDDixpyTHqk0J5_7i17Wei7MWIIUZf5RPBjPOblWDgrV5-8wTv0WmI3lWt0$ 

If your work is exploratory and localisation of effects is not crucial, then FDR will have more power than the MAX correction for instance.

Ultimately, the best way to answer your question would be to perform simulations using synthetic data or a large dataset matching your data, to estimate true and false positives, and make an informed decision about a good way forward in the long run. Short of that, the most important part is to justify explicitly all your choices.

Hope this helps,

Guillaume

—
Dr Guillaume A. Rousselet
School of Psychology and Neuroscience
University of Glasgow
[How to pronounce my name<https://urldefense.com/v3/__https://namedrop.io/guillaumerousselet__;!!Mih3wA!BCK2WQoiHbDuDgBHPJ7TpKFxs97F1_kVNf_QRJVrj48KkDDixpyTHqk0J5_7i17Wei7MWIIUZf5RPBjPOblWDgrV5-8wTv0W6ADT0So$ >]
—

https://urldefense.com/v3/__https://garstats.wordpress.com__;!!Mih3wA!BCK2WQoiHbDuDgBHPJ7TpKFxs97F1_kVNf_QRJVrj48KkDDixpyTHqk0J5_7i17Wei7MWIIUZf5RPBjPOblWDgrV5-8wTv0WqE6FLHs$ <https://urldefense.com/v3/__https://garstats.wordpress.com/__;!!Mih3wA!BCK2WQoiHbDuDgBHPJ7TpKFxs97F1_kVNf_QRJVrj48KkDDixpyTHqk0J5_7i17Wei7MWIIUZf5RPBjPOblWDgrV5-8wTv0WteQZdRA$ >

“Forget about getting definitive results from a single experiment; instead embrace variation, accept uncertainty, and learn what you can.” Gelman 2018

The University of Glasgow, charity number SC004401

From: eeglablist <eeglablist-bounces at sccn.ucsd.edu> on behalf of Haneen Alsuradi via eeglablist <eeglablist at sccn.ucsd.edu>
Date: Friday, 6 June 2025 at 20:03
To: EEGLAB List <eeglablist at sccn.ucsd.edu>
Subject: [Eeglablist] Stat test for eeg data
Dear All

I have a question regarding a statistical testing methodology for eeg data.

I conducted an experiment where i have recruited people to go through three
conditions.

I know the frequency bands of interest based on literature.

I found the region of interest (spatial) using a data driven approach.

Given this region of interest, I averaged over channels within the region
(power values in dB). I conducted pair wise statistical comparisons between
the conditions (three in total) using the bootstrapping test. I corrected
for the three pvalues using fdr. Please note that my data violates
normality.


I would like to seek advice if this is a valid approach, or if this is too
lenient of a method? Would using friedman test followed by bonferroni is a
more appropriate method? Or is bootstrapping test is equivalently suitable?
I read on eeglab documentation that bootstrapping test is ideal for eeg
data. Is the way I described for using it sensible?

Please advise and share your thoughts.


Best
Haneen
_______________________________________________
To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu or visit https://urldefense.com/v3/__https://eur03.safelinks.protection.outlook.com/?url=https*3A*2F*2Fsccn.ucsd.edu*2Fmailman*2Flistinfo*2Feeglablist&data=05*7C02*7CGuillaume.Rousselet*40glasgow.ac.uk*7Cde7c8630a70e442353ba08dda52cc11b*7C6e725c29763a4f5081f22e254f0133c8*7C1*7C0*7C638848333827133258*7CUnknown*7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ*3D*3D*7C0*7C*7C*7C&sdata=ttZVhG0rqxCiZE2x3WV5eNyvqyPo41*2Bmz5*2F0QYdjGU4*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!Mih3wA!BCK2WQoiHbDuDgBHPJ7TpKFxs97F1_kVNf_QRJVrj48KkDDixpyTHqk0J5_7i17Wei7MWIIUZf5RPBjPOblWDgrV5-8wTv0W4OJHKtI$ <https://sccn.ucsd.edu/mailman/listinfo/eeglablist >.