[Eeglablist] Inconsistent statistical output from cluster-based permutation

[Jacobsen,Noelle A]

Hello,

I am trying to use cluster-based permutation in Fieldtrip (called by EEGLAB's std_stat) to test for differences between two event-related spectral perturbation plots (ERSPs). I find that when I re-run std_stat, I don't get the same result every time. Sometimes a pixel cluster is significant, and sometimes it is not. For example, when running std_stat 10 times, it found a significant cluster 80% of the time.

I'm guessing the inconsistency in the output is related to the randomization involved in calculating the parametric threshold for clustering. What's the best method to achieve reproducible results? I could repeat std_stat a fixed number of times and only consider a pixel cluster significant if it exceeds the significance threshold a majority of the time, but then how would I calculate the p-values and corresponding effect sizes? Am I doing something wrong? Please advise.

I am using Matlab 2022a, EEGLAB 2022.0 and the following Fieldtrip plugin: Fieldtrip-lite20230716. My statistical parameters are as follows:


  *   stats.paired: 'on'
  *   stats.effect: 'marginal'
  *   stats.mode: 'fieldtrip'
  *   stats.fieldtrip:
     *   naccu: 10000
     *   method: 'montecarlo'
     *   alpha: 0.0500
     *   mcorrect: 'cluster'
     *   clusterparam: ''clusterstatistic','maxsum''
     *   channelneighbor: [0×0 struct]
     *   channelneighborparam: ''method','triangulation''

Thanks,

Noelle Jacobsen
University of Florida / Imperial College London