[Eeglablist] Inconsistent statistical output from cluster-based permutation

[Cedric Cannard]

Hi Noelle,

Yes this is likely because of the permutation statistics, results can vary slightly. Your effet must be on the edge of the significance threshold (i.e., small effect or false positive?). 

You could try the threshold-free cluster enhancement (TFCE) method, as self-explained by the name, it is threshold-free, unlike the spatiotemporal cluster correction that has an arbitrary threshold. 
I know it is available via the LIMO-EEG plugin (Hierarchical linear modeling), but can't remember if it is also available in the standard EEGLAB/Fieldtrip statistics. 

You could try LIMO anyway, the weighted least squares (WLS) optimization might help increase the signal-to-noise ratio (SNR) in your data (by down-weighting any remaining bad trials). see here for tuto/wiki: https://urldefense.com/v3/__https://github.com/LIMO-EEG-Toolbox/limo_tools/wiki__;!!Mih3wA!Bh79I16GAtpQZrsgDIpQBVx45YHKo9Gdxy4BYeta17WnyBR1ZsJhaZJG_yRDMpvpQxXeuHoW6e2vPK0urEm1hEO3LQ$ 

Hope this helps,

Cedric



On Monday, March 4th, 2024 at 8:35 AM, Jacobsen,Noelle A via eeglablist <eeglablist at sccn.ucsd.edu> wrote:

> Hello,
> 
> I am trying to use cluster-based permutation in Fieldtrip (called by EEGLAB's std_stat) to test for differences between two event-related spectral perturbation plots (ERSPs). I find that when I re-run std_stat, I don't get the same result every time. Sometimes a pixel cluster is significant, and sometimes it is not. For example, when running std_stat 10 times, it found a significant cluster 80% of the time.
> 
> I'm guessing the inconsistency in the output is related to the randomization involved in calculating the parametric threshold for clustering. What's the best method to achieve reproducible results? I could repeat std_stat a fixed number of times and only consider a pixel cluster significant if it exceeds the significance threshold a majority of the time, but then how would I calculate the p-values and corresponding effect sizes? Am I doing something wrong? Please advise.
> 
> I am using Matlab 2022a, EEGLAB 2022.0 and the following Fieldtrip plugin: Fieldtrip-lite20230716. My statistical parameters are as follows:
> 
> 
> * stats.paired: 'on'
> * stats.effect: 'marginal'
> * stats.mode: 'fieldtrip'
> * stats.fieldtrip:
> * naccu: 10000
> * method: 'montecarlo'
> * alpha: 0.0500
> * mcorrect: 'cluster'
> * clusterparam: ''clusterstatistic','maxsum''
> * channelneighbor: [0×0 struct]
> * channelneighborparam: ''method','triangulation''
> 
> Thanks,
> 
> Noelle Jacobsen
> University of Florida / Imperial College London
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu