[Eeglablist] Permutation Tests for Agreement of Three or More Confusion Matrices

[Arnaud Delorme]

Dear Aleksnadra,

> a.     do we check all possible options (e.g. if I have three groups then I’ll swap values between 1 and 2, leave 3 unchanged, then between 1 and 3 leave 2 unchanged etc) or we do just a certain percentage of it, and what percentage?

No, we use a montecarlo method as described in the document you sent. However, at the STUDY level, EEGLAB also allows you to interfaces statistics in Fieldtrip and you may check all possible combination when using these statistics. The percentage depends on the number of conditions. Basically, EEGLAB will perform between 2000 and 20000 permutations. The exact number is shown on the command line. You can then calculate the percentage.

> b.     Would bootstrapping for the same analysis be any more reliable?

Bootstrapping picks conditions and/or trials with replacement. In your example in (a), it means that "1" could be picked twice. Bootstrapping and permutation return similar results for most practical purposes. Both assumes that the best estimate of the population is the sample data. However, bootstrap is more rigorous since picking up one trial does not depend on what you picked before. In other words, it is a more realistic sampling of your population.

Best,

Arno


>  
> Many thanks,
> Aleksnadra
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" toeeglablist-request at sccn.ucsd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20130328/c8446fc6/attachment.html>