[Eeglablist] Permutation Analysis, Discrepancy Between Bonferroni and FDR

[Makoto Miyakoshi]

Dear Jay,

FDR goes liberal only when you sufficiently amount of significant results.
Imagine a guy who is nice to rich guys but mean to poor guys... FDR is like
that.

Multiple comparisons on time-frequency maps are always pain in the neck. I
haven't tried it yet, but I heard the current EEGLAB supports cluster-level
statistics that should address the problem better. If you haven't tried it
yet please do (and let us know how it works!)

Makoto

2013/1/13 J. L. Sanguinetti <sanguine at email.arizona.edu>

> Hello,
>
> I'm using EEGLAB v11 to run permutation statistics on a 2x2 design. I'm
> correcting for multiple comparisons with both the FDR and Bonferroni
> methods. As I understand those corrections, Bonferroni should be more
> conservative and less sensitive. FDR should be the opposite: less
> conservative but more sensitive.
>
> When I run the analysis (using 1000 permutations), the output suggests
> that FDR is (much) more conservative than Bonferroni. For now assume my
> hypothesis is about an N400, I'm only using Cz, and I've constrained my
> time window to 250 ms to 500 ms. If I do not set the alpha level, then I
> get a flat line for the interaction graph (meaning p-values do not move
> from 1) with the FDR. However if I run the same exact analysis with
> Bonferroni on, then I get some significant points almost down to .01 in my
> interaction graph.
>
> The same is true for the 1x2 plots that also come up--Bonferroni shows
> greater p-values than FDR in the N400 time range. I've also tried Holms (in
> v12), which gives me almost identical outputs to Bonferroni (although Holms
> should also be more sensitive than Bon).
>
> Am I misunderstanding? Asking another way, which correction is most
> appropriate for ERP data with a constrained time window?
>
> Thanks,
> Jay
>
> --
> --
> J. L. Sanguinetti
> Graduate Student
> University of Arizona
> http://tegestologist.googlepages.com
>
>
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to
> eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
>



-- 
Makoto Miyakoshi
JSPS Postdoctral Fellow for Research Abroad
Swartz Center for Computational Neuroscience
Institute for Neural Computation, University of California San Diego
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20130114/1b670d41/attachment.html>