[Eeglablist] Permutation Analysis, Discrepancy Between Bonferroni and FDR

[J. L. Sanguinetti]

Hello,

I'm using EEGLAB v11 to run permutation statistics on a 2x2 design. I'm
correcting for multiple comparisons with both the FDR and Bonferroni
methods. As I understand those corrections, Bonferroni should be more
conservative and less sensitive. FDR should be the opposite: less
conservative but more sensitive.

When I run the analysis (using 1000 permutations), the output suggests that
FDR is (much) more conservative than Bonferroni. For now assume my
hypothesis is about an N400, I'm only using Cz, and I've constrained my
time window to 250 ms to 500 ms. If I do not set the alpha level, then I
get a flat line for the interaction graph (meaning p-values do not move
from 1) with the FDR. However if I run the same exact analysis with
Bonferroni on, then I get some significant points almost down to .01 in my
interaction graph.

The same is true for the 1x2 plots that also come up--Bonferroni shows
greater p-values than FDR in the N400 time range. I've also tried Holms (in
v12), which gives me almost identical outputs to Bonferroni (although Holms
should also be more sensitive than Bon).

Am I misunderstanding? Asking another way, which correction is most
appropriate for ERP data with a constrained time window?

Thanks,
Jay

-- 
-- 
J. L. Sanguinetti
Graduate Student
University of Arizona
http://tegestologist.googlepages.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20130113/fec6d827/attachment.html>