[Eeglablist] Bootstrap significance probability level
arno at salk.edu
Mon Nov 7 13:21:18 PST 2005
>I wonder, whether anyone has some practical considerations about which level
>of significance probability should be chosen for bootstrap statistics.
>Since we have to deal with multiple comparisons, the significance threshold
>should be decreased, but the question is how much. As much as I understand,
>the Bonferroni correction for multiple testing would correspond to a level
>of p < 0.05/(number of time points * number of frequency bins). However,
>this would require a very high amount of surrogate data.
The problem is that in any time-frequency decomposition, neighboring
time-frequency points are highly correlated so that standard bonferoni
correction (that assumes independent observations) are overly
conservative. In the following paper, a method is decribed to calculated
the minimum time and frequency resolution (number of independent
time-frequency estimates - about 200 in our case per plot):
High-frequency gamma-band activity in the basal temporal cortex during
picture-naming and lexical-decision tasks.
J Neurosci. 2005 Mar 30;25(13):3287-93.
Tanji K, Suzuki K, Delorme A, Shamoto H, Nakasato N.
Better than the bonferoni methos is also the Holm's method. First choose
a significance level p= (e.g., p=0.05). Then compute the exact p-value
for each test (which is usually possible using modern computerized
approaches). Rank the collection of p-values from smallest to largest.
The smallest p-value is tested against /N, where N is the number of
tests. If the smallest p-value is not less than /N, stop the procedure.
However, if it is less than /N, proceed to test the second smallest
p-value against /(N-1), etc...
>Most of the studies seem to use p-values < 0.01. What's the argumentation
>In their J Neurosci Meth; 2004:9-21 Paper, Arnaud Delorme and Scott Makeig
>write that they have
>"implemented a method to fit the observed data distribution using a forth
>order distribution fit. This feature will be available in a near-term release
>Does this feature allow a significance level of 0.05? Is there a way to obtain it?
Based on discussions with R Oostenveld, we actually changed our mind
about this idea and now prefer to accumulate more values. The reason is
that, using these functions, we were fitting a distribution and trying
to make inference about the tails of this distribution using the fitted
curve. However the fitting might be especially innacurate at the tails
(where there is few data points) so to be conservative it seems more
appropriate to accumulate more values than to try to fit the distribution.
>Thank you very much for your help.
>Adrian Guggisberg, MD
>Neurologische Klinik und Poliklinik
>Your Site for Swiss Maps: http://www.swissinfo-geo.org/
>eeglablist mailing list eeglablist at sccn.ucsd.edu
>Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
*Arnaud Delorme, Ph.D.*
Swartz Center for Computational Neuroscience, INC, University of San
La Jolla, CA92093-0961, USA
*Tel* :/(+1)-858-458-1927 ext 15/
*Web page*: sccn.ucsd.edu/~arno <http://www.sccn.ucsd.edu/%7Earno>
*To think upon*:
In spite of all our wanderings HAPPINESS is always found within a
narrow compass and among objects which lie within our immediate reach.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the eeglablist