[Eeglablist] basic statistic question bootstrap

[Allan Campopiano]

Hi David,


I have not used the native bootstrapped ANOVA function in EEGLAB so I can't
comment on its output. However, based on your questions, I will comment on
a few things that I hope will help. Most of what I know about these methods
comes from Rand Wilcox's writings, especially "Introduction to Robust
Estimation and Hypothesis Testing".


Of all the bootstrapped methods, two are talked about most frequently:
bootstrap-t and the percentile bootstrap. Both methods can be extended to
compare multiple groups, as you would in an ANOVA. I assume that EEGLAB
uses the bootstrap-t method when dealing with ANOVAs (just a guess), so
I'll describe what I think is going on under the hood.


The bootstrapped-t method uses the data at hand to estimate an appropriate
critical value. This is accomplished by (1) centering the group means so
that the null hypothesis is true (i.e.,  Yij = Xij - µj), (2) resampling
with replacement from the centered values, and then (3) calculating your F
statistic (see Wilcox's book for details on this). This is repeated
thousands of times yielding an empirically derived null distribution. If
the F statistic based on the original (non-centered) data exceeds the .95
quantile of this distribution, reject the null (publish, get famous, etc…)


For the write-up, I would describe the method and report the F value,
degrees of freedom, number of bootstrap samples, and the p value. See Dien,
Michelson, and Franklin (2010) for an example.


You are probably correct in that EEGLAB gives you a parametric F value in
the output somewhere. That is the test statistic based on the original data
that is being compared to the critical value of the null distribution.


Lastly, it does not surprise me that the p values are different when
comparing the bootstrap test to the traditional ANOVA. The two methods rely
on different sampling distributions (empirical vs. theoretical).
Furthermore, EEGLAB's bootstrapped ANOVA might implement some kind of
outlier control (trimmed means), or protection against violations of
normality or heteroscedasticity (Welch-type method), which all affect the p
value.


I hope this helps in some way,


Al



*Allan Campopiano* | MA Candidate
Laboratory of Cognitive and Affective Neuroscience
Brock University | Psychology Department | 500 Glenridge Ave.
St. Catharines, ON Canada L2S 3A1
*T* 905-688-5550 x3451 *F *905-688-6922


On Thu, Aug 28, 2014 at 6:58 PM, david grahms <david.grahms at gmail.com>
wrote:

> Hi,
>
> I would appreciate some help in understanding the output the bootstrap
> statistic implemented in eeglab and how to report it. Should I report the
> df and F-value when I perform bootstrap analysis? Am I correct in that I
> get the parametric F-value when I run the bootstrap analysis?
>
> Also
> I find it hard to understand that the bootstrap analysis gives me a
> p=0.00024988 and parametric gives me F(3,30)=0.96, p=0.42629.
>
> Regards,
> David
>
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to
> eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20140901/843945e2/attachment.html>