[Eeglablist] 4 questions about Study statistics

[Arnaud Delorme]

Hi Andrew,


> 1) Bootstrap and paired data: from what I gather I should not use bootstrap when looking at my Session data, so I've only looked at "Group" in this way.  If I look at paired data (Session) could I just used an alpha of .025 and assume equivalence to a .05 level if the bootstrapping was working properly with paired data?


Yes, as the warning indicates, the bootstraping method is not working properly for paired data. Simply use the permutation approach.


> 2) Non-bootstrap options:  for permutation or parametric, and calculating stats on first and second variable, the matlab window output suggests that I should not be running this type of ANOVA on the parametric/permutation data, e.g.:
> 
> ... (2 balanced 1-way ANOVAS and then 4 paired t-tests)
> 4 x 2, unpaired data, computing F values
> Using balanced 2-way ANOVA (not suitable for parametric testing, only bootstrap)
> ...


You have a mixed design (paired for one variable and unpaired for the other one). Therefore the balanced Anova is not strictly correct (both variables should be unpaired). For permutation, it does not really matter (the ANOVA computation is just a way to compute a combined 2x2 measure - and the pairing of values is preserved based on your settings during the permutation). There is no functions in Matlab to handle this special case to our knowledge, but there is one in "R". Email David Groppe for more info <dgroppe at cogsci.ucsd.edu>.


> 3) Use single trials (when available):
> I understand from reading old list posts that the null hypothesis when using trial based stats is restricted to consideration of a specific subject's trials against the possible population of trials for that subject.  What I don't understand is when I select "trials" with a multi-subject design, what is actually happening, e.g.:   Did I just violate the assumptions of the analysis, or is this analysis now running at the subject level for my paired data, and correcting somehow to calculate/plot the stats for a study with many subjects? 


No you are fine. You can use single-trial in a single subject and compute relevant statistics. If you have several subjects, then single trials from the different subjects will be concatenated. The NULL hypothesis is now about single trials of the subjects you have selected. I agree that this NULL hypothesis is quite limited.


> For (3), when I use this in my study and calculate the Session variable differences in spectrum, many/most of the electrodes are significant when I check several 3-hz bins.  When I don't use trial-based stats, almost none are (unless I un-check FDR).  


Yes, as mentioned above the NULL hypothesis is quite different. The degree of freedom is also quite different (for 10 subjects with 100 trials each, it would be 9 if you use averages and 999 if you use single trials).


> 4) FDR and Parametric:  Unchecking FDR for Parametric seems plausible, since wouldn't that mcorrect be some sort of Bonferroni, which would be overly conservative given the highly correlated local information in ERP waveforms?


Yes, you can set a threshold yourself using Bongeroni. For instance, if you know you are plotting 100 frequencies in your data spectrum, you may set your threshold at 0.05/100 = 0.0005 and you would not need to use FDR. This is true for both parametric and bootstrap/permutation methods. However, FDR is less conservative than Bonferoni.

Best wishes,

Arno

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20110314/4c0560d4/attachment.html>