[Eeglablist] SIFT resampling surrogate distributions with 1 trial

[Winslow Strong]

Hi Tim,

Yes I was searching for some approximate test stats and p-vals generated by
creating pseudotrials within each trial.  I'll try this out.  I'm thinking
it might be wise to leave a gap between the pseudo trials (i.e. not make
them contiguous EEG segments) to make them closer to independent.  Leaving
out every-other pseudotrial might be a reasonable tradeoff.  One could get
2 test stats or just 2 sample variances this way: one from the even
pseudotrials and one from the odds.

This is a bit hacky though, and I wonder if there are canonical methods to
deal with the lack of independence.

On Mon, Aug 22, 2016 at 12:59 PM, Tim Mullen <mullen.tim at gmail.com> wrote:

> Winslow, Makoto,
>
> As a statistical principle, bootstrapping can only be used when you have
> multiple independent and identically distributed (i.i.d) observations
> available. The observations are resampled with replacement from the
> original set to construct an empirical probability distribution.
>
> It is not possible to use bootstrapping to test for statistical
> differences between only two observations (i.e. two trials). In general,
> with any test, your statistical power will be extremely low if you have
> only one observation per condition.
>
> You can try to mitigate this by segmenting your long continuous trials
> into short 'pseudo-trials' and then testing for differences in the
> pseudo-trial between conditions. Make sure that you average your causal
> measure over time within each trial before computing your stats. One
> concern is that the pseudotrials may be far from i.i.d within a condition,
> so if using bootstrap, your bootstrap distribution may not converge to the
> true distribution of the estimator and your stats will be biased.
>
> Depending on your specific null hypothesis and protocol, however, there
> may be alternative parametric and nonparametric tests you can apply.
>
> Otherwise try to collect data for more subjects (then you simply bootstrap
> across subjects e.g. using statcond with SIFT matrices) or more trials (run
> your experiment more than once per condition).
>
> Tim
>
> On Aug 18, 2016 11:09 AM, "Makoto Miyakoshi" <mmiyakoshi at ucsd.edu> wrote:
>
>> Dear Winslow,
>>
>> Yes, unfortunately the bootstrap seems to be designed for across trials.
>>
>> Makoto
>>
>> On Sat, Aug 13, 2016 at 4:57 PM, Winslow Strong <winslow.strong at gmail.com
>> > wrote:
>>
>>> I'd like to use a resampling technique (e.g. bootstrap) to get p-values
>>> and test stats for SIFT connectivity metrics for 1 subject across n
>>> conditions.
>>>
>>> This is a steady-state condition study, hence there's only 1 trial per
>>> condition.  I'm trying to analyze whether certain connectivity metrics
>>> (i.e. their averages over a condition) are statistically significantly
>>> different across the conditions.  I was under the impression I could use
>>> SIFT's surrogate distribution generator to obtain the surrogate
>>> distribution for these calculations, but when I run that from the GUI for
>>> bootstrap, I get the error:
>>>
>>> "Unable to compute bootstrap distributions for a single trial"
>>>
>>> Is this surrogate function only designed to do boostrapping over trials?
>>> Or is there a way to do it over windows within a condition?
>>>
>>> _______________________________________________
>>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.uc
>>> sd.edu
>>> For digest mode, send an email with the subject "set digest mime" to
>>> eeglablist-request at sccn.ucsd.edu
>>>
>>
>>
>>
>> --
>> Makoto Miyakoshi
>> Swartz Center for Computational Neuroscience
>> Institute for Neural Computation, University of California San Diego
>>
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.uc
>> sd.edu
>> For digest mode, send an email with the subject "set digest mime" to
>> eeglablist-request at sccn.ucsd.edu
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20160823/85d0861e/attachment.html>