[Eeglablist] ERSP bootstrap outside of newtimef

Marius Klug marius.s.klug at gmail.com
Mon Nov 13 02:27:34 PST 2017

Hi Max,

okay first of all: I can very much recommend Mike X Cohen's Book "Analyzing
Neural Time Series Data", this, together with the scripts and your previous
thoughts should clarify a lot.

Short notes on your questions: 1. You want to compare 2 data sets
statistically: A baseline and the ERSP. If you have no baseline, what would
a significant pixel be anyway? Significantly different from what? For any
other baseline you need to compare the ERSP to that specific baseline.

Your understanding that this gives a pixel-by-pixel comparison is correct.
This is also the reason why this has nothing to do with multiple
comparison. It's just that: A pixel-by-pixel comparison of 2 conditions.
You have several pixels in your data, if you simulate an ERSP with random
data, some of the pixels will be signifcant, and that's what you want to
correct for.

The clustering of ICs in the study is absolutely different from ERSP
cluster-based multiple comparison and I at least can't think of a way this
would be of any use. The latter is using image processing to find clusters
of significant pixels that are next to each other, it's not a k-means
clustering or some of the like.

I can't tell you if it would be easier to use the EEGLAB study, I don't
know what's the state and nature of you data. You have to check that
yourself ;-) Be sure to stop by the eeglab tutorial on studies, if you
haven't yet!

The clusters you get for multiple comparison are just the significance
masking for your ERSP, nothing else. Cluster analysis throws out small
individual pixels that are significant but apear to be more of random noise
than actual significance. I don't know of a method in eeglab to do this
though and haven't used it myself yet (I'd like to, but don't have the

Again: Read the book. It will help you, especially the chapters about

Good luck!

2017-11-08 16:40 GMT+01:00 Max Cantor <Max.Cantor at colorado.edu>:

> Thanks for the advice! I've currently been looking at 3 cases- one where
> the ersp is not baselined, one where it is average baselined, and one where
> it is not baselined, and in all cases I've decided to use the mean baseline
> of each respective ersp in bootstat to test against. It sounds like my
> interpretation that this is comparing the effect point-by-point and
> frequency-by-frequency to the baseline was correct after all, which is
> reassuring. I understand that bootstrapping and monte carlo permutation are
> not exactly the same, but for my purposes I think are comparable enough.
> While I understood that bootstrapping is not itself clustering, I was
> under the impression that resampling is itself somewhat of a way around
> multiple comparisons (by comparing it against a probability distribution
> produced by the sample itself), so this is disconcerting! Do you have any
> advice for how to apply a cluster analysis either to the output of bootstat
> (or in some way as part of its input?), either in matlab or r? Or will this
> require me to completely write the entire statistical method from scratch!?
> Thanks,
> Max
> On Wed, Nov 8, 2017 at 8:25 AM, Marius Klug <marius.s.klug at gmail.com>
> wrote:
>> Hi Max,
>> I've recently done the same procedure myself (manually creating ERSPs and
>> calculating statistics). What you want to do in bootstat is to compare the
>> NON-baselined ERSPs to an ERSP of the same size, consisting only of the
>> baseline (repeated for each time point). I believe it essentially is the
>> same to just enter the already baselined ERSP and let it be compared to
>> zero (which should mathematically not make any difference, I just don't
>> know the implementation so beware), so maybe you've just done the right
>> thing already. But make sure it is so!
>> What you might also want to consider is to correct your ERSP for multiple
>> comparison using fdr for example, or better use a cluster-based approach.
>> The latter will require you to basically calculate everything yourself,
>> though.
>> I hope this has helped you, I know it's a tricky thing to do the stuff
>> manually ;-)
>> Cheers,
>> Marius
>> 2017-10-26 18:00 GMT+02:00 Max Cantor <Max.Cantor at colorado.edu>:
>>> Following up on my attempt to use bootstat on an ersp array outside of
>>> pop_newtimef: I think I have now gotten the former implementation working
>>> (shuffling over samples on the across-channels mean difference grand
>>> average ersp)! Hopefully this will be useful if anyone else tries to do
>>> what I'm doing in the future.
>>> I was including a baseline vector in the bootstat input since my ersps
>>> were baselined along those samples and I assumed that that should be
>>> accounted for in bootstat. However, when I take the basevect out (meaning,
>>> if I understand correctly, that bootstat demeans across the whole ersp,
>>> which I guess is appropriate since the ersp is already baselined?) and
>>> compare ersp plots of the "raw" esrp and the masked ersp, I get what look
>>> like sensible time-frequency clusters. Maybe it would be better to not
>>> baseline my ersps, but then include the baseline in bootstat?
>>> Does what I've done make methodological sense? Conceptually, am I
>>> correct in assuming that this is similar to the fieldtrip cluster
>>> permutation test? I see a paper cited in the bootstrap function that I
>>> intend to read, although some direct confirmation/explanation would
>>> certainly be appreciated!
>>> Thanks,
>>> Max
>>> On Mon, Oct 23, 2017 at 5:05 PM, Max Cantor <Max.Cantor at colorado.edu>
>>> wrote:
>>>> Ok, last followup for today! I have tried two things- the above, and
>>>> another method which I'll discuss below. In both cases, comparing tftopo
>>>> plots both with the signifs argument and without it turned out identically,
>>>> which makes me skeptical of whether what I'm doing is working at all.
>>>> The second method I'm trying is:
>>>> *    idx = 1;*
>>>> *    for k = freqs*
>>>> *        [rsignif(idx,:) rbot{idx}] = bootstat({a(idx,:) b(idx,:)},
>>>> 'arg1 - arg2', ...*
>>>> *            'basevect', [27 28 29 30 31 32 33 34 35 36 37], 'alpha',
>>>> 0.01, 'dimaccu', 2);*
>>>> *        idx = idx+1;*
>>>> *    end*
>>>> Where a and b are freqs x samples matrices for the two conditions,
>>>> averaged across my channels of interest. So it loops through each
>>>> frequency, shuffles samples, and if I understand correctly, should make a
>>>> comparison across permutations based on the difference between the two
>>>> conditions. I feel like either method should be fine for the purpose of
>>>> masking the ersps and I'm not even sure if the rsignifs for these two
>>>> methods would even be all that different, but the fact that it doesn't seem
>>>> to be working suggests I'm doing something wrong or misunderstanding
>>>> something.
>>>> Any advice appreciated!
>>>> Best,
>>>> Max
>>>> On Mon, Oct 23, 2017 at 4:13 PM, Max Cantor <Max.Cantor at colorado.edu>
>>>> wrote:
>>>>> To follow up, I believe I have gotten the code to "work", although I
>>>>> would appreciate confirmation on whether what I'm doing is conceptually
>>>>> sound, or if I am misunderstanding the code or purpose of the statistic.
>>>>> Below is my code:
>>>>> *[rsignif rbot] = bootstat(permute(g_ersp, [2,1,3]), 'mean(arg1,3);',
>>>>> 'basevect', [27 28 29 30 31 32 33 34 35 36 37], 'alpha', 0.01, 'dimaccu',
>>>>> 2);*
>>>>> Where g_ersp is a grand average of ersp of 200 samples (of data from ~
>>>>> -1000 to 2000ms) x 48 frequencies (log-spaced) x 7 channels. This grand
>>>>> average is the difference between two conditions, averaged across all
>>>>> subjects, where each subjects ersp is an average across all trials. I
>>>>> permute the matrix so that it is freqs x samples x chans, which is similar
>>>>> to what bootstat is doing on a subject-by-subject basis in my pop_newtimef
>>>>> code, except there it's only one condition, not a difference between
>>>>> conditions, and the third dimension is trials instead of channels.
>>>>> The mean(arg1,3) formula came directly from debugging in bootstat from
>>>>> pop_newtimef. While in pop_newtimef this would be averaging across trials,
>>>>> if I'm interested in finding which time-frequency clusters are significant
>>>>> across all channels (or from the average across all channels), does this
>>>>> still make sense?
>>>>> The baseline vector comes from the baseline I used on these ersp data,
>>>>> alpha is consistent with what I did on the single-subject/single-channel
>>>>> level, and I don't even know what dimaccu is but this was also taken from
>>>>> debug.
>>>>> rsignif ends up being my freq x 2 matrix which I can input in tftopo,
>>>>> so I run this:
>>>>> *tftopo(mean(g_ersp,3), times, freqs, 'logfreq', 'native', 'signifs',
>>>>> rsignif);*
>>>>> So I'm plotting the grand-average difference ersp averaged across all
>>>>> channels, using the appropriate times and freqs axis, the ersp were
>>>>> natively log-scaled, and my mask is rsignif from above.
>>>>> To reiterate, is this mask appropriate for these data? Is this showing
>>>>> me the time-frequency clusters averaged across all channels which are
>>>>> significant (relative to my alpha)?
>>>>> Thanks,
>>>>> Max
>>>>> On Thu, Oct 19, 2017 at 5:33 PM, Max Cantor <Max.Cantor at colorado.edu>
>>>>> wrote:
>>>>>> Hi,
>>>>>> I've produced grand-averaged TFRs across all of my subjects, at
>>>>>> select channels, and taken the difference between two conditions, by
>>>>>> looping through pop_newtimef for each subject and each channel of interest
>>>>>> across both conditions separately, outputting these into separate 4-D
>>>>>> arrays, averaging across subjects, and then taking the difference between
>>>>>> the two arrays, such that I end up with an ersp difference array of freq x
>>>>>> timepoint x chan.
>>>>>> This array is compatible with the tftopo function and I would like to
>>>>>> apply a bootstrapped statistical mask to this array through the tftopo
>>>>>> function. Testing it with an erspboot mask outputted by pop_newtimef was
>>>>>> functional, however this erspboot mask was obviously specific to that
>>>>>> particular subject and channel and not appropriate for my grand averaged
>>>>>> array.
>>>>>> Is there a conceptually and computationally simple way to run this
>>>>>> array through the bootstat function (which I believe is what's producing
>>>>>> the statistical masking) outside of pop_newtimef and produce a freqs x 2
>>>>>> erspboot mask that would be compatible with tftopo, or would it be easier
>>>>>> for me to just reformat my array into a fieldtrip structure and use their
>>>>>> functions for nonparametric cluster-based permutation testing, which to the
>>>>>> best of my knowledge is the mathematical implementation eeglab uses anyway?
>>>>>> Any advice appreciated.
>>>>>> Thanks,
>>>>>> Max
>>>>>> --
>>>>>> Max Cantor
>>>>>> Graduate Student
>>>>>> Cognitive Neuroscience of Language Lab
>>>>>> University of Colorado Boulder
>>>>> --
>>>>> Max Cantor
>>>>> Graduate Student
>>>>> Cognitive Neuroscience of Language Lab
>>>>> University of Colorado Boulder
>>>> --
>>>> Max Cantor
>>>> Graduate Student
>>>> Cognitive Neuroscience of Language Lab
>>>> University of Colorado Boulder
>>> --
>>> Max Cantor
>>> Graduate Student
>>> Cognitive Neuroscience of Language Lab
>>> University of Colorado Boulder
>>> _______________________________________________
>>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.uc
>>> sd.edu
>>> For digest mode, send an email with the subject "set digest mime" to
>>> eeglablist-request at sccn.ucsd.edu
> --
> Max Cantor
> Graduate Student
> Cognitive Neuroscience of Language Lab
> University of Colorado Boulder
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.
> ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20171113/ea150e7e/attachment.html>

More information about the eeglablist mailing list