[Eeglablist] ERSP bootstrap outside of newtimef

[Max Cantor]

Thanks for the advice! I've currently been looking at 3 cases- one where
the ersp is not baselined, one where it is average baselined, and one where
it is not baselined, and in all cases I've decided to use the mean baseline
of each respective ersp in bootstat to test against. It sounds like my
interpretation that this is comparing the effect point-by-point and
frequency-by-frequency to the baseline was correct after all, which is
reassuring. I understand that bootstrapping and monte carlo permutation are
not exactly the same, but for my purposes I think are comparable enough.

While I understood that bootstrapping is not itself clustering, I was under
the impression that resampling is itself somewhat of a way around multiple
comparisons (by comparing it against a probability distribution produced by
the sample itself), so this is disconcerting! Do you have any advice for
how to apply a cluster analysis either to the output of bootstat (or in
some way as part of its input?), either in matlab or r? Or will this
require me to completely write the entire statistical method from scratch!?

Thanks,
Max

On Wed, Nov 8, 2017 at 8:25 AM, Marius Klug <marius.s.klug at gmail.com> wrote:

> Hi Max,
>
> I've recently done the same procedure myself (manually creating ERSPs and
> calculating statistics). What you want to do in bootstat is to compare the
> NON-baselined ERSPs to an ERSP of the same size, consisting only of the
> baseline (repeated for each time point). I believe it essentially is the
> same to just enter the already baselined ERSP and let it be compared to
> zero (which should mathematically not make any difference, I just don't
> know the implementation so beware), so maybe you've just done the right
> thing already. But make sure it is so!
>
> What you might also want to consider is to correct your ERSP for multiple
> comparison using fdr for example, or better use a cluster-based approach.
> The latter will require you to basically calculate everything yourself,
> though.
>
> I hope this has helped you, I know it's a tricky thing to do the stuff
> manually ;-)
>
> Cheers,
> Marius
>
> 2017-10-26 18:00 GMT+02:00 Max Cantor <Max.Cantor at colorado.edu>:
>
>> Following up on my attempt to use bootstat on an ersp array outside of
>> pop_newtimef: I think I have now gotten the former implementation working
>> (shuffling over samples on the across-channels mean difference grand
>> average ersp)! Hopefully this will be useful if anyone else tries to do
>> what I'm doing in the future.
>>
>> I was including a baseline vector in the bootstat input since my ersps
>> were baselined along those samples and I assumed that that should be
>> accounted for in bootstat. However, when I take the basevect out (meaning,
>> if I understand correctly, that bootstat demeans across the whole ersp,
>> which I guess is appropriate since the ersp is already baselined?) and
>> compare ersp plots of the "raw" esrp and the masked ersp, I get what look
>> like sensible time-frequency clusters. Maybe it would be better to not
>> baseline my ersps, but then include the baseline in bootstat?
>>
>> Does what I've done make methodological sense? Conceptually, am I correct
>> in assuming that this is similar to the fieldtrip cluster permutation test?
>> I see a paper cited in the bootstrap function that I intend to read,
>> although some direct confirmation/explanation would certainly be
>> appreciated!
>>
>> Thanks,
>> Max
>>
>> On Mon, Oct 23, 2017 at 5:05 PM, Max Cantor <Max.Cantor at colorado.edu>
>> wrote:
>>
>>> Ok, last followup for today! I have tried two things- the above, and
>>> another method which I'll discuss below. In both cases, comparing tftopo
>>> plots both with the signifs argument and without it turned out identically,
>>> which makes me skeptical of whether what I'm doing is working at all.
>>>
>>> The second method I'm trying is:
>>>
>>> *    idx = 1;*
>>> *    for k = freqs*
>>> *        [rsignif(idx,:) rbot{idx}] = bootstat({a(idx,:) b(idx,:)},
>>> 'arg1 - arg2', ...*
>>> *            'basevect', [27 28 29 30 31 32 33 34 35 36 37], 'alpha',
>>> 0.01, 'dimaccu', 2);*
>>> *        idx = idx+1;*
>>> *    end*
>>>
>>> Where a and b are freqs x samples matrices for the two conditions,
>>> averaged across my channels of interest. So it loops through each
>>> frequency, shuffles samples, and if I understand correctly, should make a
>>> comparison across permutations based on the difference between the two
>>> conditions. I feel like either method should be fine for the purpose of
>>> masking the ersps and I'm not even sure if the rsignifs for these two
>>> methods would even be all that different, but the fact that it doesn't seem
>>> to be working suggests I'm doing something wrong or misunderstanding
>>> something.
>>>
>>> Any advice appreciated!
>>>
>>> Best,
>>> Max
>>>
>>> On Mon, Oct 23, 2017 at 4:13 PM, Max Cantor <Max.Cantor at colorado.edu>
>>> wrote:
>>>
>>>> To follow up, I believe I have gotten the code to "work", although I
>>>> would appreciate confirmation on whether what I'm doing is conceptually
>>>> sound, or if I am misunderstanding the code or purpose of the statistic.
>>>>
>>>> Below is my code:
>>>>
>>>> *[rsignif rbot] = bootstat(permute(g_ersp, [2,1,3]), 'mean(arg1,3);',
>>>> 'basevect', [27 28 29 30 31 32 33 34 35 36 37], 'alpha', 0.01, 'dimaccu',
>>>> 2);*
>>>>
>>>> Where g_ersp is a grand average of ersp of 200 samples (of data from ~
>>>> -1000 to 2000ms) x 48 frequencies (log-spaced) x 7 channels. This grand
>>>> average is the difference between two conditions, averaged across all
>>>> subjects, where each subjects ersp is an average across all trials. I
>>>> permute the matrix so that it is freqs x samples x chans, which is similar
>>>> to what bootstat is doing on a subject-by-subject basis in my pop_newtimef
>>>> code, except there it's only one condition, not a difference between
>>>> conditions, and the third dimension is trials instead of channels.
>>>>
>>>> The mean(arg1,3) formula came directly from debugging in bootstat from
>>>> pop_newtimef. While in pop_newtimef this would be averaging across trials,
>>>> if I'm interested in finding which time-frequency clusters are significant
>>>> across all channels (or from the average across all channels), does this
>>>> still make sense?
>>>>
>>>> The baseline vector comes from the baseline I used on these ersp data,
>>>> alpha is consistent with what I did on the single-subject/single-channel
>>>> level, and I don't even know what dimaccu is but this was also taken from
>>>> debug.
>>>>
>>>> rsignif ends up being my freq x 2 matrix which I can input in tftopo,
>>>> so I run this:
>>>>
>>>>
>>>> *tftopo(mean(g_ersp,3), times, freqs, 'logfreq', 'native', 'signifs',
>>>> rsignif);*
>>>>
>>>> So I'm plotting the grand-average difference ersp averaged across all
>>>> channels, using the appropriate times and freqs axis, the ersp were
>>>> natively log-scaled, and my mask is rsignif from above.
>>>>
>>>> To reiterate, is this mask appropriate for these data? Is this showing
>>>> me the time-frequency clusters averaged across all channels which are
>>>> significant (relative to my alpha)?
>>>>
>>>> Thanks,
>>>> Max
>>>>
>>>>
>>>>
>>>> On Thu, Oct 19, 2017 at 5:33 PM, Max Cantor <Max.Cantor at colorado.edu>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've produced grand-averaged TFRs across all of my subjects, at select
>>>>> channels, and taken the difference between two conditions, by looping
>>>>> through pop_newtimef for each subject and each channel of interest across
>>>>> both conditions separately, outputting these into separate 4-D arrays,
>>>>> averaging across subjects, and then taking the difference between the two
>>>>> arrays, such that I end up with an ersp difference array of freq x
>>>>> timepoint x chan.
>>>>>
>>>>> This array is compatible with the tftopo function and I would like to
>>>>> apply a bootstrapped statistical mask to this array through the tftopo
>>>>> function. Testing it with an erspboot mask outputted by pop_newtimef was
>>>>> functional, however this erspboot mask was obviously specific to that
>>>>> particular subject and channel and not appropriate for my grand averaged
>>>>> array.
>>>>>
>>>>> Is there a conceptually and computationally simple way to run this
>>>>> array through the bootstat function (which I believe is what's producing
>>>>> the statistical masking) outside of pop_newtimef and produce a freqs x 2
>>>>> erspboot mask that would be compatible with tftopo, or would it be easier
>>>>> for me to just reformat my array into a fieldtrip structure and use their
>>>>> functions for nonparametric cluster-based permutation testing, which to the
>>>>> best of my knowledge is the mathematical implementation eeglab uses anyway?
>>>>>
>>>>> Any advice appreciated.
>>>>>
>>>>> Thanks,
>>>>> Max
>>>>>
>>>>> --
>>>>> Max Cantor
>>>>> Graduate Student
>>>>> Cognitive Neuroscience of Language Lab
>>>>> University of Colorado Boulder
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Max Cantor
>>>> Graduate Student
>>>> Cognitive Neuroscience of Language Lab
>>>> University of Colorado Boulder
>>>>
>>>
>>>
>>>
>>> --
>>> Max Cantor
>>> Graduate Student
>>> Cognitive Neuroscience of Language Lab
>>> University of Colorado Boulder
>>>
>>
>>
>>
>> --
>> Max Cantor
>> Graduate Student
>> Cognitive Neuroscience of Language Lab
>> University of Colorado Boulder
>>
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.uc
>> sd.edu
>> For digest mode, send an email with the subject "set digest mime" to
>> eeglablist-request at sccn.ucsd.edu
>>
>
>


-- 
Max Cantor
Graduate Student
Cognitive Neuroscience of Language Lab
University of Colorado Boulder
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20171108/d3f636ab/attachment.html>