[Eeglablist] Fwd: How to high-pass filter eeg data containing NaNs?

[leo budinich]

---------- Forwarded message ----------
From: leo budinich <leo.budinich at edu.unito.it>
Date: 2016-05-13 10:56 GMT+02:00
Subject: Re: [Eeglablist] How to high-pass filter eeg data containing NaNs?
To: Andreas Widmann <widmann at uni-leipzig.de>


Thanks Andreas! Your answer has been really helpful!
Now I understand a little better how filtering works...
I'll make a try with cutting away the NaN parts and then applying the
high-pass filter, keeping in mind that it's highly possible to incur in
edge-effects.
I think also that I am going make a try with a more traditional approach to
baseline correction (subtracting the mean amplitude of the 100ms preceding
the target stimulus).

I would like also to try what Carolina proposed (transforming NaNs in zeros
and then filtering the set): do you think it could work?

Thanks,
Leo

2016-05-12 16:25 GMT+02:00 Andreas Widmann <widmann at uni-leipzig.de>:

> Hi Leo,
>
> filtering is the convolution of two signals-the data and the filter’s
> impulse response. The impulse response has a duration, 1651 samples or 6.6
> seconds in your case. If any sample of both signals is NaN the output is
> also NaN. That’s why your filter output mainly consists of NaNs.
> Unfortunately, there’s is not so much one could do. Thus, I would consider
> the decision NaNing large parts of the continuous data as, hm, suboptimal.
>
> Pragmatically, the only thing you could do is cutting away the NaN parts
> (the data are effectively kind of „epoched“ already anyway).
>
> tmp = any( isnan( EEG.data ), 1 );
> onsetArray = find( diff( [ 0 tmp ] ) == 1 );
> offsetArray = find( diff( [ tmp 0 ] ) == -1 );
> EEG = eeg_eegrej( EEG, [ onsetArray; offsetArray ]' );
> [ ALLEEG, EEG, CURRENTSET ] = pop_newset( ALLEEG, EEG, CURRENTSET );
> eeglab redraw
>
> Now, at least there is signal in the filter output where there was signal
> in the filter input.
>
> I would, however, recommend being careful with the further analysis of the
> data and interpretation of the results. High-pass filtering should always
> be done on the continuous data. You have to expect edge-effects up to the
> duration of the impulse response at the beginning and end of each epoch
> (actually all data regions which were signal in the filter input and NaN in
> the filter output before; essentially the observed NaNs reflect some kind
> of edge-effect themselves). Type, size, and duration of the edge-effects
> depend on impulse response length and whether and how data are padded
> during filtering but there is nothing you could do against edge-effects in
> general. Given the minimal required durations of the impulse responses for
> the filter cutoffs suggested in the paper I would consider the final
> sentence of footnote 3 as incorrect. This does also not depend on filter
> type (unfortunately not described in detail in the paper).
>
> Finally, you have to be careful with high-pass filtering with higher
> cutoffs in ERP analysis in general. Recent relevant literature on this
> issue you may find at the end of this page:
> http://sccn.ucsd.edu/wiki/Firfilt_FAQ
>
> Hope this helps!
> Andreas
>
> > Am 11.05.2016 um 15:38 schrieb leo budinich <leo.budinich at edu.unito.it>:
> >
> > Hi Eeglablist,
> >
> > I'm writing here because I'm a new user of EEGLAB and I am facing a
> problem with high-pass filtering some data.
> >
> > I am trying to apply on a freely downloadable dataset some of the same
> processing steps that have been used by the researchers that provided the
> data (see http://www.stefanfrank.info/pubs/BL2015.pdf).
> > The set I want to filter has 32 channels and has already been been
> band-pass filtered at 0.05Hz - 25Hz, recalibrated and re-referenced to the
> mastoids.
> >
> > An important detail is that the EEG recordings concern the reading of
> two hundred english sentences; the data recorded between the presentations
> of two sentences have been "set to NaN" by the researchers who provided the
> dataset, so the channel data of, e.g. subject01, presents this aspect when
> you inspect it:
> >
> >
> <-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
> >
> >
> signal-signal-signal-signal-signal-NaN-NaN-NaN-NaN-signal-signal-signal-signal
> >
> >
> > What I would like to do is high-pass filter the entire set to 0.50Hz, as
> a way to "mitigate the baseline problem by reducing the correlation between
> the baselines and amplitudes by applying an additional high-pass filter
> with a sufficiently high cut-off frequency" (see the reference, pp.4).
> >
> > Unfortunately, when I apply a FIR filter with a 0.50Hz lower-edge I
> obtain a lot of NaNs on the areas that previously were 'signal', but not
> everywhere (and, curiously, the processing is really fast), so that it gets
> this kind of aspect:
> >
> >
> >
> <-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
> >
> > NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-signal-NaN-NaN
> >
> >
> > Here's the eeglab output:
> >
> > pop_eegfiltnew() - performing 1651 point highpass filtering.
> > pop_eegfiltnew() - transition band width: 0.5 Hz
> > pop_eegfiltnew() - passband edge(s): 0.5 Hz
> > pop_eegfiltnew() - cutoff frequency(ies) (-6 dB): 0.25 Hz
> > pop_eegfiltnew() - filtering the data (zero-phase)
> > firfilt(): |====================| 100%, ETE 00:00
> > Done.
> >
> >
> > I imagined that the filter function doesn't produce the right output
> because of the NaNs present between the sentences, but as my comprehension
> of the functioning of filters and of eeglab in general is extremely limited
> at the moment, it turns out to be just a speculation.
> >
> > Could you help me understanding what's wrong with my filtering?
> >
> > And, if my hypothesis is correct, i.e. the filter cannot be applied to
> data containing NaNs, how would you apply a high-pass filter to data
> structured as I indicated above (i.e., containing NaNs in some parts)?
> >
> > Thank you!
> > Leo
> >
> > ------------------------
> >
> >
> > Indirizzo istituzionale di posta elettronica degli studenti e dei
> laureati dell'Università degli Studi di Torino
> > Official University of Turin email address for students and graduates
> > _______________________________________________
> > Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> > To unsubscribe, send an empty email to
> eeglablist-unsubscribe at sccn.ucsd.edu
> > For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
>
>

-- 
------------------------

Indirizzo istituzionale di posta elettronica degli studenti e dei laureati 
dell'Università degli Studi di Torino
Official University of Turin email address for students and graduates 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20160513/f75d586b/attachment.html>