[Eeglablist] Fwd: How to high-pass filter eeg data containing NaNs?

Wed May 18 02:31:22 PDT 2016

Hi Pål,

thanks for your advice; mmmh, from what you and Andreas wrote it seems that
substituting NaN with zeros would produce a lot of problems; I liked the
method because of its simplicity, but clearly if it produces artefacts I
will not apply it.

Ciao,
Leo

2016-05-18 9:13 GMT+02:00 Pål Gunnar Larsson <pall at ous-hf.no>:

> I think you should be very careful handling NaNs:
>
>
>
> Substitution by zero gives that time point a value that e.g. will add
> frequency content to the signal. Would you think about substituting any
> other value with zero?
>
>
>
> You may remove the NaNs, but then you end up with a varying sampling
> interval and have to use math feasible for that (and the index as a time
> measure).
>
>
>
> Pål
>
>
>
> Pål G. Larsson M.D., PhD.
>
> Head of Clinical Neurophysiology
>
> Department of Neurosurgery
>
> Division of Surgery and Clinical Neuroscience
>
>
>
> Oslo University Hospital
>
> Po.box 4950 Nydalen
>
> 0424 Oslo
>
> Norway
>
> Tel:  (+47) 23074407
>
> Mobile: (+47) 93429791
>
> E-mail: pall at ous-hf.no
>
> not sensitive
>
>
>
> *Fra:* eeglablist-bounces at sccn.ucsd.edu [mailto:
> eeglablist-bounces at sccn.ucsd.edu] *På vegne av* leo budinich
> *Sendt:* 13. mai 2016 14:33
> *Til:* eeglablist at sccn.ucsd.edu
> *Emne:* [Eeglablist] Fwd: How to high-pass filter eeg data containing
> NaNs?
>
>
>
>
>
> ---------- Forwarded message ----------
> From: *leo budinich* <leo.budinich at edu.unito.it>
> Date: 2016-05-13 10:56 GMT+02:00
> Subject: Re: [Eeglablist] How to high-pass filter eeg data containing NaNs?
> To: Andreas Widmann <widmann at uni-leipzig.de>
>
> Thanks Andreas! Your answer has been really helpful!
> Now I understand a little better how filtering works...
>
> I'll make a try with cutting away the NaN parts and then applying the
> high-pass filter, keeping in mind that it's highly possible to incur in
> edge-effects.
> I think also that I am going make a try with a more traditional approach
> to baseline correction (subtracting the mean amplitude of the 100ms
> preceding the target stimulus).
>
>
> I would like also to try what Carolina proposed (transforming NaNs in
> zeros and then filtering the set): do you think it could work?
>
> Thanks,
>
> Leo
>
>
>
> 2016-05-12 16:25 GMT+02:00 Andreas Widmann <widmann at uni-leipzig.de>:
>
> Hi Leo,
>
> filtering is the convolution of two signals-the data and the filter’s
> impulse response. The impulse response has a duration, 1651 samples or 6.6
> seconds in your case. If any sample of both signals is NaN the output is
> also NaN. That’s why your filter output mainly consists of NaNs.
> Unfortunately, there’s is not so much one could do. Thus, I would consider
> the decision NaNing large parts of the continuous data as, hm, suboptimal.
>
> Pragmatically, the only thing you could do is cutting away the NaN parts
> (the data are effectively kind of „epoched“ already anyway).
>
> tmp = any( isnan( EEG.data ), 1 );
> onsetArray = find( diff( [ 0 tmp ] ) == 1 );
> offsetArray = find( diff( [ tmp 0 ] ) == -1 );
> EEG = eeg_eegrej( EEG, [ onsetArray; offsetArray ]' );
> [ ALLEEG, EEG, CURRENTSET ] = pop_newset( ALLEEG, EEG, CURRENTSET );
> eeglab redraw
>
> Now, at least there is signal in the filter output where there was signal
> in the filter input.
>
> I would, however, recommend being careful with the further analysis of the
> data and interpretation of the results. High-pass filtering should always
> be done on the continuous data. You have to expect edge-effects up to the
> duration of the impulse response at the beginning and end of each epoch
> (actually all data regions which were signal in the filter input and NaN in
> the filter output before; essentially the observed NaNs reflect some kind
> of edge-effect themselves). Type, size, and duration of the edge-effects
> depend on impulse response length and whether and how data are padded
> during filtering but there is nothing you could do against edge-effects in
> general. Given the minimal required durations of the impulse responses for
> the filter cutoffs suggested in the paper I would consider the final
> sentence of footnote 3 as incorrect. This does also not depend on filter
> type (unfortunately not described in detail in the paper).
>
> Finally, you have to be careful with high-pass filtering with higher
> cutoffs in ERP analysis in general. Recent relevant literature on this
> issue you may find at the end of this page:
> http://sccn.ucsd.edu/wiki/Firfilt_FAQ
>
> Hope this helps!
> Andreas
>
>
> > Am 11.05.2016 um 15:38 schrieb leo budinich <leo.budinich at edu.unito.it>:
> >
> > Hi Eeglablist,
> >
> > I'm writing here because I'm a new user of EEGLAB and I am facing a
> problem with high-pass filtering some data.
> >
> > I am trying to apply on a freely downloadable dataset some of the same
> processing steps that have been used by the researchers that provided the
> data (see http://www.stefanfrank.info/pubs/BL2015.pdf).
> > The set I want to filter has 32 channels and has already been been
> band-pass filtered at 0.05Hz - 25Hz, recalibrated and re-referenced to the
> mastoids.
> >
> > An important detail is that the EEG recordings concern the reading of
> two hundred english sentences; the data recorded between the presentations
> of two sentences have been "set to NaN" by the researchers who provided the
> dataset, so the channel data of, e.g. subject01, presents this aspect when
> you inspect it:
> >
> >
> <-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
> >
> >
> signal-signal-signal-signal-signal-NaN-NaN-NaN-NaN-signal-signal-signal-signal
> >
> >
> > What I would like to do is high-pass filter the entire set to 0.50Hz, as
> a way to "mitigate the baseline problem by reducing the correlation between
> the baselines and amplitudes by applying an additional high-pass filter
> with a sufficiently high cut-off frequency" (see the reference, pp.4).
> >
> > Unfortunately, when I apply a FIR filter with a 0.50Hz lower-edge I
> obtain a lot of NaNs on the areas that previously were 'signal', but not
> everywhere (and, curiously, the processing is really fast), so that it gets
> this kind of aspect:
> >
> >
> >
> <-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
> >
> > NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-signal-NaN-NaN
> >
> >
> > Here's the eeglab output:
> >
> > pop_eegfiltnew() - performing 1651 point highpass filtering.
> > pop_eegfiltnew() - transition band width: 0.5 Hz
> > pop_eegfiltnew() - passband edge(s): 0.5 Hz
> > pop_eegfiltnew() - cutoff frequency(ies) (-6 dB): 0.25 Hz
> > pop_eegfiltnew() - filtering the data (zero-phase)
> > firfilt(): |====================| 100%, ETE 00:00
> > Done.
> >
> >
> > I imagined that the filter function doesn't produce the right output
> because of the NaNs present between the sentences, but as my comprehension
> of the functioning of filters and of eeglab in general is extremely limited
> at the moment, it turns out to be just a speculation.
> >
> > Could you help me understanding what's wrong with my filtering?
> >
> > And, if my hypothesis is correct, i.e. the filter cannot be applied to
> data containing NaNs, how would you apply a high-pass filter to data
> structured as I indicated above (i.e., containing NaNs in some parts)?
> >
> > Thank you!
> > Leo
> >
>
> > ------------------------
> >
> >
> > Indirizzo istituzionale di posta elettronica degli studenti e dei
> laureati dell'Università degli Studi di Torino
> > Official University of Turin email address for students and graduates
> > _______________________________________________
> > Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> > To unsubscribe, send an empty email to
> eeglablist-unsubscribe at sccn.ucsd.edu
> > For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
>
>
>
>
>
>
>
> ------------------------
>
> [image: Bilde er fjernet av sender.]
>
>
>
> Indirizzo istituzionale di posta elettronica degli studenti e dei laureati
> dell'Università degli Studi di Torino
>
> Official University of Turin email address for students and graduates
>

-- 
------------------------

Indirizzo istituzionale di posta elettronica degli studenti e dei laureati 
dell'Università degli Studi di Torino
Official University of Turin email address for students and graduates 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20160518/25faa22b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ~WRD000.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20160518/25faa22b/attachment.jpg>