[Eeglablist] Fwd: Fwd: How to high-pass filter eeg data containing NaNs?

Wed May 18 02:10:40 PDT 2016

I think you should be very careful handling NaNs:

Substitution by zero gives that time point a value that e.g. will add
frequency content to the signal. Would you think about substituting any
other value with zero?

You may remove the NaNs, but then you end up with a varying sampling
interval and have to use math feasible for that (and the index as a time
measure).

Pål

Pål G. Larsson M.D., PhD.

Head of Clinical Neurophysiology

Department of Neurosurgery

Division of Surgery and Clinical Neuroscience

Oslo University Hospital

Po.box 4950 Nydalen

0424 Oslo

Norway

Tel:  (+47) 23074407

Mobile: (+47) 93429791

E-mail: pall at ous-hf.no

not sensitive

*Fra:* eeglablist-bounces at sccn.ucsd.edu [mailto:
eeglablist-bounces at sccn.ucsd.edu] *På vegne av* leo budinich
*Sendt:* 13. mai 2016 14:33
*Til:* eeglablist at sccn.ucsd.edu
*Emne:* [Eeglablist] Fwd: How to high-pass filter eeg data containing NaNs?

---------- Forwarded message ----------
From: *leo budinich* <leo.budinich at edu.unito.it>
Date: 2016-05-13 10:56 GMT+02:00
Subject: Re: [Eeglablist] How to high-pass filter eeg data containing NaNs?
To: Andreas Widmann <widmann at uni-leipzig.de>

Thanks Andreas! Your answer has been really helpful!
Now I understand a little better how filtering works...

I'll make a try with cutting away the NaN parts and then applying the
high-pass filter, keeping in mind that it's highly possible to incur in
edge-effects.
I think also that I am going make a try with a more traditional approach to
baseline correction (subtracting the mean amplitude of the 100ms preceding
the target stimulus).

I would like also to try what Carolina proposed (transforming NaNs in zeros
and then filtering the set): do you think it could work?

Thanks,

Leo

2016-05-12 16:25 GMT+02:00 Andreas Widmann <widmann at uni-leipzig.de>:

Hi Leo,

filtering is the convolution of two signals-the data and the filter’s
impulse response. The impulse response has a duration, 1651 samples or 6.6
seconds in your case. If any sample of both signals is NaN the output is
also NaN. That’s why your filter output mainly consists of NaNs.
Unfortunately, there’s is not so much one could do. Thus, I would consider
the decision NaNing large parts of the continuous data as, hm, suboptimal.

Pragmatically, the only thing you could do is cutting away the NaN parts
(the data are effectively kind of „epoched“ already anyway).

tmp = any( isnan( EEG.data ), 1 );
onsetArray = find( diff( [ 0 tmp ] ) == 1 );
offsetArray = find( diff( [ tmp 0 ] ) == -1 );
EEG = eeg_eegrej( EEG, [ onsetArray; offsetArray ]' );
[ ALLEEG, EEG, CURRENTSET ] = pop_newset( ALLEEG, EEG, CURRENTSET );
eeglab redraw

Now, at least there is signal in the filter output where there was signal
in the filter input.

I would, however, recommend being careful with the further analysis of the
data and interpretation of the results. High-pass filtering should always
be done on the continuous data. You have to expect edge-effects up to the
duration of the impulse response at the beginning and end of each epoch
(actually all data regions which were signal in the filter input and NaN in
the filter output before; essentially the observed NaNs reflect some kind
of edge-effect themselves). Type, size, and duration of the edge-effects
depend on impulse response length and whether and how data are padded
during filtering but there is nothing you could do against edge-effects in
general. Given the minimal required durations of the impulse responses for
the filter cutoffs suggested in the paper I would consider the final
sentence of footnote 3 as incorrect. This does also not depend on filter
type (unfortunately not described in detail in the paper).

Finally, you have to be careful with high-pass filtering with higher
cutoffs in ERP analysis in general. Recent relevant literature on this
issue you may find at the end of this page:
http://sccn.ucsd.edu/wiki/Firfilt_FAQ

Hope this helps!
Andreas

> Am 11.05.2016 um 15:38 schrieb leo budinich <leo.budinich at edu.unito.it>:
>
> Hi Eeglablist,
>
> I'm writing here because I'm a new user of EEGLAB and I am facing a
problem with high-pass filtering some data.
>
> I am trying to apply on a freely downloadable dataset some of the same
processing steps that have been used by the researchers that provided the
data (see http://www.stefanfrank.info/pubs/BL2015.pdf).
> The set I want to filter has 32 channels and has already been been
band-pass filtered at 0.05Hz - 25Hz, recalibrated and re-referenced to the
mastoids.
>
> An important detail is that the EEG recordings concern the reading of two
hundred english sentences; the data recorded between the presentations of
two sentences have been "set to NaN" by the researchers who provided the
dataset, so the channel data of, e.g. subject01, presents this aspect when
you inspect it:
>
>
<-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
>
>
signal-signal-signal-signal-signal-NaN-NaN-NaN-NaN-signal-signal-signal-signal
>
>
> What I would like to do is high-pass filter the entire set to 0.50Hz, as
a way to "mitigate the baseline problem by reducing the correlation between
the baselines and amplitudes by applying an additional high-pass filter
with a sufficiently high cut-off frequency" (see the reference, pp.4).
>
> Unfortunately, when I apply a FIR filter with a 0.50Hz lower-edge I
obtain a lot of NaNs on the areas that previously were 'signal', but not
everywhere (and, curiously, the processing is really fast), so that it gets
this kind of aspect:
>
>
>
<-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
>
> NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-signal-NaN-NaN
>
>
> Here's the eeglab output:
>
> pop_eegfiltnew() - performing 1651 point highpass filtering.
> pop_eegfiltnew() - transition band width: 0.5 Hz
> pop_eegfiltnew() - passband edge(s): 0.5 Hz
> pop_eegfiltnew() - cutoff frequency(ies) (-6 dB): 0.25 Hz
> pop_eegfiltnew() - filtering the data (zero-phase)
> firfilt(): |====================| 100%, ETE 00:00
> Done.
>
>
> I imagined that the filter function doesn't produce the right output
because of the NaNs present between the sentences, but as my comprehension
of the functioning of filters and of eeglab in general is extremely limited
at the moment, it turns out to be just a speculation.
>
> Could you help me understanding what's wrong with my filtering?
>
> And, if my hypothesis is correct, i.e. the filter cannot be applied to
data containing NaNs, how would you apply a high-pass filter to data
structured as I indicated above (i.e., containing NaNs in some parts)?
>
> Thank you!
> Leo
>

> ------------------------
>
>
> Indirizzo istituzionale di posta elettronica degli studenti e dei
laureati dell'Università degli Studi di Torino
> Official University of Turin email address for students and graduates
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to
eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to
eeglablist-request at sccn.ucsd.edu

------------------------

[image: Bilde er fjernet av sender.]

Indirizzo istituzionale di posta elettronica degli studenti e dei laureati
dell'Università degli Studi di Torino

Official University of Turin email address for students and graduates

-- 
------------------------

Indirizzo istituzionale di posta elettronica degli studenti e dei laureati 
dell'Università degli Studi di Torino
Official University of Turin email address for students and graduates 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20160518/a1dc4200/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ~WRD000.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20160518/a1dc4200/attachment.jpg>