[Eeglablist] How to high-pass filter eeg data containing NaNs?

[Andreas Widmann]

Hi Leo,

filtering is the convolution of two signals-the data and the filter’s impulse response. The impulse response has a duration, 1651 samples or 6.6 seconds in your case. If any sample of both signals is NaN the output is also NaN. That’s why your filter output mainly consists of NaNs. Unfortunately, there’s is not so much one could do. Thus, I would consider the decision NaNing large parts of the continuous data as, hm, suboptimal.

Pragmatically, the only thing you could do is cutting away the NaN parts (the data are effectively kind of „epoched“ already anyway).

tmp = any( isnan( EEG.data ), 1 );
onsetArray = find( diff( [ 0 tmp ] ) == 1 );
offsetArray = find( diff( [ tmp 0 ] ) == -1 );
EEG = eeg_eegrej( EEG, [ onsetArray; offsetArray ]' );
[ ALLEEG, EEG, CURRENTSET ] = pop_newset( ALLEEG, EEG, CURRENTSET );
eeglab redraw

Now, at least there is signal in the filter output where there was signal in the filter input.

I would, however, recommend being careful with the further analysis of the data and interpretation of the results. High-pass filtering should always be done on the continuous data. You have to expect edge-effects up to the duration of the impulse response at the beginning and end of each epoch (actually all data regions which were signal in the filter input and NaN in the filter output before; essentially the observed NaNs reflect some kind of edge-effect themselves). Type, size, and duration of the edge-effects depend on impulse response length and whether and how data are padded during filtering but there is nothing you could do against edge-effects in general. Given the minimal required durations of the impulse responses for the filter cutoffs suggested in the paper I would consider the final sentence of footnote 3 as incorrect. This does also not depend on filter type (unfortunately not described in detail in the paper).

Finally, you have to be careful with high-pass filtering with higher cutoffs in ERP analysis in general. Recent relevant literature on this issue you may find at the end of this page:
http://sccn.ucsd.edu/wiki/Firfilt_FAQ

Hope this helps!
Andreas

> Am 11.05.2016 um 15:38 schrieb leo budinich <leo.budinich at edu.unito.it>:
> 
> Hi Eeglablist,
> 
> I'm writing here because I'm a new user of EEGLAB and I am facing a problem with high-pass filtering some data.
> 
> I am trying to apply on a freely downloadable dataset some of the same processing steps that have been used by the researchers that provided the data (see http://www.stefanfrank.info/pubs/BL2015.pdf).
> The set I want to filter has 32 channels and has already been been band-pass filtered at 0.05Hz - 25Hz, recalibrated and re-referenced to the mastoids.
> 
> An important detail is that the EEG recordings concern the reading of two hundred english sentences; the data recorded between the presentations of two sentences have been "set to NaN" by the researchers who provided the dataset, so the channel data of, e.g. subject01, presents this aspect when you inspect it:
> 
> <-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
> 
> signal-signal-signal-signal-signal-NaN-NaN-NaN-NaN-signal-signal-signal-signal  
> 
> 
> What I would like to do is high-pass filter the entire set to 0.50Hz, as a way to "mitigate the baseline problem by reducing the correlation between the baselines and amplitudes by applying an additional high-pass filter with a sufficiently high cut-off frequency" (see the reference, pp.4). 
> 
> Unfortunately, when I apply a FIR filter with a 0.50Hz lower-edge I obtain a lot of NaNs on the areas that previously were 'signal', but not everywhere (and, curiously, the processing is really fast), so that it gets this kind of aspect:
> 
> 
> <-------------sentence_1-------------><---------wait----------><---------sentence_2--------->
> 
> NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-NaN-signal-NaN-NaN
> 
> 
> Here's the eeglab output:
> 
> pop_eegfiltnew() - performing 1651 point highpass filtering.
> pop_eegfiltnew() - transition band width: 0.5 Hz
> pop_eegfiltnew() - passband edge(s): 0.5 Hz
> pop_eegfiltnew() - cutoff frequency(ies) (-6 dB): 0.25 Hz
> pop_eegfiltnew() - filtering the data (zero-phase)
> firfilt(): |====================| 100%, ETE 00:00
> Done.
> 
> 
> I imagined that the filter function doesn't produce the right output because of the NaNs present between the sentences, but as my comprehension of the functioning of filters and of eeglab in general is extremely limited at the moment, it turns out to be just a speculation. 
> 
> Could you help me understanding what's wrong with my filtering?
> 
> And, if my hypothesis is correct, i.e. the filter cannot be applied to data containing NaNs, how would you apply a high-pass filter to data structured as I indicated above (i.e., containing NaNs in some parts)?
> 
> Thank you!
> Leo
> 
> ------------------------
> 
> 
> Indirizzo istituzionale di posta elettronica degli studenti e dei laureati dell'Università degli Studi di Torino
> Official University of Turin email address for students and graduates 
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu