[Eeglablist] 0.01 Hz highpass filter isn't running

[Andreas Widmann]

Dear all,

To make sure that no problem has been introduced in recent versions of firfilt, MATLAB, or EEGLAB, I double-checked. Filtering an EEG file with

* 64 channels
* 1000 Hz sampling rate and
* 3.600.000 samples (1h recording time)

filtered with

[EEG, com, b] = pop_eegfiltnew(EEG, 'locutoff', 0.01, 'usefftfilt', 1, 'plotfreqz', 0);

takes 18 seconds on a 5-year-old Ryzen desktop PC (and 10 seconds on an entry-level M2 MacBook).

There is actually a small problem with plotting the frequency response. The default FFT length for plotting is too short. Evaluate manually with a longer FFT instead:
freqz(b, 1, 2^16, EEG.srate)

> Am 07.02.2024 um 22:18 schrieb Lena Marreel via eeglablist <eeglablist at sccn.ucsd.edu>:
> EEG = pop_basicfilter( EEG, 1:64, 'Filter', 'highpass', 'Design', 'butter',
> 'Cutoff', 0.01, 'Order', 4, 'RemoveDC', 'on' );
Took 9 seconds. BUT ERPLAB cheated a little on you. It did not compute an order 4 Buttwerworth filter but filtered the data twice (forward and backward) with an order 2 Butterworth filter giving different results. An order 4 0.01 Hz Butterworth filter would not be stable with double precision. Lack of stability is a persistent problem of more extrem IIR filters.

EEG.srate = 1000;
[b, a] = butter(4, 0.01 / (EEG.srate / 2), 'high');
isstable(b,a)

Best,
Andreas

> Am 08.02.2024 um 00:52 schrieb Cedric Cannard via eeglablist <eeglablist at sccn.ucsd.edu>:
> 
> Hi,
> 
> I'll let Andreas provide a real answer to this. 
> 
> To spare him time, this is described in his paper(https://urldefense.com/v3/__https://ul.qucosa.de/api/qucosa*3A32715/attachment/ATT-0/__;JQ!!Mih3wA!EweSRN-X3Cvkq_vmhzvZy78NtIjVMTFyJJJbWzFtYZ5gFxA_sAAGBpI_Rrj1aEQ0wniFd4UuYTHCIseNTnVYXs-Lrg$ ): "the passband edges are to be specified rather than the cutoff frequencies. If filter order is not specified, a default for filter order is implemented, providing a transition bandwidth of 25% of the lower passband edge but, where possible, not lower than 2 Hz and otherwise the distance from the passband edge to the critical frequency (DC, Nyquist)."
> 
> As he reminded me recently, pop_eegfiltnew() doesn't take the center of the transition bandwidth as cutoff like it is normal done, simply to maintain backward compatibility with old EEGLAB versions. However, rest assured that this is taken care of by pop_eegfiltnew().
> 
> Regarding speed, I guess that it is working in the background but the filter order is so high that it takes forever (depending on your computer capacities). To address this, you could:
> - downsample your data if the sample rate is unnecessarily high
> - use the FFT version of the algorithm (much much faster for high filter orders)
> - with such a low cutoff, you could almost simply apply a baseline removal (subtract the signal mean), which is instantaneous and doesn't introduce potential filter artifacts. 
> 
> You can design your own FIR filter parameters (cutoff, transition bandwidth, order, etc.) in EEGLAB using Andreas Widmann's plugin:
> Tools > Filter data > Windowed sinc FIR filter
> 
> If want to work with command line, after having done this manually in the GUi, you can type 'eegh' in command window to get the commands. 
> 
> And my understanding is that IIR filters are not recommended for the required orders as they will introduce massive, undesired ripples. 
> 
> Hope this helps,
> 
> Cedric Cannard
> 
> 
> 
> On Wednesday, February 7th, 2024 at 1:18 PM, Lena Marreel via eeglablist <eeglablist at sccn.ucsd.edu> wrote:
> 
>> Hi Metin,
>> 
>> I recently went down a similar route on highpass filters.
>> Some practical points I found out, EEGlab FIR filters (the new default)
>> don't offer a filter order high enough to be able to handle those low
>> frequency cut offs. An alternative is to use IIR butterworth filters
>> instead.
>> I tend to just report the filter order, which informs about the role off of
>> your filter in the frequency domain. I recommend ERPlab (
>> https://urldefense.com/v3/__https://erpinfo.org/erplab__;!!Mih3wA!AVaLZlRIT0nEkiq_O4CriQT7S1krYWhGT8mL_WWktZN5mdeaVnC6QAePhj76drsQnSyZZUJh5uZxGVdyYyLvt_ExzQ0$ ) as a useful tool to play around and see how
>> different parameters and filters interact with your data. The ERPlab
>> interface has some useful plots to show you this. ERPlab becomes
>> an extension of your EEGlab GUI and has functions that build on EEGlab
>> functions.
>> I don't know if pop_eegfiltnew() offers an IIR setting, but here is an
>> alternative ERPlab filter I use
>> EEG = pop_basicfilter( EEG, 1:64, 'Filter', 'highpass', 'Design', 'butter',
>> 'Cutoff', 0.01, 'Order', 4, 'RemoveDC', 'on' );
>> 
>> I too noticed the filters running slow. Though, admitingly, we (lab
>> colleagues and I) consider it something we cannot fix ourselves and just
>> accepted it as 'the way things are'. The best cause we could guess at was
>> related to how EEGlab calls in the function over and over taking a lot of
>> memory.
>> 
>> Hope this helps!
>> 
>> Best,
>> Lena
>> 
>> On Wed, Feb 7, 2024 at 8:26 PM Metin Özyağcılar via eeglablist <
>> eeglablist at sccn.ucsd.edu> wrote:
>> 
>>> Dear EEGLAB community,
>>> 
>>> I have recently tried to change the settings for my highpass filter that I
>>> used to pre-process my data. I was using a 0.05Hz highpass filter, now
>>> trying out 0.01 Hz, and I am using the function pop_eegfiltnew(). However,
>>> when I tried to run the function, even after 10-20minutes I see no
>>> progress.
>>> 
>>> After doing some research and reading the manuals online, I’ve seen a
>>> recommendation that for FIR filters a lower limit of about 0.1 is
>>> recommendable and limits lower than that are not. I’ve additionally learned
>>> that the lower and higher limits used in this function do not directly
>>> correspond to the “cut-off” (e.g using 0.05, the “cut-off frequency” is set
>>> to 0.025). All of these led me to have some questions:
>>> 
>>> 1) Is there any way to still run 0.01 highpass filter with pop_eegfiltnew()
>>> and if so how?
>>> 2) If I chose to run a 0.05 Hz highpass filter instead, would it be
>>> appropriate to still set the lower limit as 0.05 but also report the
>>> outcome of the function when writing the paper (i.e. transition band width:
>>> 0.05 Hz / passband edge(s): 0.05 Hz / cutoff frequency(ies) (-6 dB): 0.025
>>> Hz / filtering the data; zero-phase, non-causal)? I feel not so confident
>>> playing around with the parameters and the default settings of the
>>> function.
>>> 3) On an unrelated note: the filters are run very slow, even the 0.05Hz.
>>> Does anyone else maybe have the same issue and if so how are you dealing
>>> with it? I updated MATLAB, EEGLAB, even my operating system to the latest
>>> version + freed up my storge but nothing has changed.
>>> 
>>> Any feedback would be highly appreciated. Thank you!
>>> 
>>> Best,
>>> Metin
>>> _______________________________________________
>>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>>> To unsubscribe, send an empty email to
>>> eeglablist-unsubscribe at sccn.ucsd.edu
>>> For digest mode, send an email with the subject "set digest mime" to
>>> eeglablist-request at sccn.ucsd.edu
>> 
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
>> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu