[Eeglablist] 0.01 Hz highpass filter isn't running

[Metin Özyağcılar]

Dear all,

thank you all so very much for all the valuable input and introducing me
other ways of applying these filters (e.g. ERPLAB, usefftfilt  ... ) that I
was not aware of.

I have also tested myself (just to test the speed) and they run indeed very
fast, as compared to the pop_eegfiltnew with its default parameters.

To answer @Andreas' question: I read it here (
https://urldefense.com/v3/__https://eeglab.org/others/Firfilt_FAQ*q-what-is-the-difference-between-the-basic-fir-filter-legacy-and-the-basic-fir-filter-new-which-should-i-use__;Iw!!Mih3wA!EYBa2soB7twj10AhKjo3fAqZLdkF-8w_SkkVfD5AfjsuwHXpGUK4QAb68TWNdoywHqmY0dreQqNmQDWBhyPTane8IDENL7c$ ;
under the question "Q. What is the lower limit for cutoff frequency for
high-pass filter?") that "a lower limit of about 0.1 Hz is recommendable
for FIR filters. For lower cutoff frequencies consider IIR filters combined
with a reduced sampling frequency of the signal." But, on that page, the
stability issue of IIR is also mentioned somewhere.

Thanks!

All the best,
Metin

On Thu, Feb 8, 2024 at 1:58 PM Andreas Widmann <widmann at uni-leipzig.de>
wrote:

> Dear all,
>
> To make sure that no problem has been introduced in recent versions of
> firfilt, MATLAB, or EEGLAB, I double-checked. Filtering an EEG file with
>
> * 64 channels
> * 1000 Hz sampling rate and
> * 3.600.000 samples (1h recording time)
>
> filtered with
>
> [EEG, com, b] = pop_eegfiltnew(EEG, 'locutoff', 0.01, 'usefftfilt', 1,
> 'plotfreqz', 0);
>
> takes 18 seconds on a 5-year-old Ryzen desktop PC (and 10 seconds on an
> entry-level M2 MacBook).
>
> There is actually a small problem with plotting the frequency response.
> The default FFT length for plotting is too short. Evaluate manually with a
> longer FFT instead:
> freqz(b, 1, 2^16, EEG.srate)
>
> > Am 07.02.2024 um 22:18 schrieb Lena Marreel via eeglablist <
> eeglablist at sccn.ucsd.edu>:
> > EEG = pop_basicfilter( EEG, 1:64, 'Filter', 'highpass', 'Design',
> 'butter',
> > 'Cutoff', 0.01, 'Order', 4, 'RemoveDC', 'on' );
> Took 9 seconds. BUT ERPLAB cheated a little on you. It did not compute an
> order 4 Buttwerworth filter but filtered the data twice (forward and
> backward) with an order 2 Butterworth filter giving different results. An
> order 4 0.01 Hz Butterworth filter would not be stable with double
> precision. Lack of stability is a persistent problem of more extrem IIR
> filters.
>
> EEG.srate = 1000;
> [b, a] = butter(4, 0.01 / (EEG.srate / 2), 'high');
> isstable(b,a)
>
> Best,
> Andreas
>
> > Am 08.02.2024 um 00:52 schrieb Cedric Cannard via eeglablist <
> eeglablist at sccn.ucsd.edu>:
> >
> > Hi,
> >
> > I'll let Andreas provide a real answer to this.
> >
> > To spare him time, this is described in his paper(
> https://urldefense.com/v3/__https://ul.qucosa.de/api/qucosa*3A32715/attachment/ATT-0/__;JQ!!Mih3wA!EweSRN-X3Cvkq_vmhzvZy78NtIjVMTFyJJJbWzFtYZ5gFxA_sAAGBpI_Rrj1aEQ0wniFd4UuYTHCIseNTnVYXs-Lrg$
> ): "the passband edges are to be specified rather than the cutoff
> frequencies. If filter order is not specified, a default for filter order
> is implemented, providing a transition bandwidth of 25% of the lower
> passband edge but, where possible, not lower than 2 Hz and otherwise the
> distance from the passband edge to the critical frequency (DC, Nyquist)."
> >
> > As he reminded me recently, pop_eegfiltnew() doesn't take the center of
> the transition bandwidth as cutoff like it is normal done, simply to
> maintain backward compatibility with old EEGLAB versions. However, rest
> assured that this is taken care of by pop_eegfiltnew().
> >
> > Regarding speed, I guess that it is working in the background but the
> filter order is so high that it takes forever (depending on your computer
> capacities). To address this, you could:
> > - downsample your data if the sample rate is unnecessarily high
> > - use the FFT version of the algorithm (much much faster for high filter
> orders)
> > - with such a low cutoff, you could almost simply apply a baseline
> removal (subtract the signal mean), which is instantaneous and doesn't
> introduce potential filter artifacts.
> >
> > You can design your own FIR filter parameters (cutoff, transition
> bandwidth, order, etc.) in EEGLAB using Andreas Widmann's plugin:
> > Tools > Filter data > Windowed sinc FIR filter
> >
> > If want to work with command line, after having done this manually in
> the GUi, you can type 'eegh' in command window to get the commands.
> >
> > And my understanding is that IIR filters are not recommended for the
> required orders as they will introduce massive, undesired ripples.
> >
> > Hope this helps,
> >
> > Cedric Cannard
> >
> >
> >
> > On Wednesday, February 7th, 2024 at 1:18 PM, Lena Marreel via eeglablist
> <eeglablist at sccn.ucsd.edu> wrote:
> >
> >> Hi Metin,
> >>
> >> I recently went down a similar route on highpass filters.
> >> Some practical points I found out, EEGlab FIR filters (the new default)
> >> don't offer a filter order high enough to be able to handle those low
> >> frequency cut offs. An alternative is to use IIR butterworth filters
> >> instead.
> >> I tend to just report the filter order, which informs about the role
> off of
> >> your filter in the frequency domain. I recommend ERPlab (
> >>
> https://urldefense.com/v3/__https://erpinfo.org/erplab__;!!Mih3wA!AVaLZlRIT0nEkiq_O4CriQT7S1krYWhGT8mL_WWktZN5mdeaVnC6QAePhj76drsQnSyZZUJh5uZxGVdyYyLvt_ExzQ0$
> ) as a useful tool to play around and see how
> >> different parameters and filters interact with your data. The ERPlab
> >> interface has some useful plots to show you this. ERPlab becomes
> >> an extension of your EEGlab GUI and has functions that build on EEGlab
> >> functions.
> >> I don't know if pop_eegfiltnew() offers an IIR setting, but here is an
> >> alternative ERPlab filter I use
> >> EEG = pop_basicfilter( EEG, 1:64, 'Filter', 'highpass', 'Design',
> 'butter',
> >> 'Cutoff', 0.01, 'Order', 4, 'RemoveDC', 'on' );
> >>
> >> I too noticed the filters running slow. Though, admitingly, we (lab
> >> colleagues and I) consider it something we cannot fix ourselves and just
> >> accepted it as 'the way things are'. The best cause we could guess at
> was
> >> related to how EEGlab calls in the function over and over taking a lot
> of
> >> memory.
> >>
> >> Hope this helps!
> >>
> >> Best,
> >> Lena
> >>
> >> On Wed, Feb 7, 2024 at 8:26 PM Metin Özyağcılar via eeglablist <
> >> eeglablist at sccn.ucsd.edu> wrote:
> >>
> >>> Dear EEGLAB community,
> >>>
> >>> I have recently tried to change the settings for my highpass filter
> that I
> >>> used to pre-process my data. I was using a 0.05Hz highpass filter, now
> >>> trying out 0.01 Hz, and I am using the function pop_eegfiltnew().
> However,
> >>> when I tried to run the function, even after 10-20minutes I see no
> >>> progress.
> >>>
> >>> After doing some research and reading the manuals online, I’ve seen a
> >>> recommendation that for FIR filters a lower limit of about 0.1 is
> >>> recommendable and limits lower than that are not. I’ve additionally
> learned
> >>> that the lower and higher limits used in this function do not directly
> >>> correspond to the “cut-off” (e.g using 0.05, the “cut-off frequency”
> is set
> >>> to 0.025). All of these led me to have some questions:
> >>>
> >>> 1) Is there any way to still run 0.01 highpass filter with
> pop_eegfiltnew()
> >>> and if so how?
> >>> 2) If I chose to run a 0.05 Hz highpass filter instead, would it be
> >>> appropriate to still set the lower limit as 0.05 but also report the
> >>> outcome of the function when writing the paper (i.e. transition band
> width:
> >>> 0.05 Hz / passband edge(s): 0.05 Hz / cutoff frequency(ies) (-6 dB):
> 0.025
> >>> Hz / filtering the data; zero-phase, non-causal)? I feel not so
> confident
> >>> playing around with the parameters and the default settings of the
> >>> function.
> >>> 3) On an unrelated note: the filters are run very slow, even the
> 0.05Hz.
> >>> Does anyone else maybe have the same issue and if so how are you
> dealing
> >>> with it? I updated MATLAB, EEGLAB, even my operating system to the
> latest
> >>> version + freed up my storge but nothing has changed.
> >>>
> >>> Any feedback would be highly appreciated. Thank you!
> >>>
> >>> Best,
> >>> Metin
> >>> _______________________________________________
> >>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> >>> To unsubscribe, send an empty email to
> >>> eeglablist-unsubscribe at sccn.ucsd.edu
> >>> For digest mode, send an email with the subject "set digest mime" to
> >>> eeglablist-request at sccn.ucsd.edu
> >>
> >> _______________________________________________
> >> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> >> To unsubscribe, send an empty email to
> eeglablist-unsubscribe at sccn.ucsd.edu
> >> For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
> > _______________________________________________
> > Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> > To unsubscribe, send an empty email to
> eeglablist-unsubscribe at sccn.ucsd.edu
> > For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
>
>
>