[Eeglablist] Filter distortion around boundary events

[Andreas Widmann]

Hi Roy,

> When I then apply a highpass filter [pop_eegfiltnew(EEG,0.3,0)], I often see subtle but clear jumps right at the edge of where my epoch boundaries used to be. Critically, these jumps were there even if no data had been removed in between. In contrast, just filtering the original continuous data works fine.

This is the expected behavior of a high-pass filter (a filter effect rather than a filter artifact). Besides removing the global DC potential a high-pass filter will also compensate local DC shifts (rate depending on cutoff and order). If you have a particular feature (a drift, a large unipolar ERP, etc.) in the data close to the edge of an epoch, the filter will compensate for the shift of the DC potential until the end of the epoch but not beyond even if necessary. When filtering the next epoch, the filter will not compensate for the DC shift introduced by the particular feature at the end of the previous epoch (actually temporally close but now invisible to the filter). In general, I would consider this case as rule rather than as exception. In some cases the discontinuity/step might just be so small that it cannot be identified by eye but it is still there.

I will add some MATLAB code illustrating the problem below. In summary, filtering should be done on the continuous data whenever possible. Data discontinuities should be marked by boundary events when stitching back together data filtered in epochs.

Best,
Andreas

fs = 500;
sig = zeros( fs * 60, 1 ); % 60 sec
sig( 14251:14750, 1 ) = exp( -( -250:249 ) .^ 2 / ( 2 * 62.5 ^ 2 ) ); % test signal: Gaussian close to end of first epoch

plot( ( 0:length( sig ) - 1 ) / fs, sig )
xlim( [ 25 35 ] )
hold all

b = firws( 5500, 0.15 / ( fs / 2 ), 'high', windows('hamming', 5501 ) ); % filter coefficients

fsig_cnt = filter( b, 1, [ zeros( 2750, 1 ); sig; zeros( 2750, 1 ) ] ); % filter continuous
fsig_cnt = fsig_cnt( 5501:end, 1); % remove padding

plot( ( 0:length( fsig_cnt ) - 1 ) / fs, fsig_cnt )

tmp = filter( b, 1, [ zeros( 2750, 1 ); sig( 1:15000, 1 ); zeros( 2750, 1 ) ] ); % filter first epoch
fsig_epo( 1:15000, 1 ) = tmp( 5501:end, 1); % remove padding
tmp = filter( b, 1, [ zeros( 2750, 1 ); sig( 15001:30000, 1 ); zeros( 2750, 1 ) ] ); % filter second epoch
fsig_epo( 15001:30000, 1 ) = tmp( 5501:end, 1); % remove padding

plot( ( 0:length( fsig_epo ) - 1 ) / fs, fsig_epo )
legend( 'raw', 'filtered continuous', 'filtered epoched and stitched' )

> Am 17.07.2018 um 18:20 schrieb Roy Cox <roycox.roycox at gmail.com>:
> 
> hi all,
> 
> Just wanted to chime in that I've noticed this (or something similar), too.
> 
> I typically epoch continuous sleep data into 30 s "trials" to exclude bad epochs, before stitching everything back together using eeg_epoch2continuous.
> 
> When I then apply a highpass filter [pop_eegfiltnew(EEG,0.3,0)], I often see subtle but clear jumps right at the edge of where my epoch boundaries used to be. Critically, these jumps were there even if no data had been removed in between. In contrast, just filtering the original continuous data works fine.
> 
> I checked to see if the continuous->epoched and epoched->continuous transformations somehow resulted in skipping a sample, or some other change of amplitude, but amplitude values were identical across boundaries in all cases. It's as if the pop_eegfiltnew function somehow notices when continuous data previously existed in an epoched form, and alters its behavior based on that...
> 
> My ugly workaround has been to do all my filtering on continuous data (even if it includes lots that I know will be rejected anyway), but would be good if this issue were remedied.
> 
> Roy