[Eeglablist] Filter distortion around boundary events

Tue Jul 17 14:55:12 PDT 2018

Dear Brian, Roy, and Andreas,

I saw Brian's data. Two possibilities.

1. The data before and after the boundary are not filtered due to the fact
that filter respects the boundary events in the way that it does not go
over it but stops and re-starts before and after the boundary.

2. This is more likely the case--I see a high-amplitude artifact is present
before and after the boundary; in this case, this is due to filter ringing
which became noticeable by high-amplitude noise.

> My ugly workaround has been to do all my filtering on continuous data
(even if it includes lots that I know will be rejected anyway), but would
be good if this issue were remedied.

Roy, I think that's the right solution. A big ringing arising from
filtering high-amplitude artifact is inevitable. This basically means there
is no magic solution to 'erase' high-amplitude short-bursts like TMS
artifacts.

This why I wrote ARfitStudio plugin to process such artifacts.
https://sccn.ucsd.edu/wiki/InterpolateSpike
This replaces these datapoints with AR-learned interpolations. Replace
these high-amplitude short-burst artifacts first, then filter the data.
This does not creates boundaries.

Makoto

On Tue, Jul 17, 2018 at 9:21 AM Roy Cox <roycox.roycox at gmail.com> wrote:

> hi all,
>
> Just wanted to chime in that I've noticed this (or something similar), too.
>
> I typically epoch continuous sleep data into 30 s "trials" to exclude bad
> epochs, before stitching everything back together
> using eeg_epoch2continuous.
>
> When I then apply a highpass filter [pop_eegfiltnew(EEG,0.3,0)], I often
> see subtle but clear jumps right at the edge of where my epoch boundaries
> used to be. Critically, these jumps were there even if no data had been
> removed in between. In contrast, just filtering the original continuous
> data works fine.
>
> I checked to see if the continuous->epoched and epoched->continuous
> transformations somehow resulted in skipping a sample, or some other change
> of amplitude, but amplitude values were identical across boundaries in all
> cases. It's as if the pop_eegfiltnew function somehow notices when
> continuous data previously existed in an epoched form, and alters its
> behavior based on that...
>
> My ugly workaround has been to do all my filtering on continuous data
> (even if it includes lots that I know will be rejected anyway), but would
> be good if this issue were remedied.
>
> Roy
>
>
>
> On Tue, Jul 17, 2018 at 6:54 AM Erickson <ericksonb.eng at gmail.com> wrote:
>
>> Thanks for your responses.
>>
>> Andreas, I used the "basic FIR filter" on automatic order calculation
>> filtering from 1 to 55.
>>
>> Here is a drive link to a zip file with 30s of example data. These data
>> were imported using fileIO and the event channel (65) was imported as
>> events (Nothing else has been done to this example data).
>>
>>> https://drive.google.com/open?id=1KPlpeBCrCnQJHCJOcMYsnNSJVOjj0bTO
>>
>>
>> Then I plotted the data and cut out one section around a TMS event, and
>> another section NOT around a TMS event;
>>
>>> EEG = eeg_eegrej( EEG, [4615 5372;11482 12002]);
>>> [ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG, 1,'setname','with
>>> Rejections','gui','off');
>>
>>
>> filtered the data using the FIR filter from 1 to 55hz;
>>
>>> EEG = pop_eegfiltnew(EEG, 1,55,6600,0,[],1);
>>> [ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG, 2,'setname','with
>>> Rejections and Filter','gui','off');
>>>
>>
>> then plotting the data, the artifact is visible as a "pinching" around
>> each boundary, with a faster oscillation riding on top. So basically just
>> reject any section of data and filter and the artifact appears.
>>
>> Makoto, to answer the question about the spectra - if I cut out the TMS
>> pulses and then filter, the PSD looks pretty normal. Prior to filtering
>> there is so much DC power I can't visually inspect the data very well.
>>
>> The room is very noisy, but this artifact appears only wherever I make a
>> boundary and filter so it seems like an edge effect due to filtering - but
>> I've never encountered this kind of artifact before since EEGLAB knows not
>> to filter over boundaries. Perhaps I cannot use the automatic filter order
>> here due to some noise profile in my data?
>>
>> Thanks to you both for considering the problem!
>>
>> Brian
>>
>>
>> On Thu, Jul 12, 2018 at 8:54 PM, Makoto Miyakoshi <mmiyakoshi at ucsd.edu>
>> wrote:
>>
>>> Dear Erickson
>>>
>>> Is this really a distortion introduced by the filter? Rather, isn't it
>>> the case that that portion of the data was poorly filtered i.e., your
>>> original data, before filtering, had that noise constantly? Could you
>>> please check your original data by eyeballing the raw time series and by
>>> checking power spectral density?
>>>
>>> Makoto
>>>
>>> On Thu, Jul 12, 2018 at 10:33 AM Erickson <ericksonb.eng at gmail.com>
>>> wrote:
>>>
>>>> Hello EEGLAB list,
>>>>
>>>> We are experiencing a strange artifact on only some of our EEG data.
>>>> For some subjects, when we cut out portions of data (creating boundary
>>>> events) and then filter (from 1 to 40 or 50hz) we observe a distortion
>>>> around the boundary.
>>>>
>>>> Specifically, we see a rising or falling wave on either side of the
>>>> boundary, with a fast oscillation riding on top of it. See here:
>>>> https://drive.google.com/open?id=1yt6h_WLgI6jWAApVd52nwVLuiduYgXIH
>>>>
>>>> Although this data is from concurrent TMS-EEG recording, the distortion
>>>> has nothing to do with the TMS pulse itself - we get the same
>>>> boundary-related distortion when we filter after cutting out a random
>>>> section of continuous data (a section with no TMS in it).
>>>>
>>>> I am at a loss - we've tried a few different filters and nothing seems
>>>> to solve this issue. Meanwhile, on some subjects there is no
>>>> boundary-related filter distortion at all! There is a lot of noise in
>>>> the room but I'm not sure how that could create this specific issue.
>>>>
>>>> Any perspectives on this issue are appreciated! Thank you,
>>>>
>>>> Brian
>>>>
>>>> *Brian Erickson**, Ph.D.*
>>>> *Postdoctoral Researcher, CogNeW Lab*
>>>>
>>>> *Drexel University*
>>>> 3141 Chestnut Street
>>>> <https://maps.google.com/?q=3141+Chestnut+Street&entry=gmail&source=g>
>>>> Stratton Hall Room 320
>>>> Philadelphia, PA 19104
>>>>
>>>> _______________________________________________
>>>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>>>> To unsubscribe, send an empty email to
>>>> eeglablist-unsubscribe at sccn.ucsd.edu
>>>> For digest mode, send an email with the subject "set digest mime" to
>>>> eeglablist-request at sccn.ucsd.edu
>>>
>>>
>>>
>>> --
>>> Makoto Miyakoshi
>>> Swartz Center for Computational Neuroscience
>>> Institute for Neural Computation, University of California San Diego
>>>
>>
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to
>> eeglablist-unsubscribe at sccn.ucsd.edu
>> For digest mode, send an email with the subject "set digest mime" to
>> eeglablist-request at sccn.ucsd.edu
>
>

-- 
Makoto Miyakoshi
Swartz Center for Computational Neuroscience
Institute for Neural Computation, University of California San Diego
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20180717/40babf56/attachment.html>