[Eeglablist] ASR/clean_rawdata producing very inconsistent results

Kelly Michaelis kcmichaelis at gmail.com
Sat Sep 22 11:16:30 PDT 2018


Hi Tarik,

No, I didn’t. I tested this several times, with different datasets and different choices computers. On cleaner data set, there was still some slight variability (after epoching, resulted in a different of 1-2 epochs at most), but on noisier datasets I continued to get huge differences (>100 epoch difference) when running the same file through with the same settings. 

I am frustrated because I was hoping to use ASR on my data, and now I don’t think I can. As I mentioned in my last post, it seems that on the nosier datasets it just doesn’t work correctly about half the time. 

If anyone has any insight please let me know!

Kelly

Sent from my iPhone

> On Sep 22, 2018, at 1:37 PM, Tarik S Bel-Bahar <tarikbelbahar at gmail.com> wrote:
> 
> Hello Kelly, curious if you got some resolution from eeglablist about ASR's dynamics ?
> 
>> On Wed, Sep 12, 2018 at 11:48 AM Kelly Michaelis <kcmichaelis at gmail.com> wrote:
>> Hi Everyone,
>> 
>> I am using ASR to clean my datasets prior to ICA. I've been trying to figure out what level to set each of the ASR parameters to, and in doing so I noticed that the same input is producing massively different results each time I run it. 
>> 
>> More specifically, here's what I did:
>> 
>> 1. filter, resample
>> 2. Manually remove bad channels (I noticed ASR wasnt' catching some)
>> 3. ASR using the following parameters: 
>> 
>>  EEG = clean_rawdata(EEG, -1, -1, -1, -1, 24, .25);
>> 
>> 4. Reference to average
>> 5. Epoch
>> 
>> After I noticed some inconsistencies, I decided to run this exact series of steps several times on the same dataset (i.e. starting with the raw continuous dataset each time). On the first run, ASR inserted so many boundary events that I could't get any full epochs out. On the second run, I got 161 epochs (out of 200). On the third run, I it was back to zero full epochs. On the fourth run, I got 162 epochs.
>> 
>> Does anyone else have this issue?? I want to apply the same settings to multiple subject files, but if the results are this inconsistent that's a huge issue. I'll put the command window output below. It looks like on the runs where it inserts lots of boundary events it finishes running much faster and is maybe not doing the cleaning?
>> 
>> Please help! I want to be able to use ASR.
>> 
>> Thanks,
>> Kelly
>> 
>> From the run that produced no viable epochs: 
>> 
>> >> EEG = clean_rawdata(EEG, -1, -1, -1, -1, 24, .25);
>> 
>> flatchan rej disabled.
>> highpass disabled.
>> badchan rej disabled.
>> noise-based rej disabled.
>> Finding a clean section of the data...
>> Determining time window rejection thresholds...done.
>> Keeping 21.1% (475 seconds) of the data.
>> eeg_insertbound(): 477 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 1682 events removed.
>> eeg_checkset note: upper time limit (xmax) adjusted so (xmax-xmin)*srate+1 = number of frames
>> eeg_checkset note: value format of event field 'stimname' made uniform
>> eeg_checkset note: value format of event field 'answer' made uniform
>> eeg_insertbound(): 477 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 1686 events removed.
>> BUG 1971 WARNING: IF YOU ARE USING A SCRIPT WITTEN FOR A PREVIOUS VERSION OF
>> EEGLAB TO CALL THIS FUNCTION, BECAUSE YOU ARE REJECTING THE ONSET OF THE DATA,
>> EVENTS WERE CORRUPTED. EVENT LATENCIES ARE NOW CORRECT (SEE https://sccn.ucsd.edu/wiki/EEGLAB_bug1971);
>> Estimating calibration statistics; this may take a while...
>> Warning: A value of class "com.sun.management.UnixOperatingSystem" was indexed with no subscripts specified. Currently the result of
>> this operation is the indexed value itself, but in a future release, it will be an error. 
>> > In asr_calibrate>hlp_memfree (line 266)
>>   In asr_calibrate (line 114)
>>   In clean_asr (line 164)
>>   In clean_artifacts (line 219)
>>   In clean_rawdata (line 83) 
>> Determining per-component thresholds...done.
>> Warning: A value of class "com.sun.management.UnixOperatingSystem" was indexed with no subscripts specified. Currently the result of
>> this operation is the indexed value itself, but in a future release, it will be an error. 
>> > In asr_process>hlp_memfree (line 232)
>>   In asr_process (line 105)
>>   In clean_asr (line 174)
>>   In clean_artifacts (line 219)
>>   In clean_rawdata (line 83) 
>> Now doing final post-cleanup of the output.
>> Determining time window rejection thresholds...done.
>> Keeping 28.5% (642 seconds) of the data.
>> eeg_insertbound(): 587 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 1595 events removed.
>> eeg_checkset note: upper time limit (xmax) adjusted so (xmax-xmin)*srate+1 = number of frames
>> eeg_checkset note: value format of event field 'stimname' made uniform
>> eeg_checkset note: value format of event field 'answer' made uniform
>> eeg_insertbound(): 587 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 1599 events removed.
>> BUG 1971 WARNING: IF YOU ARE USING A SCRIPT WITTEN FOR A PREVIOUS VERSION OF
>> EEGLAB TO CALL THIS FUNCTION, BECAUSE YOU ARE REJECTING THE ONSET OF THE DATA,
>> EVENTS WERE CORRUPTED. EVENT LATENCIES ARE NOW CORRECT (SEE https://sccn.ucsd.edu/wiki/EEGLAB_bug1971);
>> Use vis_artifacts to compare the cleaned data to the original.
>> Re-referencing data
>> Removing 1 channel(s)...
>> Saving dataset...
>> 
>> 
>> From the run that produced 162 epochs:
>> 
>> EEG = clean_rawdata(EEG, -1, -1, -1, -1, 24, .25);
>> 
>> flatchan rej disabled.
>> highpass disabled.
>> badchan rej disabled.
>> noise-based rej disabled.
>> Finding a clean section of the data...
>> Determining time window rejection thresholds...done.
>> Keeping 21.1% (475 seconds) of the data.
>> eeg_insertbound(): 477 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 1682 events removed.
>> eeg_checkset note: upper time limit (xmax) adjusted so (xmax-xmin)*srate+1 = number of frames
>> eeg_checkset note: value format of event field 'stimname' made uniform
>> eeg_checkset note: value format of event field 'answer' made uniform
>> eeg_insertbound(): 477 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 1686 events removed.
>> BUG 1971 WARNING: IF YOU ARE USING A SCRIPT WITTEN FOR A PREVIOUS VERSION OF
>> EEGLAB TO CALL THIS FUNCTION, BECAUSE YOU ARE REJECTING THE ONSET OF THE DATA,
>> EVENTS WERE CORRUPTED. EVENT LATENCIES ARE NOW CORRECT (SEE https://sccn.ucsd.edu/wiki/EEGLAB_bug1971);
>> Estimating calibration statistics; this may take a while...
>> Warning: A value of class "com.sun.management.UnixOperatingSystem" was indexed with no subscripts specified. Currently the result of
>> this operation is the indexed value itself, but in a future release, it will be an error. 
>> > In asr_calibrate>hlp_memfree (line 266)
>>   In asr_calibrate (line 114)
>>   In clean_asr (line 164)
>>   In clean_artifacts (line 219)
>>   In clean_rawdata (line 83) 
>> Determining per-component thresholds...done.
>> Warning: A value of class "com.sun.management.UnixOperatingSystem" was indexed with no subscripts specified. Currently the result of
>> this operation is the indexed value itself, but in a future release, it will be an error. 
>> > In asr_process>hlp_memfree (line 232)
>>   In asr_process (line 105)
>>   In clean_asr (line 174)
>>   In clean_artifacts (line 219)
>>   In clean_rawdata (line 83) 
>> Now cleaning data in 1268 blocks.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (NOTE: missing from previous run)
>> Now doing final post-cleanup of the output.
>> Determining time window rejection thresholds...done.
>> Keeping 90.5% (2035 seconds) of the data.
>> eeg_insertbound(): 83 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 132 events removed.
>> eeg_checkset note: value format of event field 'stimname' made uniform
>> eeg_checkset note: value format of event field 'answer' made uniform
>> eeg_insertbound(): 83 boundary (break) events added.
>> eeg_insertbound(): event latencies recomputed and 132 events removed.
>> BUG 1971 WARNING: IF YOU ARE USING A SCRIPT WITTEN FOR A PREVIOUS VERSION OF
>> EEGLAB TO CALL THIS FUNCTION, BECAUSE YOU ARE REJECTING THE ONSET OF THE DATA,
>> EVENTS WERE CORRUPTED. EVENT LATENCIES ARE NOW CORRECT (SEE https://sccn.ucsd.edu/wiki/EEGLAB_bug1971);
>> Use vis_artifacts to compare the cleaned data to the original.
>> Saving dataset...
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
>> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20180922/7f9c2fd6/attachment.html>


More information about the eeglablist mailing list