[Eeglablist] Large Unexplained Differences in Power Values
Naomi Heffer
nrh31 at bath.ac.uk
Wed Mar 24 08:40:39 PDT 2021
Hi Gideon,
Thank you for taking the time to write such a detailed reply. I will definitely invest the time trying to create a test-dataset as it sounds like a very valuable thing to do. Wish me luck.
Many thanks,
Naomi
From: Gideon P Caplovitz <gcaplovitz at unr.edu>
Sent: 13 March 2021 21:59
To: Naomi Heffer <nrh31 at bath.ac.uk>
Subject: Re: [Eeglablist] Large Unexplained Differences in Power Values
CAUTION: This email came from outside of the University. To keep your account safe, only click on links and open attachments if you know the person who sent the email, or you expected to receive this communication.
Hi Naomi,
Again, I am not experienced with EEGLAB and don't know what clean_artifacts is doing, but one naive approach would be to simply subtract the data array containing the clean data from the data array for the original. This of course makes assumptions about data formats etc... but in theory should give you the ability to really see what had been removed and whether or not it makes sense for your purposes.
As for constructing a test-dataset. Note: this is not necessarily trivial and can be a lot of work depending on how deep you want to dive into it and your comfort level dealing with data.
Start by thinking about your experimental design and anticipated pattern of results derived as the output of your pipeline: i.e.my output waveform for Condition A is bigger than condition B on these electrodes and not those electrodes, etc...
work with the idea that your recorded signal Y(t) = Signal (t) + Noise (t). In practice you cant separate the signal from noise. But in a test data set you create the Signal (t) and can set Noise (t) to zero.
Then create a datafile made of idealized waveforms for each condition that match these effects (i.e. no noise: every trial gives identical response) matching the exact structure of your paradigm/actual data recording. i.e. replace the ALLDATA.data from your actual recording with your idealized dataset. Then run through your pipeline. Your pipeline should be able to recover EXACTLY what you put in. Any unexpected discrepancy between what you put and get out reflects either a bug in your code or a misunderstanding of what your pipeline is doing, and worth checking into.
To get a really solid understanding of various preprocessing steps, you can set signal (t) to be zero and start exploring with noise(t) by making various types of noise/artifacts, really anything you can or want to explore and observing the effects of various stages of your preprocessing/analyses when run through your pipeline. You could use segments of raw eeg, or use random number generators, really anything you can think of. Carrying this forward, you can in theory create a complete power-analysis testbed to explore the effects of noise when added into your test data ( signal(t) + noise(t) ) and your ability to statistically detect the known effects you embedded in the test data. How many trials do you need?, how many subjects etc... given a various assumptions of effect size and noise. Obviously because the EEG is non-stationary, this will always be guesswork but in general, would give you a very solid understanding of the nuts and bolts of what you are doing.
Like I said, not a trivial process, but if time and energy permits, likely worth the effort.
Good luck!
Gideon
On Mar 12, 2021, at 1:22 AM, Naomi Heffer <nrh31 at bath.ac.uk<mailto:nrh31 at bath.ac.uk>> wrote:
Hi Gideon,
Thank you very much for your helpful suggestions.
I am keen to take both of your suggestions on board, but I am not super confident working with Matlab. I wonder if you could provide any more information, or highlight any useful resources, to help me work out how I might extract what was removed by the clean_artifacts command and the best way to go about creating a fabricated dataset?
Many thanks,
Naomi
From: Gideon P Caplovitz <gcaplovitz at unr.edu<mailto:gcaplovitz at unr.edu>>
Sent: 10 March 2021 16:52
To: Naomi Heffer <nrh31 at bath.ac.uk<mailto:nrh31 at bath.ac.uk>>
Subject: Re: [Eeglablist] Large Unexplained Differences in Power Values
CAUTION: This email came from outside of the University. To keep your account safe, only click on links and open attachments if you know the person who sent the email, or you expected to receive this communication.
Hi Naomi,
I am no EEG_LAB expert so take my suggestion with a grain of salt: perhaps you could extract exactly what was removed by the clean_artifacts command. Power spectrum on the original, cleaned and removed should all line up in an intuitive fashion. If not, then maybe something fishy is going on.
Good luck!
GPC
P.S. I always recommend the fabrication of a *test* dataset, created with hypothetical/anticipated signals and embedded with known noise... all under your control. The dataset can be used to test/debug your entire pipeline start to finish prior to analyzing any real data. Time-consuming yes, but virtually priceless and one of the few ways of being 100% confident in your results.
=======================
Gideon P. Caplovitz
Associate Professor
Director of the Cognitive and Brain Sciences Graduate Program
Department of Psychology
University of Nevada Reno
https://urldefense.com/v3/__http://labs.psych.unr.edu/C-Lab__;!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZb9PUVBtA$ <https://urldefense.com/v3/__https://eur01.safelinks.protection.outlook.com/?url=http*3A*2F*2Flabs.psych.unr.edu*2FC-Lab&data=04*7C01*7Cnrh31*40bath.ac.uk*7C56e4fa6e382e413c3f0008d8e66b3121*7C377e3d224ea1422db0ad8fcc89406b9e*7C0*7C0*7C637512695393112189*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=8qJEdKLi1Amjnhjy6aqPU4ciHmkqRnMN4Wps68yXlVc*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSU!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZbEJ1Q8xA$ >
https://urldefense.com/v3/__http://groups.unr.edu/unr_psych_cbs/CBS_grad_program/__;!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZZ3dndJ8A$ <https://urldefense.com/v3/__https://eur01.safelinks.protection.outlook.com/?url=http*3A*2F*2Fgroups.unr.edu*2Funr_psych_cbs*2FCBS_grad_program*2F&data=04*7C01*7Cnrh31*40bath.ac.uk*7C56e4fa6e382e413c3f0008d8e66b3121*7C377e3d224ea1422db0ad8fcc89406b9e*7C0*7C0*7C637512695393122184*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=aQMc8pcbLQ*2BoIenXQlEzI8D30Bx6e8i44EVv*2BBfRNbc*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUl!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZZIgpPBsQ$ >
https://urldefense.com/v3/__https://www.attentioninthebrain.com__;!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZYDDbyTlw$ <https://urldefense.com/v3/__https://eur01.safelinks.protection.outlook.com/?url=https*3A*2F*2Fwww.attentioninthebrain.com*2F&data=04*7C01*7Cnrh31*40bath.ac.uk*7C56e4fa6e382e413c3f0008d8e66b3121*7C377e3d224ea1422db0ad8fcc89406b9e*7C0*7C0*7C637512695393122184*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=UJY3FcUfdNHBcsVjv*2BcgGChXwFAhghXo1g7tD4grq5w*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUl!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZZta4ezew$ >
=======================
On Mar 10, 2021, at 5:43 AM, Naomi Heffer <nrh31 at bath.ac.uk<mailto:nrh31 at bath.ac.uk>> wrote:
Hi Clement,
Yes, I was comparing 'EEG' and 'clean' in my code.
I used all of the defaults for clean_artifacts except I turned the channel criterion off - thank you for highlighting that this means I will have high-pass filtered the data twice. I have now corrected my code so I only filter the data once but this doesn't seem to make much difference to the absolute power values.
The output I get to the command line when I run clean_artifacts seems to suggest that no channels are removed and that a large amount of the data had to be cleaned, but most was ultimately retained:
Keeping 54.5% (205 seconds) of the data.
eeg_insertbound(): 27 boundary (break) events added.
eeg_insertbound(): 27 boundary (break) events added.
Estimating calibration statistics; this may take a while...
Determining per-component thresholds...done.
Now cleaning data in 13 blocks.............
Now doing final post-cleanup of the output.
Determining time window rejection thresholds...done.
Keeping 96.5% (364 seconds) of the data.
eeg_insertbound(): 10 boundary (break) events added.
eeg_insertbound(): 10 boundary (break) events added.
Looking at the plot from vis_artifacts suggests that the cleaning has removed some very noisy sections with large amplitude changes. Do you think this is enough to be causing the 10^3 difference in magnitude in the obtained power values?
Many thanks,
Naomi
From: Clement Lee <cll008 at eng.ucsd.edu<mailto:cll008 at eng.ucsd.edu>>
Sent: 09 March 2021 17:47
To: Naomi Heffer <nrh31 at bath.ac.uk<mailto:nrh31 at bath.ac.uk>>
Cc: eeglablist at sccn.ucsd.edu<mailto:eeglablist at sccn.ucsd.edu>
Subject: Re: [Eeglablist] Large Unexplained Differences in Power Values
CAUTION: This email came from outside of the University. To keep your account safe, only click on links and open attachments if you know the person who sent the email, or you expected to receive this communication.
Hi Naomi,
Are you comparing the power between 'EEG' and 'clean' in your code? What arguments did you use for clean_artifacts (are you using default settings thereby high pass filtering the data twice?) and what does the output in the command line look like (e.g. how many channels or data points removed). Does the vis_artifact step give you any insight on this power difference?
Best,
Clement Lee
Applications Programmer
Swartz Center for Computational Neuroscience
Institute for Neural Computation, UC San Diego
858-822-7535
On Tue, Mar 9, 2021 at 9:13 AM Naomi Heffer <nrh31 at bath.ac.uk<mailto:nrh31 at bath.ac.uk><mailto:nrh31 at bath.ac.uk>> wrote:
Hi there,
I have been trying to calculate power in different frequency bands but I find that whether or not I clean the data using clean_artifacts before running the analysis has a very large impact on the magnitude of the power values.
Here is my code below, along with the absolute power values in each of the frequency bands when (a) I clean the data before running the analysis and (b) I don't use the clean_artifacts function to clean the data. Do you have any idea why I am seeing such massive differences in magnitude? And can you suggest which of the two sets of values is likely to be more realistic?
Many thanks
Naomi
%% 1. Opening & Importing to EEGLAB
% Get data from .easy to EEGLAB .set/.fdt format
EEG = pop_easy(filename, 1, 0,[]);
% Setting the right channel locations. Reading montage file
EEG = pop_chanedit(EEG, 'load',config,'save','mychans.loc');
%% 2. Filtering
% Filter data between 0.1 and 40Hz
% High-pass filter
EEG = pop_eegfiltnew(EEG, 0.5,[], 1690, 0, [], 0);
% Low-pass filter
EEG = pop_eegfiltnew(EEG, [], 35, 86, 0, [], 0);
%% 3.Cleaning of continuous data
clean = clean_artifacts(EEG, 'ChannelCriterion', 'off');
vis_artifacts(clean, EEG)
%% 4. Spectral Analysis
[spectra,freqs] = spectopo(EEG.data(:,:,:), 0, EEG.srate, 'freqrange', [1 80], 'plotmean', 'on', 'overlap', 250);
% delta=1-4, theta=4-8, alpha=8-13, beta=13-30, gamma=30-80
deltaIdx = find(freqs>1 & freqs<4);
thetaIdx = find(freqs>4 & freqs<8);
alphaIdx = find(freqs>8 & freqs<13);
betaIdx = find(freqs>13 & freqs<30);
gammaIdx = find(freqs>30 & freqs<80);
% compute absolute power
deltaPower = mean(10.^(spectra(deltaIdx)/10))
thetaPower = mean(10.^(spectra(thetaIdx)/10))
alphaPower = mean(10.^(spectra(alphaIdx)/10))
betaPower = mean(10.^(spectra(betaIdx)/10))
gammaPower = mean(10.^(spectra(gammaIdx)/10))
1. Values with cleaning:
deltaPower = 3.2846
thetaPower = 2.4440
alphaPower = 8.9593
betaPower = 6.9579
gammaPower = 3.3659
1. Values without cleaning:
deltaPower = 1.5560e+03
thetaPower = 1.3541e+03
alphaPower = 2.7626e+03
betaPower = 1.0275e+03
gammaPower = 97.3422
_______________________________________________
Eeglablist page: https://urldefense.com/v3/__https://nam04.safelinks.protection.outlook.com/?url=http*3A*2F*2Fsccn.ucsd.edu*2Feeglab*2Feeglabmail.html&data=04*7C01*7Cgcaplovitz*40unr.edu*7C6b8da270d1664bb996a908d8e3e12528__;JSUlJSUlJSUl!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZYcoazpXw$ <https://urldefense.com/v3/__https://nam04.safelinks.protection.outlook.com/?url=http*3A*2F*2Fsccn.ucsd.edu*2Feeglab*2Feeglabmail.html&data=04*7C01*7Cgcaplovitz*40unr.edu*7C6b8da270d1664bb996a908d8e3e12528__;JSUlJSUlJSUl!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZZw4N7RFA$ >%7C523b4bfc0ebd4c03b2b96f6a17fd31d8%7C1%7C0%7C637509903444643054%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rejT8GHbRMkqcIXcuc%2F%2B4avjDzt%2Fx3eVxAdz4u6L2II%3D&reserved=0<https://urldefense.com/v3/__https://nam04.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Feur01.safelinks.protection.outlook.com*2F*3Furl*3Dhttp*3A*2F*2Fsccn.ucsd.edu*2Feeglab*2Feeglabmail.html*26data*3D04*7C01*7Cnrh31*40bath.ac.uk*7C1824263b754245fe85dc08d8e323713c*7C377e3d224ea1422db0ad8fcc89406b9e*7C0*7C0*7C637509088666717196*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26sdata*3DiAv9y5*2Fiw4qJXap8eAyQ9LLMEzO2g1rnutx8RDMkkRg*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJQ!!Mih3wA!WTzuiGz0Gq5eIdQ4N8IfWwwfwK1mP-h8c1agVR0aVNY1M_HuBjr2VS7HRu3vE68_t60yGA*24&data=04*7C01*7Cgcaplovitz*40unr.edu*7C6b8da270d1664bb996a908d8e3e12528*7C523b4bfc0ebd4c03b2b96f6a17fd31d8*7C1*7C0*7C637509903444653048*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=*2FlRlkjeuOA01ln5eGIm*2BQRGWWg*2FiBSW1YuP3BIWVQGc*3D&reserved=0__;JSUlJSUlJSUlJSUqKioqKiUlKioqKioqKioqKioqJSUqKiUlJSUlJSUlJSUlJSUlJSUlJSUl!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZb_7APwfg$ <https://urldefense.com/v3/__https://eur01.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Feur01.safelinks.protection.outlook.com*2F*3Furl*3Dhttp*3A*2F*2Fsccn.ucsd.edu*2Feeglab*2Feeglabmail.html*26data*3D04*7C01*7Cnrh31*40bath.ac.uk*7C1824263b754245fe85dc08d8e323713c*7C377e3d224ea1422db0ad8fcc89406b9e*7C0*7C0*7C637509088666717196*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26sdata*3DiAv9y5*2Fiw4qJXap8eAyQ9LLMEzO2g1rnutx8RDMkkRg*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJQ!!Mih3wA!WTzuiGz0Gq5eIdQ4N8IfWwwfwK1mP-h8c1agVR0aVNY1M_HuBjr2VS7HRu3vE68_t60yGA*24&data=04*7C01*7Cnrh31*40bath.ac.uk*7C56e4fa6e382e413c3f0008d8e66b3121*7C377e3d224ea1422db0ad8fcc89406b9e*7C0*7C0*7C637512695393132177*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=zqKCuGE*2BRRbWc8BN8fQZTbQDbfIYfDn17qUw4vk1uaE*3D&reserved=0__;JSUlJSUlJSUlJSUqKioqKiUlKioqKioqKioqKioqJSUqKiUlJSUlJSUlJSUlJSUlJSUlJQ!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZZhW0yENQ$ > >
To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu<mailto:eeglablist-unsubscribe at sccn.ucsd.edu><mailto:eeglablist-unsubscribe at sccn.ucsd.edu>
For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu<mailto:eeglablist-request at sccn.ucsd.edu><mailto:eeglablist-request at sccn.ucsd.edu>
_______________________________________________
Eeglablist page: https://urldefense.com/v3/__https://nam04.safelinks.protection.outlook.com/?url=http*3A*2F*2Fsccn.ucsd.edu*2Feeglab*2Feeglabmail.html&data=04*7C01*7Cgcaplovitz*40unr.edu*7C6b8da270d1664bb996a908d8e3e12528*7C523b4bfc0ebd4c03b2b96f6a17fd31d8*7C1*7C0*7C637509903444653048*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=z7407qt8IVORXL4CtQO4A29cI*2FT5L*2Fd7rWQtxG5M*2FC4*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUl!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZZqUWPGlQ$ <https://urldefense.com/v3/__https://eur01.safelinks.protection.outlook.com/?url=http*3A*2F*2Fsccn.ucsd.edu*2Feeglab*2Feeglabmail.html&data=04*7C01*7Cnrh31*40bath.ac.uk*7C56e4fa6e382e413c3f0008d8e66b3121*7C377e3d224ea1422db0ad8fcc89406b9e*7C0*7C0*7C637512695393142172*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=s*2FgkNTy2cwA6cgAKUjyNRawlKhikaDISM17whnkUxb8*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJQ!!Mih3wA!SrRbMPJq6uc0tH1YxwKNZUNdXJQOVOSzO3PNTWQDt9XuDWZFfb0dlanH4aWTeZbEH0wtrQ$ >
To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu<mailto:eeglablist-unsubscribe at sccn.ucsd.edu>
For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu<mailto:eeglablist-request at sccn.ucsd.edu>
More information about the eeglablist
mailing list