<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:black;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Hi Kevin,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Yes, the doPCA flag is definitely not supported! ;-) You should just use the pcakeep flag for pca dimensionalit reduction. In this case amica will keep the number of pca dimensions you specify. Even though a full S sphering matrix is output, the actual sphering matrix used is just the first pcakeep rows. So icaact = W * S(1:pcakeep,:) * EEG.data; And icawinv = pinv(W*S(1:pcakeep)).<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>If you don’t specify pcakeep less than nchan, then it will still do the sphering decorrelation to get the starting point, but there will be no dimensionality reduction.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>You can also do the sphering separately if you want. The amica reduced dimension sphering matrix is actually an attempted rotation of the first pcakeep eigenvectors back to a more data-like space, similar to the advantage of the sqrt covariance sphering over eigenvector sphering, as this starting point seems to be better (resulting in faster convergence) than using the eigenvectors, which produce starting components that are relatively more dependent. So the Amica S output in the reduced pcakeep case won’t be the eigenvectors exactly, but a rotation of that space.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Best,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Jason<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Kevin Tan [mailto:kevintan@cmu.edu] <br><b>Sent:</b> Tuesday, August 18, 2015 10:52 PM<br><b>To:</b> japalmer@ucsd.edu; EEGLAB List<br><b>Subject:</b> Re: AMICA lrate gets stuck<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>Hi Jason, <o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I have some PCA-related questions for you. <o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Using the 'doPCA' flag (either 0 or 1) in <span style='font-family:"Courier New"'>runamica12.m</span> results in this error:<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><p class=MsoNormal><span style='font-family:"Courier New"'>runamica(): unknown flag: dopca<br>Error in runamica12 (line 100)<br>if ~isempty(strfind(computer,'64'))<br>Output argument "weights" (and maybe others) not assigned during call to<br>"/home/kevinalt/MATLAB/eeglab13_4_4b/plugins/amica/runamica12.m>runamica12".</span><o:p></o:p></p></blockquote><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Not using 'doPCA' flag still results in "doPCA = 1" in the config file. Not using 'pcakeep' flag results in "pcakeep = chans" in config file.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Is PCA still being performed if pcakeep = chans? Does it matter whether I use command arguments ('pcakeep', chans) vs. just not using the 'pcakeep' flag?<o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>If I do want dimension reduction, is it best to use arguments ('pcakeep', comps) or to perform PCA beforehand then combine PCA eigenvector & ICA matrices? Like this:<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><p class=MsoNormal><span style='font-family:"Courier New"'>[eigenvectors, eigenvalues, PCAdata] = pcsquash(EEG.data(:, :), nmIC);<br>[weights, sphere, mods] = runamica12(PCAdata);<br>weights = weights * sphere * eigenvectors(:, 1:nmIC)';<br>sphere = eye(chans);</span><o:p></o:p></p></blockquote><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Many thanks for the continued help, I really appreciate it!<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>--Kevin <o:p></o:p></p></div></div></div><div><p class=MsoNormal><br clear=all><o:p></o:p></p><div><div><div><div><p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>--</span><o:p></o:p></p></div><p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Kevin Alastair M. Tan</span><o:p></o:p></p><div><p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Lab Manager/Research Assistant</span><o:p></o:p></p><div><p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Department of Psychology & Center for the Neural Basis of Cognition</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Carnegie Mellon University</span><o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'><a href="https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0" target="_blank">Baker Hall 434</a> | <a href="mailto:kevintan@cmu.edu" target="_blank">kevintan@cmu.edu</a> | <a href="http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan" target="_blank">tarrlab.org/kevintan</a></span><o:p></o:p></p></div></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Mon, Aug 17, 2015 at 10:12 PM, Jason Palmer <<a href="mailto:japalmer29@gmail.com" target="_blank">japalmer29@gmail.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Yes single channel sources are common for ICA of EEG with over 100 channels. Single-channel artifacts are probably responsible for them. Cleaning the data more intensely can reduce their number. But the number of “good” components, i.e. detectable, instantaneously independent, relatively large, synchronous patch brain sources, seems to be limited to about to 20-30, with additional degrees of freedom (additional number of channels) being used to deal with non-stationarity and artifacts, so you shouldn’t expect all “good” components.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Best,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Jason</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Kevin Tan [mailto:<a href="mailto:kevintan@cmu.edu" target="_blank">kevintan@cmu.edu</a>] <br><b>Sent:</b> Monday, August 17, 2015 6:51 PM</span><o:p></o:p></p><div><div><p class=MsoNormal><br><b>To:</b> <a href="mailto:japalmer@ucsd.edu" target="_blank">japalmer@ucsd.edu</a>; EEGLAB List<br><b>Subject:</b> Re: AMICA lrate gets stuck<o:p></o:p></p></div></div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Ah ok, did not know the wchange output in Infomax is calculated in that way, thanks for clearing that up. <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>My data is 136ch (but usually 125-130 after PREP) with ~700,000 samples (depending on epoch rejection). Is 10^-5 still a good nd weight change to shoot for?<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Also, is it common to see some single-channel weighted ICs (usually weak ones) even after ensuring the rank is correct? This also occurs with Infomax, wanted to see if AMICA can get rid of it.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thanks again, <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Kevin<o:p></o:p></p></div></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br clear=all><o:p></o:p></p><div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>--</span><o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Kevin Alastair M. Tan</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Lab Manager/Research Assistant</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Department of Psychology & Center for the Neural Basis of Cognition</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Carnegie Mellon University</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'><a href="https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0" target="_blank">Baker Hall 434</a> | <a href="mailto:kevintan@cmu.edu" target="_blank">kevintan@cmu.edu</a> | <a href="http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan" target="_blank">tarrlab.org/kevintan</a></span><o:p></o:p></p></div></div></div></div></div></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On Mon, Aug 17, 2015 at 7:35 PM, Jason Palmer <<a href="mailto:japalmer29@gmail.com" target="_blank">japalmer29@gmail.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Hi Kevin,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>The Infomax wchange is actually the weight change TIMES the lrate, which is going to 1e-7. So the actual final wchange for extended infomax is 1e7 * wchange.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>For Amica, if the nd weight change gets down to the 10^-5 magnitude, that is usually about the best you can expect with the large number of parameters being estimated and the finite computer precision. How small it can get depends on the number of samples you have compared to the number of channels. More channels = more parameters (nchan^2) = relatively little data = “noisier” convergence. More data = better determined optimum = less noisy convergence = smaller nd. For 64 channels with 100,000 samples, an nd of 10^-5 sounds appropriate.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>However you can change “maxiter” from the default 2000, using the ‘maxiter’ keyword. This LL should continue to increase and the nd decrease (or at least not increase) beyond 2000 iterations, but not significantly. There should be a weight change “noise floor” reached, where the LL continues to increase by less and less, with possible reductions in lrate, and the nd hovers around the “noise floor”.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Best,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Jason</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Kevin Tan [mailto:<a href="mailto:kevintan@cmu.edu" target="_blank">kevintan@cmu.edu</a>] <br><b>Sent:</b> Monday, August 17, 2015 4:21 PM<br><b>To:</b> <a href="mailto:japalmer@ucsd.edu" target="_blank">japalmer@ucsd.edu</a>; EEGLAB List<br><b>Subject:</b> Re: AMICA lrate gets stuck</span><o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi Jason, <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thanks so much for the detailed response, really helps clarify what drives the lrate changes between the two implementations. <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>However, for the same dataset processed the same way, AMICA yields higher wchange at last iteration (0.0000464763) versus extended Infomax (0.000000). <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>What are some reasons for this discrepancy, and what can I do improve it? Or is weight change between the two implementations not comparable? The entire AMICA log is linked in original post if that helps. <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thanks again, <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Kevin<o:p></o:p></p></div></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br clear=all><o:p></o:p></p><div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>--</span><o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Kevin Alastair M. Tan</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Lab Manager/Research Assistant</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Department of Psychology & Center for the Neural Basis of Cognition</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Carnegie Mellon University</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'><a href="https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0" target="_blank">Baker Hall 434</a> | <a href="mailto:kevintan@cmu.edu" target="_blank">kevintan@cmu.edu</a> | <a href="http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan" target="_blank">tarrlab.org/kevintan</a></span><o:p></o:p></p></div></div></div></div></div></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On Mon, Aug 17, 2015 at 7:06 PM, Jason Palmer <<a href="mailto:japalmer29@gmail.com" target="_blank">japalmer29@gmail.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Hi Kevin,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>The Amica lrate is not supposed to decrease. The algorithm is a more typical gradient descent / Newton optimization algorithm, as opposed to the Infomax implementation in runica.m, which uses a type of simulated annealing, deciding whether to reduce the learning rate based on the angle between recent update directions. The idea there is that this angle will be small when the algorithm is near an optimum, as though it is “heading right for it”, so the lrate gets reduced if the algorithm is moving “erratically” with a large angle between consecutive directions, and doesn’t get reduced if the estimate is “moving smoothly”. In practice, this annealing method usually ends up in fact reducing the learning rate continuously until it reaches the preset minimum, which usually happens at around 500 iterations (500 reductions). I.e. the angle is never actually small, so the stopping condition is essentially a maximum number of iterations, with the updates being of smaller and smaller magnitude due to the lrate decreasing.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Amica only reduces the lrate if the likelihood decreases. In theory, with a reasonable optimum, an optimization algorithm should be able to converge without reducing the learning rate. The convergence is measured by the weight change (the nd in the amica output) independently of the lrate. That is, the weight change should theoretically decrease to zero with a constant (sufficiently small) lrate—the higher the better since higher lrate means faster convergence. A potential issue with the runica Infomax is early convergence if you are starting far from the optimum. Fortunately the optimum is usually not far from the “sphering” starting point, so 500 iterations is usually enough to converge (even with decreasing lrate).</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>So in Amica, the convergence is judged by the “nd”, not the lrate. The lrate should be ideally be 0.5 or 1.0, and the LL should be increasing, and the nd should be decreasing to zero.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Hope that is helpful.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Best,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Jason</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Kevin Tan [mailto:<a href="mailto:kevintan@cmu.edu" target="_blank">kevintan@cmu.edu</a>] <br><b>Sent:</b> Monday, August 17, 2015 2:31 PM<br><b>To:</b> <a href="mailto:jason@sccn.ucsd.edu" target="_blank">jason@sccn.ucsd.edu</a>; EEGLAB List<br><b>Subject:</b> AMICA lrate gets stuck</span><o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi Dr. Palmer & EEGLAB list, <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I'm trying out AMICA for artifact rejection and DIPFIT. In my tests, the lrate consistently gets stuck at 0.5, stopping only due to max iteration limit. This does not happen with extended Infomax. <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>This happens whether I use the cluster (128 threads) or a normal PC (4 threads). I run AMICA 'locally' as it's called within a matlab script already run via PBS, not sure if that makes a difference. <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Here's the AMICA test stream:<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>- PREP pipeline<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>- Remove PREP-interpolated channels<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>- Remove 1 additional channel for rank consistency<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>- 1hz FIR hi-pass<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>- Epoch -500 to 1000ms no baseline correction<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>- Epoch rejection<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>- AMICA (using EEG(:,:) -- is it ok to concatenate epochs like this?)<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Here's the output log (using the cluster):<o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><a href="https://cmu.box.com/s/t7j3shmwjj1wj8to80au8mdm6b5676rh" target="_blank">https://cmu.box.com/s/t7j3shmwjj1wj8to80au8mdm6b5676rh</a><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Many thanks, <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Kevin<o:p></o:p></p></div><div><div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>--</span><o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Kevin Alastair M. Tan</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Lab Manager/Research Assistant</span><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Department of Psychology & Center for the Neural Basis of Cognition</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Carnegie Mellon University</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'><a href="https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0" target="_blank">Baker Hall 434</a> | <a href="mailto:kevintan@cmu.edu" target="_blank">kevintan@cmu.edu</a> | <a href="http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan" target="_blank">tarrlab.org/kevintan</a></span><o:p></o:p></p></div></div></div></div></div></div></div></div></div></div></div></div></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div></div></div></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div></body></html>