[Eeglablist] AMICA lrate gets stuck

Wed Aug 19 11:40:12 PDT 2015

So if I want to put the weights and sphere into EEG.data struct for
downstream EEGLab processing, should I do:

EEG.icaweights = weights;
EEG.icasphere = sphere(1:pcakeep, :);

Thanks again,
Kevin

--
Kevin Alastair M. Tan
Lab Manager/Research Assistant
Department of Psychology & Center for the Neural Basis of Cognition
Carnegie Mellon University

Baker Hall 434
<https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0>
 | kevintan at cmu.edu | tarrlab.org/kevintan
<http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan>

On Wed, Aug 19, 2015 at 2:14 AM, Jason Palmer <japalmer29 at gmail.com> wrote:

> Hi Kevin,
>
>
>
> Yes, the doPCA flag is definitely not supported!  ;-)  You should just use
> the pcakeep flag for pca dimensionalit reduction. In this case amica will
> keep the number of pca dimensions you specify. Even though a full S
> sphering matrix is output, the actual sphering matrix used is just the
> first pcakeep rows. So  icaact = W * S(1:pcakeep,:) * EEG.data; And icawinv
> = pinv(W*S(1:pcakeep)).
>
>
>
> If you don’t specify pcakeep less than nchan, then it will still do the
> sphering decorrelation to get the starting point, but there will be no
> dimensionality reduction.
>
>
>
> You can also do the sphering separately if you want. The amica reduced
> dimension sphering matrix is actually an attempted rotation of the first
> pcakeep eigenvectors back to a more data-like space, similar to the
> advantage of the sqrt covariance sphering over eigenvector sphering, as
> this starting point seems to be better (resulting in faster convergence)
> than using the eigenvectors, which produce starting components that are
> relatively more dependent. So the Amica S output in the reduced pcakeep
> case won’t be the eigenvectors exactly, but a rotation of that space.
>
>
>
> Best,
>
> Jason
>
>
>
> *From:* Kevin Tan [mailto:kevintan at cmu.edu]
> *Sent:* Tuesday, August 18, 2015 10:52 PM
>
> *To:* japalmer at ucsd.edu; EEGLAB List
> *Subject:* Re: AMICA lrate gets stuck
>
>
>
> Hi Jason,
>
>
>
> I have some PCA-related questions for you.
>
>
>
> Using the 'doPCA' flag (either 0 or 1) in runamica12.m results in this
> error:
>
>
>
> runamica(): unknown flag: dopca
> Error in runamica12 (line 100)
> if ~isempty(strfind(computer,'64'))
> Output argument "weights" (and maybe others) not assigned during call to
>
> "/home/kevinalt/MATLAB/eeglab13_4_4b/plugins/amica/runamica12.m>runamica12".
>
>
>
> Not using 'doPCA' flag still results in "doPCA = 1" in the config file.
> Not using 'pcakeep' flag results in "pcakeep = chans" in config file.
>
>
>
> Is PCA still being performed if pcakeep = chans? Does it matter whether I
> use command arguments ('pcakeep', chans) vs. just not using the 'pcakeep'
> flag?
>
>
>
> If I do want dimension reduction, is it best to use arguments ('pcakeep',
> comps) or to perform PCA beforehand then combine PCA eigenvector & ICA
> matrices? Like this:
>
>
>
> [eigenvectors, eigenvalues, PCAdata] = pcsquash(EEG.data(:, :), nmIC);
> [weights, sphere, mods] = runamica12(PCAdata);
> weights = weights * sphere * eigenvectors(:, 1:nmIC)';
> sphere = eye(chans);
>
>
>
> Many thanks for the continued help, I really appreciate it!
>
>
>
> --Kevin
>
>
> --
>
> Kevin Alastair M. Tan
>
> Lab Manager/Research Assistant
>
> Department of Psychology & Center for the Neural Basis of Cognition
>
> Carnegie Mellon University
>
>
>
> Baker Hall 434
> <https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0>
>  | kevintan at cmu.edu | tarrlab.org/kevintan
> <http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan>
>
>
>
> On Mon, Aug 17, 2015 at 10:12 PM, Jason Palmer <japalmer29 at gmail.com>
> wrote:
>
> Yes single channel sources are common for ICA of EEG with over 100
> channels. Single-channel artifacts are probably responsible for them.
> Cleaning the data more intensely can reduce their number. But the number of
> “good” components, i.e. detectable, instantaneously independent, relatively
> large, synchronous patch brain sources, seems to be limited to about to
> 20-30, with additional degrees of freedom (additional number of channels)
> being used to deal with non-stationarity and artifacts, so you shouldn’t
> expect all “good” components.
>
>
>
> Best,
>
> Jason
>
>
>
> *From:* Kevin Tan [mailto:kevintan at cmu.edu]
> *Sent:* Monday, August 17, 2015 6:51 PM
>
>
> *To:* japalmer at ucsd.edu; EEGLAB List
> *Subject:* Re: AMICA lrate gets stuck
>
>
>
> Ah ok, did not know the wchange output in Infomax is calculated in that
> way, thanks for clearing that up.
>
>
>
> My data is 136ch (but usually 125-130 after PREP) with ~700,000 samples
> (depending on epoch rejection). Is 10^-5 still a good nd weight change to
> shoot for?
>
>
>
> Also, is it common to see some single-channel weighted ICs (usually weak
> ones) even after ensuring the rank is correct? This also occurs with
> Infomax, wanted to see if AMICA can get rid of it.
>
>
>
> Thanks again,
>
> Kevin
>
>
> --
>
> Kevin Alastair M. Tan
>
> Lab Manager/Research Assistant
>
> Department of Psychology & Center for the Neural Basis of Cognition
>
> Carnegie Mellon University
>
>
>
> Baker Hall 434
> <https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0>
>  | kevintan at cmu.edu | tarrlab.org/kevintan
> <http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan>
>
>
>
> On Mon, Aug 17, 2015 at 7:35 PM, Jason Palmer <japalmer29 at gmail.com>
> wrote:
>
> Hi Kevin,
>
>
>
> The Infomax wchange is actually the weight change TIMES the lrate, which
> is going to 1e-7. So the actual final wchange for extended infomax is 1e7 *
> wchange.
>
>
>
> For Amica, if the nd weight change gets down to the 10^-5 magnitude, that
> is usually about the best you can expect with the large number of
> parameters being estimated and the finite computer precision. How small it
> can get depends on the number of samples you have compared to the number of
> channels. More channels = more parameters (nchan^2) = relatively little
> data = “noisier” convergence. More data = better determined optimum = less
> noisy convergence = smaller nd. For 64 channels with 100,000 samples, an nd
> of 10^-5 sounds appropriate.
>
>
>
> However you can change “maxiter” from the default 2000, using the
> ‘maxiter’ keyword. This LL should continue to increase and the nd decrease
> (or at least not increase) beyond 2000 iterations, but not significantly.
> There should be a weight change “noise floor” reached, where the LL
> continues to increase by less and less, with possible reductions in lrate,
> and the nd hovers around the “noise floor”.
>
>
>
> Best,
>
> Jason
>
>
>
> *From:* Kevin Tan [mailto:kevintan at cmu.edu]
> *Sent:* Monday, August 17, 2015 4:21 PM
> *To:* japalmer at ucsd.edu; EEGLAB List
> *Subject:* Re: AMICA lrate gets stuck
>
>
>
> Hi Jason,
>
>
>
> Thanks so much for the detailed response, really helps clarify what drives
> the lrate changes between the two implementations.
>
>
>
> However, for the same dataset processed the same way, AMICA yields higher
> wchange at last iteration (0.0000464763) versus extended Infomax
> (0.000000).
>
>
>
> What are some reasons for this discrepancy, and what can I do improve it?
> Or is weight change between the two implementations not comparable? The
> entire AMICA log is linked in original post if that helps.
>
>
>
> Thanks again,
>
> Kevin
>
>
> --
>
> Kevin Alastair M. Tan
>
> Lab Manager/Research Assistant
>
> Department of Psychology & Center for the Neural Basis of Cognition
>
> Carnegie Mellon University
>
>
>
> Baker Hall 434
> <https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0>
>  | kevintan at cmu.edu | tarrlab.org/kevintan
> <http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan>
>
>
>
> On Mon, Aug 17, 2015 at 7:06 PM, Jason Palmer <japalmer29 at gmail.com>
> wrote:
>
> Hi Kevin,
>
>
>
> The Amica lrate is not supposed to decrease. The algorithm is a more
> typical gradient descent / Newton optimization algorithm, as opposed to the
> Infomax implementation in runica.m, which uses a type of simulated
> annealing, deciding whether to reduce the learning rate based on the angle
> between recent update directions. The idea there is that this angle will be
> small when the algorithm is near an optimum, as though it is “heading right
> for it”, so the lrate gets reduced if the algorithm is moving “erratically”
> with a large angle between consecutive directions, and doesn’t get reduced
> if the estimate is “moving smoothly”. In practice, this annealing method
> usually ends up in fact reducing the learning rate continuously until it
> reaches the preset minimum, which usually happens at around 500 iterations
> (500 reductions). I.e. the angle is never actually small, so the stopping
> condition is essentially a maximum number of iterations, with the updates
> being of smaller and smaller magnitude due to the lrate decreasing.
>
>
>
> Amica only reduces the lrate if the likelihood decreases. In theory, with
> a reasonable optimum, an optimization algorithm should be able to converge
> without reducing the learning rate. The convergence is measured by the
> weight change (the nd in the amica output) independently of the lrate. That
> is, the weight change should theoretically decrease to zero with a constant
> (sufficiently small) lrate—the higher the better since higher lrate means
> faster convergence. A potential issue with the runica Infomax is early
> convergence if you are starting far from the optimum. Fortunately the
> optimum is usually not far from the “sphering” starting point, so 500
> iterations is usually enough to converge (even with decreasing lrate).
>
>
>
> So in Amica, the convergence is judged by the “nd”, not the lrate. The
> lrate should be ideally be 0.5 or 1.0, and the LL should be increasing, and
> the nd should be decreasing to zero.
>
>
>
> Hope that is helpful.
>
>
>
> Best,
>
> Jason
>
>
>
>
>
> *From:* Kevin Tan [mailto:kevintan at cmu.edu]
> *Sent:* Monday, August 17, 2015 2:31 PM
> *To:* jason at sccn.ucsd.edu; EEGLAB List
> *Subject:* AMICA lrate gets stuck
>
>
>
> Hi Dr. Palmer & EEGLAB list,
>
>
>
> I'm trying out AMICA for artifact rejection and DIPFIT. In my tests, the
> lrate consistently gets stuck at 0.5, stopping only due to max iteration
> limit. This does not happen with extended Infomax.
>
>
>
> This happens whether I use the cluster (128 threads) or a normal PC (4
> threads). I run AMICA 'locally' as it's called within a matlab script
> already run via PBS, not sure if that makes a difference.
>
>
>
> Here's the AMICA test stream:
>
> - PREP pipeline
>
> - Remove PREP-interpolated channels
>
> - Remove 1 additional channel for rank consistency
>
> - 1hz FIR hi-pass
>
> - Epoch -500 to 1000ms no baseline correction
>
> - Epoch rejection
>
> - AMICA (using EEG(:,:) -- is it ok to concatenate epochs like this?)
>
>
>
> Here's the output log (using the cluster):
>
> https://cmu.box.com/s/t7j3shmwjj1wj8to80au8mdm6b5676rh
>
>
>
> Many thanks,
>
> Kevin
>
> --
>
> Kevin Alastair M. Tan
>
> Lab Manager/Research Assistant
>
> Department of Psychology & Center for the Neural Basis of Cognition
>
> Carnegie Mellon University
>
>
>
> Baker Hall 434
> <https://www.google.com/maps/place/40%C2%B026%2729.5%22N+79%C2%B056%2744.0%22W/@40.4414869,-79.9455701,61m/data=!3m1!1e3!4m2!3m1!1s0x0:0x0>
>  | kevintan at cmu.edu | tarrlab.org/kevintan
> <http://tarrlabwiki.cnbc.cmu.edu/index.php/KevinTan>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20150819/6daeade8/attachment.html>