[Eeglablist] AMICA15 segmentation fault
Norman Forschack
forschack at cbs.mpg.de
Wed Dec 16 11:42:43 PST 2015
Hi Jason,
thanks a lot for your quick reply and yes you're right, reducing block size and steps solves the issue as well as doing amica on a subset of the data, at least for the dataset I tested it on.
Unfortunately, computation time now increased by a factor of three (with eight cores). I'll test a little further to find the optimal settings.
When having pcakeep option enabled, I still get super small but negative eigenvalues. Do you think this is a problem and would you rather recommend not using pca at all but removing channels to give full rank data to amica?
cheers
Norman
----- On Dec 16, 2015, at 4:51 PM, Jason Palmer japalmer29 at gmail.com wrote:
> Hi Nathan,
>
> Thanks for your early beta testing ... I was getting local feedback/debugging
> before announcing new version to wider list, but I guess it makes more sense to
> release a sort of beta version to the eeglablist. The new binaries (with
> sphering bug hopefully fixed) and test version of eeglab plugin is at:
> http://sccn.ucsd.edu/~jason/amica_web.html
> and there is a help link on that page for more information on arguments in the
> gui. So, anyone is welcome to try amica15, and the plugin, and help test it.
>
> I think what you are actually getting is a stack overflow because it is
> attempting to use too large a matrix block size with a large number of samples.
> I think do_opt_block is enabled by default, and it tries 256:256:1024 to find
> the fastest blocksize. You can change this with the Block sizes, etc. button.
> In the gui, or the blk_min, blk_step, blk_max keywords.
>
> Probably if you set smaller values, like blk_min = 64, blk_step = 64, blk_max =
> 256, it will get through the tests without segmentation fault. I'm still
> working on catching this error in the code.
>
> So I don't think your issue has to do with sphering--it is just printing the
> eigenvalue info twice (which I will remove) but I don't think anything is going
> wrong.
>
> For anyone who would like to look at and/or compile the source code, you can get
> it at:
> https://github.com/japalmer29/amica
> with a readme on how to compile it using a demo version of the Intel Fortran
> compiler on various platforms.
>
> Thanks, and happy winter!
> Jason
>
>
> -----Original Message-----
> From: eeglablist-bounces at sccn.ucsd.edu [mailto:eeglablist-bounces at sccn.ucsd.edu]
> On Behalf Of Norman Forschack
> Sent: Monday, December 14, 2015 4:40 PM
> To: eeglablist
> Subject: [Eeglablist] AMICA15 segmentation fault
>
> Dear eeglabbers,
>
> first of all, Jason, thanks a lot for providing the new version of amica for us,
> your input is always highly appreciated!
>
> I tested the new version with the dataset from the homepage and it works neatly
> (after adjusting the call to the Ubuntu binary to 'amica15ub').
>
> However, when running it on another EEG dataset with 62 channels, I have some
> trouble with negative but very small eigenvalues, which might be related to
> this post http://sccn.ucsd.edu/pipermail/eeglablist/2015/010104.html
> This causes amica to exit after the message 'segmentation fault'. The data is
> comprised of merged ten minutes blocks and is minimally preprocessed:
> 1. PREP pipeline robust average reference 2. resampling to 250Hz 3. 1Hz high
> pass 4. thresholding data points > 180 microV
>
> The issue occurs when pcakeep option is enabled (see below for command line
> output)
>
> I tried without pcakeep but with removed channels, mainly EOG and PREP
> interpolated channels, in order to input full rank data. This yields positive
> eigenvalues, but again, the algorithm exits with 'segmentation fault'.
>
> I have no idea how to overcome this, but I'm wondering why eigenvalues are
> calculated twice? Are they supposed to differ at the second instance? In my
> case they don't. (see command line output below) Or could there probably be
> something wrong with my data preprocessing, am I missing something?
>
> Any hint how to solve the problem, is highly anticipated.
>
> Best
> Norman
>
>
> run with data dimensions reduced:
>
> Writing data file: /.../tmpdata96489.fdt
> mkdir: cannot create directory '/.../amicaouttmp/': File exists
> 1 processor name = kambodscha
> 1 host_num = 2033974327
> This is MPI process 1 of 1 ; I am process 1 of
> 1 on node: kambodscha
> 1 : node root process 1 of 1
> Processing arguments ...
> num_files = 1
> FILES:
> /.../tmpdata96489.fdt
> num_dir_files = 1
> initial matrix block_size = 128
> do_opt_block = 1
> blk_min = 256
> blk_step = 256
> blk_max = 1024
> number of models = 1
> max_thrds = 8
> use_min_dll = 1
> min dll = 1.000000000000000E-009
> use_grad_norm = 1
> min grad norm = 1.000000000000000E-007
> number of density mixture components = 3
> pdf type = 0
> max_iter = 2000
> num_samples = 1
> data_dim = 61
> field_dim = 1115606
> do_history = 0
> histstep = 10
> share_comps = 0
> share_start = 100
> comp_thresh = 0.990000000000000
> share_int = 100
> initial lrate = 5.000000000000000E-002
> minimum lrate = 1.000000000000000E-008
> minimum data covariance eigenvalue = 1.000000000000000E-012
> lrate factor = 0.500000000000000
> initial rholrate = 5.000000000000000E-002
> rho0 = 1.50000000000000
> min rho = 1.00000000000000
> max rho = 2.00000000000000
> rho lrate factor = 0.500000000000000
> kurt_start = 3
> num kurt = 5
> kurt interval = 1
> do_newton = 1
> newt_start = 50
> newt_ramp = 10
> initial newton lrate = 1.00000000000000
> do_reject = 1
> num reject = 3
> reject sigma = 3.00000000000000
> reject start = 3
> reject interval = 3
> write step = 20
> write_nd = 0
> write_LLt = 1
> dec window = 1
> max_decs = 3
> fix_init = 0
> update_A = 1
> update_c = 1
> update_gm = 1
> update_alpha = 1
> update_mu = 1
> update_beta = 1
> invsigmax = 100.000000000000
> invsigmin = 0.000000000000000E+000
> do_rho = 1
> load_rej = 0
> load_c = 0
> load_gm = 0
> load_alpha = 0
> load_mu = 0
> load_beta = 0
> load_rho = 0
> load_comp_list = 0
> do_mean = 1
> do_sphere = 1
> pcakeep = 57
> pcadb = 30.0000000000000
> byte_size = 4
> doscaling = 1
> scalestep = 1
> mkdir: cannot create directory '/.../amicaouttmp/': File exists output
> directory = /.../amicaouttmp/
> 1 : setting num_thrds to 8 ...
> 1 : using 8 threads.
> 1 : node_thrds = 8
> bytes in real = 1
> 1 : REAL nbyte = 1
> getting segment list ...
> blocks in sample = 1115606
> total blocks = 1115606
> node blocks = 1115606
> node 1 start: file 1 sample 1 index
> 1
> node 1 stop : file 1 sample 1 index
> 1115606
> 1 : data = -4.96359205245972 -14.1346406936646
> getting the mean ...
> mean = -0.314507310018316 -0.304240499189297
> -5.144710467579993E-002
> subtracting the mean ...
> getting the covariance matrix ...
> cnt = 1115606
> doing eig nx = 61 lwork = 37210
> minimum eigenvalues = -5.406684296262385E-013 -1.285030795456605E-014
> 2.538997580317705E-013
> maximum eigenvalues = 4158.38269386037 1002.93581119057
> 747.275462874866
> num eigs kept = 57
> getting the sphering matrix ...
> minimum eigenvalues = -5.406684296262385E-013 -1.285030795456605E-014
> 2.538997580317705E-013
> maximum eigenvalues = 4158.38269386037 1002.93581119057
> 747.275462874866
> num eigs kept = 57
> sphering the data ...
> numeigs = 57
> 1 : Allocating variables ...
> 1 : Initializing variables ...
> 1 : Determining optimal block size ....
> /home/.../amica15ub /.../amicaouttmp/input.param: Segmentation fault No gm
> present, setting num_models to 1 No W present, exiting
>
>
> run with interpolated channels removed:
>
> Writing data file: /.../tmpdata95717.fdt
> mkdir: cannot create directory '/.../amicaouttmp/': File exists
> 1 processor name = kambodscha
> 1 host_num = 2033974327
> This is MPI process 1 of 1 ; I am process 1 of
> 1 on node: kambodscha
> 1 : node root process 1 of 1
> Processing arguments ...
> num_files = 1
> FILES:
> /.../tmpdata95717.fdt
> num_dir_files = 1
> initial matrix block_size = 128
> do_opt_block = 1
> blk_min = 256
> blk_step = 256
> blk_max = 1024
> number of models = 1
> max_thrds = 8
> use_min_dll = 1
> min dll = 1.000000000000000E-009
> use_grad_norm = 1
> min grad norm = 1.000000000000000E-007
> number of density mixture components = 3
> pdf type = 0
> max_iter = 2000
> num_samples = 1
> data_dim = 57
> field_dim = 1116754
> do_history = 0
> histstep = 10
> share_comps = 0
> share_start = 100
> comp_thresh = 0.990000000000000
> share_int = 100
> initial lrate = 5.000000000000000E-002
> minimum lrate = 1.000000000000000E-008
> minimum data covariance eigenvalue = 1.000000000000000E-012
> lrate factor = 0.500000000000000
> initial rholrate = 5.000000000000000E-002
> rho0 = 1.50000000000000
> min rho = 1.00000000000000
> max rho = 2.00000000000000
> rho lrate factor = 0.500000000000000
> kurt_start = 3
> num kurt = 5
> kurt interval = 1
> do_newton = 1
> newt_start = 50
> newt_ramp = 10
> initial newton lrate = 1.00000000000000
> do_reject = 1
> num reject = 3
> reject sigma = 3.00000000000000
> reject start = 3
> reject interval = 3
> write step = 20
> write_nd = 0
> write_LLt = 1
> dec window = 1
> max_decs = 3
> fix_init = 0
> update_A = 1
> update_c = 1
> update_gm = 1
> update_alpha = 1
> update_mu = 1
> update_beta = 1
> invsigmax = 100.000000000000
> invsigmin = 0.000000000000000E+000
> do_rho = 1
> load_rej = 0
> load_c = 0
> load_gm = 0
> load_alpha = 0
> load_mu = 0
> load_beta = 0
> load_rho = 0
> load_comp_list = 0
> do_mean = 1
> do_sphere = 1
> pcakeep = 57
> pcadb = 30.0000000000000
> byte_size = 4
> doscaling = 1
> scalestep = 1
> mkdir: cannot create directory '/.../amicaouttmp/': File exists
> output directory = /.../amicaouttmp/
> 1 : setting num_thrds to 8 ...
> 1 : using 8 threads.
> 1 : node_thrds = 8
> bytes in real = 1
> 1 : REAL nbyte = 1
> getting segment list ...
> blocks in sample = 1116754
> total blocks = 1116754
> node blocks = 1116754
> node 1 start: file 1 sample 1 index
> 1
> node 1 stop : file 1 sample 1 index
> 1116754
> 1 : data = -4.96453666687012 -14.1324272155762
> getting the mean ...
> mean = -0.286942247592682 -0.278907745979486
> -4.659303225201700E-002
> subtracting the mean ...
> getting the covariance matrix ...
> cnt = 1116754
> doing eig nx = 57 lwork = 32490
> minimum eigenvalues = 0.328194400980202 0.540830260003019
> 0.590640211589886
> maximum eigenvalues = 3591.26449346892 979.044648222552
> 723.438382000281
> num eigs kept = 57
> getting the sphering matrix ...
> minimum eigenvalues = 0.328194400980202 0.540830260003019
> 0.590640211589886
> maximum eigenvalues = 3591.26449346892 979.044648222552
> 723.438382000281
> num eigs kept = 57
> sphering the data ...
> numeigs = 57
> 1 : Allocating variables ...
> 1 : Initializing variables ...
> 1 : Determining optimal block size ....
> /home/.../amica15ub /.../amicaouttmp/input.param: Segmentation fault
> No gm present, setting num_models to 1
> No W present, exiting
>
>
> Sometimes I also get a more verbose feedback, but I cannot reproduce this
> consistently:
>
> forrtl: severe (174): SIGSEGV, segmentation fault occurred
> forrtl: severe (174): SIGSEGV, segmentation fault occurred
> /.../amica15ub /.../amicaouttmp/input.param: Signal 46
>
>
>
> ___________________________________________________________
> Norman Forschack, Dipl.-Psych.
> Max-Planck Institute for Human Cognitive and Brain Sciences
> Stephanstraße 1a
> 04103 Leipzig
>
> mail: forschack at cbs.mpg.de
> phone: +49341 9940171
> web: http://www.cbs.mpg.de/~forschack
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
More information about the eeglablist
mailing list