Compiling EEGLAB: Technical note for developers

From SCCN
Jump to: navigation, search

In this page, we address how to set up computational resources at SCCN to compile EEGLAB nightly. Notes and instructions are provided for three different OS: macOS, Ubuntu, and Windows. In the current setup, we are using a Mac computer (delorming.ucsd.edu) to host two virtual machines (VM). VMWare Fusion is used in the host machine to run Windows (Windows 10 x64) and Linux (Ubuntu 64-bit 18.04). The compiled EEGLAB files for macOS and Windows are stored in a common folder synchronized through Google Drive. The compiled files in the Ubuntu VM are transferred to the Google Drive file in the host machine, where they are uploaded and synchronized. Compilation of EEGLAB in the three OS is performed nightly. For this, a system daemon (cron) is used to execute the task in the Mac (host) and Ubuntu (VM) machines. To automate the compilation in the Windows VM, a job was set up in Task Scheduler. In the following section, details for each specific OS are provided. Before beginning, make sure the host Mac is connected to the Internet, and that you have access to the Google account used in the Google Drive app.

Contents

Compilation setup on macOS (Host machine: delorming)

  • Start by installing MATLAB on the host machine. In the current setup (10/18), MATLAB 2018a is installed. It is important to check and install any updates released by Mathworks for the version installed. E.g., for the 2018a version, the update here is necessary for proper operation of the compiled EEGLAB.
  • Set up the EEGLAB git repository. To be consistent across all three OS, the repository should be created in ~/program_files/eeglab. The following code can be used on the terminal to create the folder and set up the repository.
cd ~
mkdir program_files
git clone https://github.com/eeglabdevelopers/eeglab.git
git pull
git submodule update --init --recursive
git pull --recurse-submodules
  • Add EEGLAB to the MATLAB path. You may run into issues with permissions. To fix this. Locate the pathdef.m file and change the permissions. To locate the file, type in MATLAB command window: which('pathdef.m'). Use the output to locate the file in the terminal to change the permission using: chmod 777 path2pathdef
chmod 777 path2pathdef.m
  • Download and install the Google Drive App for Mac. The default name of the Google Drive folder is Google Drive. Since in a Unix-based system spaces in pathnames may raise issues (and are also aesthetically dubious), the folder is renamed as Google Drive. After renaming the folder, you need to make sure to update the link to the new folder in the Google Drive App. It is probable you will receive a notification from the application regarding the lost of the link to the file. This is a reminder of what to do in this step.
  • Copy the folder eeglab_comp_daily, located in .../program_files/eeglab/functions/adminfunc/, to the Google Drive folder. This folder contains three subfolders: compile_tools, macos, ubuntu and windows. The folder compile_tools contains the scripts necessary for the compilation and setup of crontab job. The folders macos, ubuntu and windows will be initially empty, but will be populated with the compilation files for each OS.
  • Setup a crontab job to automatize the daily compilation. For this, create a crontab job from the terminal, as indicated here;
crontab –e

This will open a text editor (e.g., vim), then use the content of the file crontab_task_mack_bckup.txt in the folder compile_tools to create the job:

# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │                                   7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * * command to execute
59 23 * * * /Users/eeglab/Google_Drive/eeglab_comp_daily/compile_tools/cronjob_eeglab_mac.sh

Notice this crontab job is pointing to the execution of the file cronjob_eeglab_mac.sh located on the same folder, which triggers the update of the EEGLAB repository and then starts the compilation, implemented in the file eeglab_compile.m.

Compilation setup in Ubuntu (Virtual machine: delorming)

Start by installing Ubuntu in the host machine using VMWare Fusion. Once installed:

  • Install MATLAB in the Ubuntu VM. In the current setup we have installed MATLAB 2018a. The MATLAB update indicated in the MAC setup need to be installed as well (See section ##).
  • Setup up the EEGLAB git repository. The repository should be created in /home/delorming/program_files. For this, the same code used in the MAC setup can be used.
cd ~
mkdir program_files
git clone https://github.com/eeglabdevelopers/eeglab.git
git pull
git submodule update --init --recursive
git pull --recurse-submodules
  • Add EEGLAB to the MATLAB path. You may run into issues with permissions. See details on the same step in the Mac configuration.
  • Install open-vmware-tools and start service. This is necessary to ensure the sharing of files across the host and virtual Ubuntu machine.
sudo apt-get install open-vm-tools
sudo /etc/init.d/vmware-tools start
  • Mount delorming, the host machine, over the network. To do this, open Files and click in Other locations (see step 1 in the figure below), then double-click in the delorming icon (see step 2 in the figure below).
Mount delorming network1.jpg


After doing this, the link showing the local connection to the host machine has been established (see the figure below).

Mount delorming network2.jpg


Once the connection is established, the folder compile_tools in the Google_Drive folder of the host machine can be accessed as:

cd /run/user/1000/gvfs/sftp:host=delorming.local/Users/eeglab/Google_Drive/eeglab_comp_daily/compile_tools
  • Before creating the crontab job, make sure the file cronjob_eeglab_ubuntu.sh is executable (change file permissions if necessary). For this you can use the following code (assuming as your current directory compile_tools):
 
sudo chmod 777 cronjob_eeglab_ubuntu.sh 
sudo chmod +x cronjob_eeglab_ubuntu.sh
  • Then, a crontab needs to be created to perform the compilation daily. For this, the code in the file crontab_task_ubuntu_bckup.txt should be used (see code below).
# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │                                   7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * * command to execute
1 1 * * * /run/user/1000/gvfs/sftp:host=delorming.local/Users/eeglab/Google_Drive/eeglab_comp_daily/compile_tools/cronjob_eeglab_ubuntu.sh

After setting up the crontab, the compilation will be done by the execution of the very same file used in the compilation on the host machine delorming. Remember that this file is located in the host machine and is accessed through a local sharing of its folder. After running the compilation, performed in the local eeglab folder, the compilation files are transferred to the host machine and stored in ~/Google_Drive/eeglab_comp_daily/ubuntu/.

Network issues experienced in the Ubuntu VM

It is possible the Ubuntu VM lose network access at some point while Windows VM is up and running. While the reason for this behavior is still unknown, a temporary solution has been to: 1-Identify the interface that is disconnected (down) (use command: ip a) 2-Start the service manually using the id for the interface, 'ens33' in this case ( use command: sudo ifup ens33)

After this, the host machine should be mounted over the network again (see previous steps).

Compilation setup in Windows (Virtual Machine)

Compilation setup in the Windows VM will repeat some of the steps for the host and Ubuntu VM compilations. However, some different steps are required, to automate the compilation process and install the git management software.

  • Start by installing MATLAB and the updates indicated for the MATLAB version. As for the host and Ubuntu virtual machine, in the current setup we use MATLAB 2018a with the updates indicated [here].
  • Download and install Google Drive, then log into it. In this VM, the name of the Google Drive folder is not modified.
  • Download and install git and git manager software. In the current setup, Git for Windows has been used.
  • Setup the EEGLAB git repository in the git bash. For this use, the same code used as in the Mac and Ubuntu compliations -- but from the Git Bash terminal.
cd ~
mkdir program_files
git clone https://github.com/eeglabdevelopers/eeglab.git
git pull
git submodule update --init --recursive
git pull --recurse-submodules
  • Add EEGLAB to the MATLAB path.
  • To automatize the daily compilation, the Task Scheduler in Windows is used. Follow the instructions below to create a new task.
  • From the Start Menu, open the Task Scheduler. if the program is not listed directly in the menu, you can type scheduler in the search bar in the same menu to look for it.
  • Once you have opened the Task Scheduler (see figure below), click on 'Create task' in the right menu (highlighted with a red dotted box).
Task scheduler main interface f1.jpg
  • Enter the name of the task (see Step 1 in the figure below) and a short description (see Step 2 in the figure below). Check the options to allow the task to run whether the users are logged in or not (see Step 3 in the figure below) and to give the task the highest privileges (see Step 4 in the figure below).
Task scheduler create task f2.jpg
  • Set up the Triggers of the task. For this, click Triggers in the upper menu (see 1 in the figure below). Then set the option for running the task daily (see 2 in the figure below) and specify when the task should run (see 3 in the figure below). Click OK. The time to run the task should be set so as to not overlap the compilations on the host and Ubuntu machines.
Task sheduler create task trigger f3.jpg

After setting the triggers, the graphics interface will display the task as follows.

Task scheduler create task trigger f3b.jpg
  • Now setup the menu Action to indicate the script that will run every day. We have created a .bat file (batchjob_eeglab_win.bat) that basically perform the same functions as the crontab jobs implemented in the Mac (host) and the virtual Linux machine: updating the git repository and then running the compilation file. To set up this option, click 'Actions in the upper menu ' (see 1 in the figure below). Then add a new action by clicking on the New button (see 2 in the figure below). A new window (New action) will pop up allowing you to specify the file to run. Look for and select the file batchjob_eeglab_win.bat located in ..\Google Drive\eeglab_comp_daily\compile_tools by clicking in the Browse.. button (see 3 in the figure below). Click OK.
Task scheduler create task actions f4.jpg
  • Finally, to create the task go back to the upper menu General and click OK. Now the task has been created and the compilation is ready to run daily.