The EEGLAB News #17

EEGLAB and Large Language Models (LLMs)

Using Large Language Models (LLMs) to generate EEGLAB code presents a fascinating intersection of advanced AI techniques and neuroscience. LLMs, trained on vast amounts of text (some claim the entire internet), can understand and replicate the structure and syntax of programming languages, including MATLAB, which is used in EEGLAB. This capability is particularly valuable in EEG data analysis, where custom scripting is often required to preprocess data, conduct statistical analyses, and visualize results. By generating EEGLAB code, LLMs can significantly reduce the time and effort researchers spend on coding, allowing them to focus more on experimental design and data interpretation. Additionally, LLMs can assist in troubleshooting and optimizing code and even suggest new methods or approaches for data analysis based on their extensive training. This not only streamlines the EEG analysis process but also democratizes access to advanced data analysis techniques, making them more accessible to researchers who may not have extensive programming expertise.

We tried the following prompt in ChatGPT (4.0; commercial version) and Bard.

“Create a simple EEGLAB script to import the "test.edf" file in EEGLAB, resample the file to 250 Hz, high pass filter the file at 0.5, run the clean_rawdata to reject bad channels and bad portions of data, then run ICA decomposition and the IClabel plugin.”

On January 7, 2024, We obtained the following result in ChatGPT

And the following result in Bard:

We also tried of free models free models such as Mistral7B without much success (the model could not even find the correct function and invented random function names). In Bard and ChatGPT cases, the model was able to import and filter the data (Bard added some extra unneeded parameters). ChatGPT added some commands to save the EEG dataset in the ALLEEG dataset (something that is shown in EEGLAB history for completion but that is not really needed). However, when it came to using clean_rawdata, both models failed to use relevant parameters (the solution of using -1 for all parameters of ChatGPT means that the plugin does not modify the data (the call does not crash)). Note that the free version of ChatGPT (3.5) returned almost the same result as ChatGPT 4.0 (it omitted the parameters to the clean_rawdata and the iclabel function). By contrast, Bard uses parameters that do not exist (such as Threshold; WindowSize actually exists, although it is not a parameter you usually modify, and 200 seconds is way too long). The call to run ica is correct in both cases, and ChatGPT is capable of calling the IClabel function while Bard is unable to do so. Although the solution provided by ChatGPT is slightly better in this case, we have found that both models have similar performances.

In other words, while these models can offer relevant suggestions and a great starting point, they are not yet able to write functional EEGLAB scripts and understand the intricacies and parameters of complex functions. We will provide updates as these models continue to improve in the following years.

For reference, a functional script to process a dataset is available here.

What about general MATLAB reasoning to write code? Can LLMs help with that. We asked the following question, which we consider advanced MATLAB programming.

“Can you rewrite this MATLAB code without using a loop and without using the conv function?"

x = [0.2819 0.5386 0.6952 0.4991 0.5358 0.4452 0.1239 0.4904 0.8530 0.8739];
convval = [-0.4 3 -0.6];
y = zeros(1,10);

for i = 2:9
y(i) = sum(x(i-1:i+1).*convval(end:-1:1));
end

Below are Bard and ChatGPT (4) answers. Both solutions failed to execute based on matrix size inconsistency, although both solutions are very close to being correct (still, when provided with the MATLAB error and asked to fix the code, both LLMs provided updated code that also failed to execute).

Bard

ChatGPT

Although LLMs failed to implement complex reasoning for MATLAB programming, they are not far from being able to do so and can still be useful for inspiration.