MatRiver software for real-time multimodal analysis and visualization
Recent performance gains in computer hardware and the introduction of multi-core CPUs have made it possible to process data in real-time using the Matlab scripting language and processing environment. The MatRiver software package is a set of Matlab functions that communicate with an online multimodal recording framework by reading and writing data streams to perform data analysis and visualization and/or produce or adjust a subject stimulus stream.
Download is not available yet although Beta testing is possible. Email email@example.com.
Ease of use
With MatRiver, users can leverage their existing knowledge of Matlab and its extensive mathematical and visualization capabilities. Existing Matlab scripts for EEG analysis and classification can be easily used in conjunction with MatRiver functions with minimal modifications. This greatly reduces the need for re-implementing the same code in different languages (for example, replicating operations originally written as Matlab scripts for EEG classification as a C program) for real-time implementation. This ease of use may allow easier exploratory development of more complex and powerful real-time analysis methods.
Fig. 1 above shows the relationship between current components of MatRiver system. Data can be read either from DataRiver system using a function now under modification, or a real-time session can be simulated by reading data from a local Matlab variable. In the simulated case, data samples are provided to mr_accumulate() function with the same pace as if they were produced with a specific sampling rate.
If data is already buffered on the local machine outside Matlab by the recording system, MatRiver system does not have to continuously check for newly arrived data. MatRiver can check the buffer through a function now under further development on regular (~0.1 s) intervals by exploiting timer objects and read all the newly arrived samples in one quick step. This feature is essential for real-time operations since it eliminates the chance of sample loss when Matlab process is busy.
Matlab timer objects execute a callback function or script in regular intervals and free up the CPU time between these runs. MatRiver functions are usually invoked by timer objects in specified intervals. At least one timer has to be executed during a real-time session to read the incoming data by running mr_accumulate() function. Other MatRiver functions can be executed by the same timer object or in other timer objects to achieve different execution rates. For example, one might decide to read data every 50 ms but visualize it in 100 ms intervals by invoking mr_view_data() in another timer object.
Data read from the online data collection system or simulated through mr_read_from_workspace() function is accumulated in a Matlab global variable mr.accumulatedData and made available to all MatRiver functions. Each of these functions acts on a subset of accumulated data related a specific modality (EEG data, Event data, Mocap data, …). For example, mr_raw_to_ic_activity() function calculates IC activity from EEG channel data or mr_accumulate_events() function read events from the event channel and accumulates them in mr.event field.
In addition to continuous execution by timer objects, MatRiver function can be invoked or triggered by events read from the event channel. This is achieved in mr_event_trigger() by defining triggering events and the duration of time after which a trigger callback function, for example a classifier, should be executed.
Matlab does not offer particularly fast image rendering or the most temporally accurate method of visual presentation. This said, our experiments show that a Matlab process running on the local machine equipped with a mid-level OpenGL capable graphics card can perform most visualization task relevant to EEG analysis with an acceptable performance.
In the example in Fig. 2 (below), near-real time changes in alpha power in a posterior component were visualized in three different ways. Processing delay was measured by the difference between the number of data samples in the accumulated data and the number of samples that should have been received at that latency, based on the time the session had started and the data sampling rate. MatRiver was executed in simulated real-time mode using a timer object to perform data acquisition (mr_accumulate()), calculating IC activity (mr_raw_to_ic_activity()) by vector multiplication, estimating alpha power, and visualizing the alpha power series. Delay was calculated after the visualization function (mr_record_lag()). The timer object was set to run its maximum (100 times per second). The actual rate at which timer was executed was dependent on the time it took for its MatRiver callback function to run.
|Condition||Median Delay (ms)|
|Calculating IC activity with no visualization||5.5|
|Visualizing alpha power by bar graph height (Fig. 2a); zbuffer renderer||52|
|Visualizing alpha power by ball height in Matlab VRML (Fig. 2b)||25|
|Visualizing alpha power by airplane height in Matlab VRML (Fig. 2c)||25|
For information about data read functions used by MatRiver is under construction ....
Initializes connection to local buffers of the recording system or sets up variables for simulated real-time.
reads real-time (or simulated real-time) data and accumulates it in the mr.accumulatedData field. To save memory and prevent slowdown due to disk-swapping, the length of data accumulated in mr.accumulatedData can be limited automatically in this function by setting the mr.maxNumberOfFramesInAccumulatedData field to a finite value.
reads data from mr.simulated.data field and provides it to mr_accumulate() to simulate a real-time session. Data is provided based on a realistic scenario in which only a limited number of frames are available (produced) based on how long the simulated session has been running.
reads events from an event channel and accumulates them in the mr.event field. Each event in mr.event structure has three fields: Type, Latency (in frames) and Triggered. The latter indicates whether the event has already triggered a callback function.
goes through events in mr.event and triggers callback functions for the event when necessary. Each trigger is defined by the types of events and the duration of time after these for which the callback has to be executed. For example, a trigger can launch a classifier after 800 ms has passed after events of type 1 or 2.
Triggers are placed in mr.trigger field.
stops and clears all timer objects.
calculate the activity of select ICs for a portion of data and places it in the mr.eeg.icActivity field. The advantage of this method is that the activity is calculated only when necessary, minimizing the CPU burden in case this information is requested infrequently (as in an infrequent trigger scenario). Also since filtering is performed on a portion of data, non-casual filtering can be used to minimize phase distortions (using the Matlab filtfilt() function). The execution time for this function increases linearly with the number of channels/components processed and the duration for which the activity is requested.
continuously calculates independent component (IC) activities and accumulates them in the mr.eeg.accumulatedIcActivity field, which is a circular buffer. Since only newly received data frames are processed, this requires less CPU time in continuous tasks. On the other hand, here ICA is calculated even when it is not used by any MatRiver function, so only casual filters (IR) can be used.
detects noisy (bad) channels and puts their indices in mr.eeg.badChannels field. It uses abs. correlation between each channel and all other channels and if this value falls below a threshold in the last ~200 EEG frames, the channel is flagged as being noisy.
updates ICA decomposition matrix to exclude noisy channels. It creates a new ICA matrix in mr.eeg.activeIcaUnmix field based on the original matrix in mr.eeg.originalIcaUnmix field.
shows a 'dashboard' which contains useful information regarding MatRiver such as execution-lags and timer status. It can also, Start, Pause and Terminate MatRiver main timer.
visualizes the real-time data flowing into the MatRiver system.
records the execution lag. This is the difference between the number of data frames currently in the accumulated data and the number of frames that should have been received based on the time the session started and the data sampling rate. These lags are recorded in the mr.executionLag field (in seconds) which can be used to monitor the performance of the system.
Circular Buffer Functions
Circular buffers contain an internal buffer field which contains data in a linear format (wrapped-around).Please use functions below to place and access data in circular buffers.
creates a circular buffer structure. These structures are used for accumulating calculated values in MatRiver, for example IC activity is accumulated in mr.eeg.accumulatedIcActivity field by mr_raw_to_ic_activity_accumulate() function. Click here for more information about circular buffers.
adds values to a circular buffer structure.
maps linear array indices to indices in circular buffer. Maximum linear index number is equal to the capacity of the buffer.
returns the number of valid (placed-in) frames in the buffer.
returns linear array (internal buffer field) indices for the last N frames placed in the buffer.
returns last N frames of data placed in the buffer.
Read and Write to data stream functions
MatRiver by default reads data from 'a buffer of recording system. Additional read or write buffers can be defined in MatRiver to send and receive data and commands in streams. All buffers should be properly closed to prevent problem in the recording system. If handles for read or write buffers are left open, the system may run out of new handles and stop working.
creates a reading buffer. Additional read buffers receive data from data streams.
creates a writing buffer. Additional write buffers send data to data streams.
closes a reading buffer and deletes all received data and associated book-keeping data in MatRiver.
closes a writing buffer and deletes all associated book-keeping data in MatRiver.
writes data into the (named) buffer.
reads data of a certain (named) buffer and accumulates it into a circular buffer.
this function is is similar to circular_buffer_add() but uses global mr variable to significantly (5x) speed-up access to buffer values.
Miscellaneous functions and scripts
runs MatRiver in simulated real-time mode.
shows power in a given frequency range on the MNI cortical image.
performs an eegOcean visualization (N. Bigdely–Shamlo, Tim Mullen) in 2-D.
performs an eegOcean visualization on the MNI cortical surface.
visualizes alpha power by changes in the height of a 3-D ball (or airplane) in VRML under Matlab.
visualizes alpha power by modulating the height of bar in a Matlab figure.