Klaus Gramann Research Interests - Mobile Brain/Body Imaging (MoBI)

MoBI in the news:

Discover Magazine
UC Newsroom
CalIT2 Newsroom
CalIT2 Newsroom
HMC Architects
Health Care IT News 2008
Health Care IT News 2009
Health Care Design

MoBI Publications:

Gwin, J.T., Gramann, K., Makeig, S., & Ferris, D.P. (1011). Electrocortical activity is coupled to gait cycle phase during treadmill walking.
NeuroImage, 54, 1289-1296.

Gwin, J.T., Gramann, K., Makeig, S., & Ferris, D.P. (2010). Removal of movement artifact from high-density EEG recorded during walking and running.
Journal of Neurophyiology, 103, 3526-3534.

Gramann, K., Gwin, J.T., Bigdely-Shamlo, N., Ferris, D.P., & Makeig, S. (2010). Visual evoked responses during standing and walking.
Frontiers in Human Neuroscience, 4:202.

Makeig, Gramann, Jung, Sejnowski, & Poizner (2009). Linking Brain, Mind, and Behavior.
International Journal of Psychophysiology, 73(2), 95-100.

Gwin, J.T., Ferris, Makeig, S., & Gramann, K. (2010). Imaging the mobile brain. Poster at the Society for Neuroscience Conference 2010.

MoBI - A New Method to Investigate Active Human Cognition

  research on high definition mobile EEG-recordings attempts to overcome the traditional restrictions of brain imaging by applying new developments in EEG-recording techniques and analyzes approaches (Makeig et al., 2002; Delorme & Makeig, 2004) to Natural Cognition, analyzing ambulatory recordings of brain electrical activity during actions in natural space with complex experimental designs.

Natural Embodied Cognition,
including self-determined information uptake and motor action to orient in space is recorded in small and large scale environments. Recordings of active behavior will inevitably be confronted with eye- and muscle activity considered artifactual. Our analyzes have to deal with complex motor behavior not encountered hitherto due to the restrictions of classical brain imaging analyzes. More specifically, subjects move in space including whole body movements as well as movements of the upper limbs and the head to reach a goal or actively search for relevant information needed to guide behavior. This kind of behavior includes the coordinated activation of sets of muscles (i.e., muscles of the neck) directly assessed by high-density EEG-recordings, introducing functional non-brain activity associated with movement and accompanying cognitive processes. See video recording of an experimental session using the high-definition Mobile Brain/Body Imaging (MoBI) technique.

The first and most important question is whether the new MoBI allows recording of brain dynamics during active motor behavior. The answer is - YES.

To prove this point we conducted a feasebility study (
Visual evoked responses during standing and walking) and had participants stand, walk, and run on a treadmill while attending to a visual oddball paradigm. In this study we demonstrated that we can replicate the visual P300 associated with processing of target-stimuli while subjects acitvely move on a treadmill. ICA was able to decompose brain and non-brain activity without advanced a priori artifact rejection methods while subjects walked with speeds up to 1.3 m/s. Even when subjects were running on a treadmill, we were able to use ICA to analyze the brain dynamics after the raw data was artifact rejected using approaches similar to the rejection of fMRI artifacts in parallel EEG-fMRI recordings (see Removal of movement artifact from high-density EEG recorded during walking and running). Here is a link to a video showing a participant running on a treadmill at Dan Ferris' Human Neuromechanic Laboratory, University of Michigan. The general setup and the results from ICA are explained below.

Setup Treadmill

Figure 1:  (A) Experimental setup: subject standing on the dual-belt treadmill facing the LCD display. Components of the experimental setup are highlighted and described in the linked text boxes. (B) Equivalent-dipole locations of independent component (IC) processes (small spheres) and IC cluster centroids (large spheres) projected on horizontal, sagittal, and coronal
views of the standard MNI brain. (Yellow) Neck-muscle ICs; (gray) eye-movement ICs; (other colors) brain-based ICs (C) (Scalp maps) Mean projections to the scalp of the indicated brain-based IC clusters. Labels give the index (Cls #), number of subjects (Ss), and number of independent components (# ICs) for each cluster.

Comparing the P300 for target and non-target trials for participants standing, slow walking, and fast walking we demonstrated that it is feasible to record and analyze brain dynamics while subjects avctively move.

Figure 5 Treadmill

Figure 2: Grand-average ERPs following ICA-based artifact removal in the standing, slow walking, and fast walking conditions. Middle-row traces show ERP time courses at electrode Pz (red, target ERPs; blue, non-target ERPs). Scalp maps show the grand-average ERP scalp distributions at 100, 150, and 400 ms after onsets of target stimuli (upper row) and non-target stimuli (lower row). White dots indicate the location of electrode Pz. Note the scalp map similarities across movement conditions.

With this first study we have demonstrated the feasibility of MoBI
studies of event-related EEG dynamics in subjects performing full-body movements in a 3-D environment. Future MoBI studies will address critical questions concerning macroscopic brain dynamic patterns supporting motivated motor behavior and more general aspects of embodied cognition. Answers to many questions that were formerly not possible to investigate using brain imaging may now be approached. For example: How are eye movements, head movements and brain activity accompanying attentional orienting interrelated? What are the accompanying brain activity of spatial cognitive processes while subjects experience natural vestibular and proprioceptive feedback associated with heading changes during navigation? New information available from detailed analysis of concurrently recorded EEG and body motion data, an imaging modality we refer to as MoBI, should open new avenues for analyzing the association of brain dynamics with specific aspects of movement and motivated action.

New Insights into Human Cognitive Neuroscience

The results of a second experiment demonstrate the potential of this new imaging method. In this experiment, subjects were required to point to, look to, or walk and subsequently point to one out of 6 possible objects lovcated around the subject in a semicircular arrey (see Figure 1).

MoBI Experimental Setup

Figure 3: Experimental setup of the first MoBI experiment. Subjects were standing in a large room facing a computer screen at a distance of approxemately 3.5 meters. Six obejcts were located in a semicircular array around the subject. Data from EEG and motion capture were recored und synchronized online via the DataRiver Software (Vankov, 2009). When the subject pointed to the monitor or one of the other objects,  the vector going through the motion capture sensors placed on the pointing finger and the forehead of the subject was registered online relative to the 3-D coordinates of the object locations. If the vector entered a hot spot defined around the objects, a feedback was given and the trial terminated. With the next pointing movement towards the central monitor the next trial was initiated. Each trial was composed of several subsequent displayes, starting with a fixation cross for 3 seconds, followed by the display of one of the six objects  for 1 second, with a subsequent fixation cross of 3 seconds, followed by tyhe indication of three possible actions to perform (look to, point to, or walk to), which was followed by a third fixations cross for 3 seconds. When this last fixation cross changed its color from black to red, the subject was allowed to react. The other half of the trials were identical with the action instruction being displayed first and the object being displayed second.

To demonstrate that we record the same brain dynamics during passive cognition we have to replicate earlier findings that were recorded using traditional EEG-approaches. One example of a well known EEG-phenomena is the motor-mu, or alpha-desynchronization over motor cortex when subject prepare or imagine movements of the limb (described by Pfurtscheller and Aranbiar, 1979). Analyzing the brain dynamics of subjects during passive cognition (while the subject is standing, looking at the central display perceiving the instructions on the monitor) we can compare our data to the data from traditional recordings. The results can be seen in figure 2 below.

MoBI Left  Motor Cortex 1

Figure 4:
Upper row displays a cluster of independent component processes (small red balls) with the centroid of the cluster (big red ball) located in or near the motor cortex (data from 8 subjects). The cluster is shown from a horizontal, sagittal, and coronal view. The second row displays the trial sequence as described above, starting with a fixation cross. The last row displays mean cluster event-related spectral perturbation in dB in log-frequency scale from 3 to 150 Hz, revealing significant deviation from baseline (warm colors indicate power increase, cold colors indicate power decrease) over the entire time period of nine seconds and a short period of time thereafter.

As can be seen in Figure 2, the event-related spectral pertubation pattern in motor cortex replicates the well known motor-mu desynchronization. While the power decrease is absent during the first fixation cross, power in the 10 Hz and first harminc frequency bands starts to decrease (copmpared to baseline as derived from the first second of the fixation cross) with onset of the first relevant information screen. During the time period of the second fixation cross, motor-mu desynchronization seems to oscillate and then decreases further with onset of the second display. After this - which is the point in time when subjects know where to and what to do - the desynchronization reaches it's maximum. This pattern of brain dynamics in motor cortex can be seen as replication of the traditional experimental setting with a desynchronization of motor-mu activity while subjects prepare voluntary movements.

Of course that still does not proof whether we can record and analyze the brain dybnamics during active movements of the subject.

Figure 3 below displays the brain dynamics of the same motor cortex cluster with onset of and during execution of movements from 8 subjects. Subjects were looking or pointing (including natural looking) to
the 6 different objects (the data shown is averaged across all objects) which included rotations of the head and upper body to the left and right up to 80 degrees.

MoBI Left Motor Cortex 2

Figure 5: Displays the identical independent component cluster as dexcibed in figure  2 above seen from horizontal, sagittal, and coronal slices. The small inset in the second row displays the tril structure of one trial. The next inset shows the last 500 msec of the last fixation cross and the time period with onset of the red fixation cross (time point 0 in the figure above) indicating the imperative stimulus. The last row displays event-related spectral pertubation (in dB) in log-frequency from 3 to 150 Hz.

The results reveal that several brain areas demonstrate remarkable sensitivity in their dynamics accompanying different phases of a movement. In other words, timing of motor behavior is reflected in brain dynamics.


Department Psychology, Ludwig-Maximilians-University Munich
Swartz Center for Computational Neuroscience, UCSD