Klaus Gramann
Research Interests - Mobile Brain/Body Imaging (MoBI)
MoBI in the news:
Discover Magazine
UC Newsroom
CalIT2 Newsroom
CalIT2 Newsroom
HMC Architects
Health Care IT News 2008
Health Care IT News 2009
Health Care Design
MoBI: Makeig, Gramann, Jung, Sejnowski, &
Poizner (2009). Linking Brain, Mind, and Behavior. International
Journal of Psychophysiology, 73(2), 95-100.
____________________________________________________________________________________________________________________________________________________________________________________________
This
research on high definition mobile EEG-recordings attempts
to overcome the traditional restrictions of brain imaging by applying
new developments in EEG-recording techniques and analyzes (Makeig et
al., 2002; Delorme & Makeig, 2004) to Natural Cognition,,
analyzing ambulatory recordings
of brain electrical activity
during actions in natural space with complex experimental designs.
Natural Cognition, including self-determined information
uptake and motor action to orient in space will be recorded in small
and large scale natural environments. Recordings of Natural
Cognition in small and large
scale space will inevitably be
confronted with complex motor behavior not encountered hitherto due
to the restrictions of classical brain imaging analyzes. More
specifically, subjects will move in space including whole body
movements as well as movements of the upper limbs and the head to
reach a goal or actively search for relevant information needed to
guide behavior. This kind of behavior includes the coordinated
activation of sets of muscles (i.e., muscles of the neck) directly
assessed by high-density EEG-recordings, introducing functional
non-brain activity associated with movement and accompanying
cognitive processes. Thus, with the aim of describing complex
cognitive processes during Natural
Cognition it is necessary
to identify and describe muscle activity accompanying Natural
Cognition in a first step.
See video recording of an experimental session using the
high-definition Mobile Brain/Body Imaging (MoBI)
technique.
The results of a first experiment demonstrate the potential of this new
imaging method. In this first experiment, subjects were required to
point to, look to, or walk and subsequently point to one out of 6
possible objects lovcated around the subject in a semicircular arrey
(see Figure 1).
Figure 1: Experimental setup of the first MoBI experiment.
Subjects were standing in a large room facing a computer screen at a
distance of approxemately 3.5 meters. Six obejcts were located in a
semicircular array around the subject. Data from EEG and motion capture
were recored und synchronized online via the DataRiver Software
(Vankov, 2009). When the subject pointed to the monitor or one of the
other objects, the vector going through the motion capture
sensors placed on the pointing finger and the forehead of the subject
was registered online relative to the 3-D coordinates of the object
locations. If the vector entered a hot spot defined around the objects,
a feedback was given and the trial terminated. With the next pointing
movement towards the central monitor the next trial was initiated. Each
trial was composed of several subsequent displayes, starting with a
fixation cross for 3 seconds, followed by the display of one of the six
objects for 1 second, with a subsequent fixation cross of 3
seconds, followed by tyhe indication of three possible actions to
perform (look to, point to, or walk to), which was followed by a third
fixations cross for 3 seconds. When this last fixation cross changed
its color from black to red, the subject was allowed to react. The
other half of the trials were identical with the action instruction
being displayed first and the object being displayed second.
The first and most important question was whether the new MoBI
approach makes it possible to record brain dynamics during active motor
behavior of the subject.
The answer is - YES.
However, to demonstrate that we record the same brain dynamics during
passive cognition we have to replicate earlier findings that were
recorded using the traditional EEG-approach. One example of a well
known EEG-phenomena is the motor-mu, or alpha-desynchronization over
motor cortex when subject prepare or imagine movements of the limb (described
by Pfurtscheller and Aranbiar, 1979). Analyzing the brain dynamics of
subjects during passive
cognition (while the subject is standing, looking at the central
display perceiving the instructions on the monitor) we can compare our
data to the data from traditional recordings. The results can be seen
in figure 2 below.
Figure 2:
Upper row displays a cluster of independent component processes
(small red balls) with the centroid of the cluster (big red ball)
located in or near the motor cortex (data from 8 subjects). The cluster
is shown from a horizontal, sagittal, and coronal view. The second row
displays the trial sequence as described above, starting with a
fixation cross. The last row displays mean cluster event-related
spectral perturbation in dB in log-frequency scale from 3 to 150 Hz,
revealing significant deviation from baseline (warm colors indicate
power increase, cold colors indicate power decrease) over the entire
time period of nine seconds and a short period of time thereafter.
As can be seen in Figure 2, the event-related spectral pertubation
pattern in motor cortex replicates the well known motor-mu
desynchronization. While the power decrease is absent during the first
fixation cross, power in the 10 Hz and first harminc frequency bands
starts to decrease (copmpared to baseline as derived from the first
second of the fixation cross) with onset of the first relevant
information screen. During the time period of the second fixation
cross, motor-mu desynchronization seems to oscillate and then decreases
further with onset of the second display. After this - which is the
point in time when subjects know where to and what to do - the
desynchronization reaches it's maximum. This pattern of brain dynamics
in motor cortex can be seen as replication of the traditional
experimental setting with a desynchronization of motor-mu activity
while subjects prepare voluntary movements.
Of course that still does not proof whether we can record and analyze
the brain dybnamics during
active movements of the subject.
Figure 3 below displays the brain dynamics of the same motor cortex
cluster with onset of and during execution of movements from 8
subjects. Subjects were looking or pointing (including natural looking)
to the 6 different
objects (the data shown is averaged across all objects) which included
rotations of the head and upper body to the left and right up to 80
degrees.
Figure 3: Displays the identical independent component cluster
as dexcibed in figure 2 above seen from horizontal, sagittal, and
coronal slices. The small inset in the second row displays the tril
structure of one trial. The next inset shows the last 500 msec of the
last fixation cross and the time period with onset of the red fixation
cross (time point 0 in the figure above) indicating the imperative
stimulus. The last row displays event-related spectral pertubation (in
dB) in log-frequency from 3 to 150 Hz.
The
results reveal that several brain areas demonstrate remarkable
sensitivity in their dynamics accompanying different phases of a
movement. In other words, timing of motor behavior is reflected in
brain dynamics.
____________________________________________________________________________________________________________________________________________________________________________________________
In
an earlier pilot-experiment testing the above described
assumptions by recording EEG-activity synchronized with the subjects’
movement during walking in a circular array (clockwise and
counter-clockwise) and subsequently pointing (while standing) to two
locations in the room (two cameras to the
left and right of the
subject). Data processing was related to i) half stride circle in the
walking condition with each cycle starting with movement-onset of one
foot and ii) onset of arm-movement during the pointing condition as
determined from the velocity profiles of sensors attached to the
lower tibia and the wrist, respectively. Continuous EEG-data from 64
channels was submitted to ICA and event-related spectral perturbation
(ERSPS) was computed for independent components representing muscle
activity. Figure 1 displays time warped ERSP, aligned to the onset of
foot movement, the maximum velocity of the movement, and heel strike
for the right and the left foot. In addition, significant differences
between ERSPs in both conditions were analysed by subtracting ERSPs
with onset of left foot movement from ERSPs with onset of right-foot
movement, and subsequent permutation statistics (p<0.001).
Figure
1. A) Posterior view scalp topography of IC 13 likely
reflecting m. splenius capitis and the respective event-related
spectral perturbation (ERSP) during the time course of half striding
circle for B) the right foot, C) the left foot,
and D) the difference ERSP. ERSP data was time warped to
distinct time points as revealed by the velocity pattern of sensors
attached to the right and left lower tibia with 1) onset of foot
movement indicated by 0 ms, 2) max. velocity of foot movement, 3)
strike of heel.
As
can be seen in Figure 1, the scalp topography of the depicted IC is
right-lateralized with a maximum current source density located
between the superior and inferior nuchal lines of the occipital
scull. This location is likely to reflect the attachment of the right
neck muscle splenius capitis to the right occipital skull. The ERSP
revealed significant decreases in a broad frequency range with onset
of foot movement until it’s maximum velocity. With right heel
strike, again a significant power decrease in a broad frequency range
was observed. In contrast, no comparable broad band activity was
present for movement of the left foot resulting in significant
differences in ERSP for the both conditions. But why should a muscle
of the head be associated with simple walking? During a striding
circle the upper torso describes a yawing, pitching, and vertical
translation due to the movement of the lower limps. This in turn
leads to a yaw, pitch, and vertical translation of the head in space.
This movement is compensated by movements of the head in the opposite
direction relative to the upper torso to support head stability and
gaze (Hirasaki et al., 1999; Imai et al, 2001). The described
activity therefore is likely reflecting muscle activity of one muscle
or a set of muscles of the right neck stabilizing the absolute
position of the head in space during the bipedal rhythm (perceptible
when both hands touch the neck during walking).
In
the second part of the experiment the subject had to point to two
distinct positions in space relative to the left and right. The
movement consisted of an upward movement of the head slightly
proceeding the upward movement of the pointing arm by a few
milliseconds to visually focus on the target. The data was analyzed
with respect to the velocity of sensors attached to the wrist of the
pointing arm (right arm). Figure 2 displays the location of two ICs
as revealed by the equivalent dipole model (A), the resultant scalp
topography (B), and the ERSP time warped to the onset, maximum
velocity, and offset of the upward movement, as well as the onset,
maximum, and offset of the downward pointing movement. The ERSP
display pointing movement to the right, the left, and the difference
in ERSPs by subtracting left movement from right pointing movements.
Figure
2: A) Location of two ICs as revealed by the equivalent dipole
model. B) Scalp topography of the depicted ICs. C) ERSP
time warped to the onset (first vertical line at time 0 ms), maximum
velocity (nd vertical line), and offset of the upward
movement (3rd vertical line), as well as the onset (4th
line), maximum (5th line), and offset (6th line) of the
downward pointing movement. The first column represents pointing
movement to the right, the second pointing movement to the left, and
the third figure depicts the resultant difference ERSP by subtracting
left movement from right pointing movements at p<0.001.
The
location of the dipole model as well as the scalp topography point to
muscles attached to the anterior part of the occipital skull at the
insertions to the mastoid process. There are three muscles attached
to the skull in this area: the Sternocleidomastoid, the Splenius
capitis, and the Longissimus capitis. All three muscles support head
flexion and rotation to the ipsilateral side. The scalp topography
shows the centroid of current source density slightly above the
superior nuchal line which is due to the low number of electrodes
with no sensors below the superior nuchal line at the lateral surface
of the head. The ERSP for the left IC (C upper image) reveals
slight power increases with onset of the movement to the right,
lasting to the maximum velocity of the upward movement. There is a
prolonged power increase in a relatively smaller frequency band
between 10 and 15 Hz. Finally a marked desynchronization following
the offset of the downward movement while the subject was pointing to
the right was observed. In contrast, the right lateralized IC
revealed significantly stronger broad band synchronization with onset
of the pointing movement to the right (C lower image). Again,
prolonged synchronization of the frequency range between 10 and 15 Hz
was observed. The desynchronization after movement offset was less
pronounced for right pointing movements. The inverse pattern of ERSP
is observed for pointing movements to the left (second column of the
ERSPs in C). Now the left lateralized IC revealed strong
synchronization in a broad band frequency range which was not
observed for the left-lateralized IC.
This
pattern reflects the time course of muscle activity of muscles
involved in an upward rotation of the head preceding the pointing
movement. The contraction of the muscles attached to the mastoid
process and the anterior occipital skull lead to a rotation and
upward movement of the head to the ispilateral side. This is in line
with significantly stronger power increases over a broad frequency
range for the right IC when the subjct pointed to the right and
thus, the head was moved to the right and vice verca for the left IC.
This
first experiment revealed muscle activity accompanying simple motor
behavior in 3-dimensional space. Such systematic muscle activity
could not be observed until now since restricted recording
environments forced the subject to sit or lie motionless during the
task. The results show the importance of describing muscle activity
during Natural Cognition in real environments. The analysis of
muscle activity during walking revealed how the neck musculature
stabilizes the head and gaze in space. More important, the
description of neck muscle activity during simple pointing behavior
revealed the systematic activation patterns that will be observed
during several cognitive tasks. For example, learning the location of
objects in space to build up an egocentric spatial representation
ultimately includes muscle activity of the neck. Proprioceptive
feedback from neck muscles is used for the computation of body
orientation in space (Bove et al., 2002). Further, proprioceptive
feedback from neck muscles is used for the computation of an
egocentric frame of reference with an impairment of the same, when
neck muscles are manipulated unilateral (Bottini et al., 2001). While
subjects orient to different objects in space, the head is tuned
towards the object resulting in neck muscle activity as one primary
source of information for computing an egocentric frame of reference.
Thus, the analyzes of neck muscle activity with respect to brain
activity will yield further insights into spatial learning and the
impact of proprioceptive feedback on the computation of an egocentric
frame of reference and the integration of spatial information in
higher order representations.
This
information should be used and muscles of the head have to be
integrated into a model for source localization in order to identify
and describe muscle activity accompanying Natural Cognition.
This way the complex interplay of motor behavior and cognition will
give further insight into the complex processes that take place
during every-day tasks.
