Consciousness studies: Neuroscience 2

Binocular Rivalry, Pattern Rivalry and Binocular Fusion
Sir Charles Wheatstone (1838) was the first scientist to systematically investigate binocular rivalry. Binocular rivalry occurs when different images are presented to the left and right eyes. The subject sees successively one image, a combined image and then the other image. The swapping of images can take a second or more. Binocular rivalry is of interest in consciousness research because the parts of the brain that contain the dominant image should also be those parts that are contributing to conscious experience. Binocular rivalry involves at least two components; the first switches from one image to a merged image and then to the other image and the second permits the view to be part of conscious experience.



The switching of one image for another may involve selecting one of the images as the percept or selecting one of the eyes. Blake et al (1979) performed an experiment in which subjects could change the image at a given eye by pressing a button. When a particular image became dominant they pressed a button to change the image at the eye receiving the dominant image for the non-dominant image. They found that the subjects immediately experienced the second image as the dominant image. This suggests that binocular rivalry is selecting between eyes rather than images. Lehky in 1988 proposed that the switching may be occurring as a result of feedback between visual cortical area V1 and the Lateral Geniculate Nucleus (a thalamic relay - see Carandini et al 2002) and Blake in 1989 also proposed that the switching occurred at the level of area V1. (Visual cortical area V1 receives visual input direct from the LGN.)

Tong (2001) has argued that, in humans, the switching of images in binocular rivalry may occur at the earliest levels in the visual cortex. In particular, Tong and Engel (2001) used an elegant technique measuring the activity in the visual cortex that represents the blind spot of the eye to show that almost complete switching to the dominant image occurs at the level of visual cortical area V1. In support of this idea of switching at the level of V1 or even before the cortex, Kreimann et al (2001, 2002) used direct electrode recordings in human cortex and found that the activity of most neurons changed with the percept. Other experiments have not shown a single locus in the brain where the suppressed sensory information gets switched out (Blake & Logothetis 2002, Leopold & Logothetis 1996, Gail et al., 2004).

Functional MRI has also shown cortical activity outside of sensory visual cortex related to both images in binocular rivalry. Lumer et al (1998) found that only the fronto-parietal areas of cortex switched with the percept, Fang & He (2005) found that activity relating to both suppressed and unsuppressed images were present in the dorsal stream of the visual system. Wunderlich et al (2005) and Haynes et al (2005) have both found suppression at the level of the lateral geniculate nucleus using fMRI in humans.

Pasley et al (2004) have shown that, even during suppression, fearful faces can produce activity in the amygdala (see Pessoa (2005) for a review).

Rivalry alternations seem to be the result of widespread activity changes that cover large parts of the brain, including but not necessarily originating at the earliest sensory stages of visual processing. Most investigators have found that, once switching has occurred, there are areas of the brain that contain activity that is solely related to the percept but this varies from most of the cortex to largely more frontal regions depending upon the study. The most likely explanation for binocular rivalry is that the switching occurs at the level of the LGN as a result of feedback from the cortex.

Pattern Rivalry is also of interest in consciousness research for the same reasons as binocular rivalry. In pattern rivalry a figure may have two or more forms that replace each other. Typical examples of such figures are the Necker cube and Rubin's face-vase. The similarity of the time course of the switching between percepts in binocular rivalry and pattern rivalry has led many authors to suggest that these involve the same mechanism. Logothetis et al (1996) used novel dichoptic stimuli (different images to each eyes) to produce a form of rivalry that seems to involve switching at levels in the cerebral cortex that are more distal to the sensory stimulus than V1. Leopold and Logothetis (1999), on the basis of their work with monkeys, state that "..many neurons throughout the visual system, both monocular and binocular, continue to respond to a stimulus even when it is perceptually suppressed.". Kleinschmidt et al (1998) investigated pattern rivalry with MRI and found activity in higher order visual areas during change of dominant pattern. Pettigrew (2001) also describes effects on rivalry due to thought and mood that may require involvement of large areas of cortex in the switching operation and stresses the way that V1 represents different visual fields in different hemispheres of the brain so that inter-hemispheric switching must also be considered.

It seems likely that the change of dominant pattern or percept is associated with higher level cortical activity but once the dominant percept is established many of the visually responsive neurons in the cortex are switched over to the new percept. This might account for the similarities in timing of binocular and pattern rivalry and the disparate results found by the various groups of authors. In the words of Kleinschmidt et al (1998):

"The transient activity fluctuations we found suggest that perceptual metastability elicited by ambiguous stimuli is associated with rapid redistributions of neural activity between separate specialized cortical and subcortical structures."

Which permits both the idea of selecting particular eyes or percepts, perhaps by feedback that switches a thalamic relay on the basis of cortical processing of patterns. Once the cortex has switched the thalamic relay most of the neurons in V1 would become exposed to the dominant percept but there would still be a few neurons in the cortical visual system receiving data from the non dominant image.

The investigations of binocular and pattern rivalry provide evidence that conscious visual experience is probably distal to V1 (ie: cortex or thalamus).

Perceptual rivalry may be part of complex decision making rather than being simply a switch to blank out unwelcome input. It is clear from the Rubin face-vase that pattern rivalry is linked to recognition and would involve a complex delineation of forms within cortical processing. This would suggest that many areas of cortex should be involved before a particular percept is made dominant. Pettigrew (2001) argues that rivalry is the result of a complex phenomenon rather than being simply a switching event. Pettigrew's discovery that laughter abolishes rivalry also points to a complex cortical system for switching percepts. Pettigrew proposes that complex cortical processes control rivalry and that the actual switching of percepts is performed sub-cortically in the Ventral Tegmental Area. He concludes his review of the problem by noting that "Rivalry may thus reflect fundamental aspects of perceptual decision making.." Pettigrew (2001).

Another effect, known as "binocular fusion", provides further compelling evidence for the non-conscious nature of the cerebral cortex. In binocular fusion images from both eyes are fused together to create a single image in experience. Moutoussis and Zeki (2002) used a form of binocular fusion in which images of faces were flashed at 100ms intervals to both eyes simultaneously. When both eyes received images of the same colour the subject could see the faces but when one eye received a green image on a red background and the other a red image on a green background the subjects reported seeing a uniform yellow field that contained no faces.

fMRI scans of the subject's brains showed that when both eyes were exposed to images of the same colour the part of the brain that deals with faces was active and when each eye received images of different colours the same areas of brain showed activity. In other words the cortex contained strong activity related to faces whether or not faces were experienced. Moutoussis and Zeki found a similar effect when they used images of houses instead of images of faces. The authors concluded that: "The present study further suggests that there are no separate processing and perceptual areas but rather that the same cortical regions are involved in both the processing and, when certain levels of activation are reached and probably in combination with the activation of other areas as well, the generation of a conscious visual percept".

This conclusion does not seem to be supported by the data. There is no evidence that any area of cortex contains the percept itself. The experiment shows that the cortex contains data relating to both red and green faces which suggests that the cortex is not the site of the conscious percept. The percept is most likely distal to the cortex in the thalamus.

It is interesting that Fries et al (1997) found that neurons that were activated by the dominant image in binocular rivalry fired synchronously whereas those that were activated by the non-dominant image did not. Thalamocorticothalamic oscillations are the most likely source for synchronising neurons over whole areas of cortex, again suggesting that the conscious percept is located in the thalamus rather than the cortex.

Synchronisation of Neural Processes
Our experience seems to contain entities with their attributes attached to them at the correct places in space and time. When a dog barks we see its jaws open at the same time as the bark and both jaws and bark are at the same location. We take this for granted but the brain must be engaging in some complex processing to achieve this synchronised and appropriately positioned set of objects and events. The illustration below shows the two basic processes that might be used to synchronise events between the different specialised processors in the cerebral cortex and brain in general.



In the first option a complete model of sensation, dream etc. may be created and then allowed to become part of conscious experience. In the second model events are released into experience as fast as possible but are synchronous when recalled, having been synchronised in a storage buffer. There is a third option in which there is no synchronisation of events so that the output from different processors would occur at different times.

The 'experience buffer' would be a volume of brain in which a succession of events could be recorded. The buffer might either be updated in steps, the previous content being discarded, or continuously updated with the oldest content being lost continuously.

In the first option events from different processes would always appear to be simultaneous unless the experience buffer were updated as a series of steps in which case any changes at around the moment of updating might appear in successive buffers. For instance, if change of position were processed before change in colour a circle on a screen that changed from green to red at the start of a motion might seem to be briefly green during the motion and then turn red.

In the second model events from different processors might appear asynchronous at the moment of experience but synchronous when recalled.

Colour vision and motion vision are processed in different parts of the visual cortex and in distinct parts of visual cortical areas V1 and V2. They are different processes and hence ideal for studying the synchronisation of cortical activity. Moutoussis and Zeki (1997) presented subjects with moving coloured squares on a computer screen that changed from red to green or vice versa as they changed direction of movement. It was found that subjects seemed to perceive changes in colour some 70-80 msecs before they perceived a change in the direction of motion of the squares. Further work by Arnold et al (2001) and Arnold and Clifford (2001) have confirmed that colour changes seem to be perceived before motion. Arnold and Clifford (2001) also found a quantitative relationship between the colour/motion asynchrony and the direction of change of motion, complete reversals of direction giving rise to the greatest asynchrony between the detection of colour and motion changes.

Moutoussis and Zeki (1997) conclude by stating that the asynchrony of neural processes shows that "..the perception of each attribute is solely the result of the activity in the specialised system involved in its processing..". It seems more likely that the experiments simply show that slow neural processes are not synchronised before they become percepts (the third option above). The experiments are excellent evidence for the concept of the cortex as a set of specialised processors that deliver their output asynchronously to some other place where the output becomes a percept.

These experiments on colour and motion suggest that there is no synchronisation between the processes that deal with these two aspects of vision. Another set of experiments by Clifford et al (2003) supports this idea of processing being asynchronous. They asked subjects to perform a variety of judgements of when visual events occurred and found that the degree of synchrony of one visual event with another depends on the type of judgement. Different judgements probably use processors in different areas of cortex and the output from these arrives asynchronously at the part of the brain that supports the percept.

When the percept is formed there must be feedback to the cortical processes that create its content. Otherwise it would not be possible to report about the percept and the cortex would be unable to direct processing to the percept in preference to other, non-conscious cortical data.

Although slow processes (20 milliseconds to 1 second) do not seem to be synchronised there is some evidence for very rapid synchronisation. Andrews et al (1996) revisited a problem raised by the famous physiologist Charles Sherrington. Sherrington considered the phenomenon of 'flicker fusion' in which a flickering light appears to be a continuous steady light if it flashes on and off at frequencies of about 45 Hz or higher. He reasoned that if the images from both eyes are brought together to form a single image then the frequency at which a flickering light appears to be steady should depend on whether one or two eyes are used. Flicker fusion should occur if each eye receives alternate flashes at only half the normal flicker fusion frequency. The flicker should disappear if the left eye receives flashes at 23 pulses per second and the right eye receives alternate flashes at 23 pulses per second. When Sherrington performed the experiment he found that this was not the case, using approximate figures, each eye required 46 pulses per second for fusion to occur. Sherrington proposed that the flicker fusion in alternate binocular presentation was occurring "psychically", outside of normal physiological processes.

Andrews et al duplicated Sherrington's result but investigated it further. They found that when lights were flashed in each eye alternately at low frequences (2 Hz) the experience was the same as a light being flashed in both eyes at this rate. At frequencies of four Hz and higher the subjects began to report that the lights being flashed alternately in both eyes seemed to flicker at the same rate as lights being flashed in both eyes at half the frequency. It seemed as if a flash in one eye followed by a flash in the other eye was being perceived as a single flash or "conflated" as the authors put it. The authors explained this effect by suggesting that the brain activity corresponding to the flashes was sampled for a short period and any number of flashes occurring during this period became perceived as a single flash. The maximum rate of sampling would be about 45 Hz. This idea is similar to option (1) above, where the buffer is filled and emptied 40 - 50 times a second.

An experience buffer that is refreshed at 40-50 times a second might also explain the results obtained with colour and motion asynchrony because synchronisation between processes may well happen too quickly to affect processes that occur at very slow rates. Singer and Gray (1995), Singer (2001) have proposed that synchronisation between neurones at about 45Hz is the discriminator between those neurones with activity that contributes to conscious experience and activity in other neurones. A rapid refresh rate in a sychronising buffer agrees with the results found by Fries et al (1997) in which visual cortical neurones that represent a percept underwent synchronous oscillations in the gamma frequency range (39-63 Hz). Tononi et al (1998) have also found synchronisation of neural activity in neurones that represent the percept.

The gamma frequency oscillations are intrinsic to the cortex but are triggered by the thalamus and are part of the 'arousal system'. Readers should be wary of the term 'arousal system' because it evokes the idea of something waking up a conscious cortex. The cortex can be fully active during sleep and even during pathological unconsciousness such as persistent vegetative state so it is possible that the arousal centres themselves or nearby structures actually host phenomenal consciousness.

EEG and synchronisation
If electrodes are placed on the scalp varying electrical potentials of a few tens of microvolts can be recorded between the electrodes. Recordings of potentials from electrodes on the scalp are known as electroencephalograms (EEGs).

The potentials recorded in the EEG are due to postsynaptic potentials in nerve cells. The EEG is insensitive to the activity of single cells and occurs as a result of relatively slow, synchronised, changes in large areas of cells. The differences in potential between two scalp electrodes are largely due to depolarisation and hyperpolarisation of the dendritic trees of cortical pyramidal cells. The folding of the cortex (gyri) is problematical for recording and interpreting EEGs because opposing layers of cortex can cancel any net potentials.

The EEG shows rhythmic activity. This is conventionally divided into the following frequency bands:

Delta waves 0-4 Hz

Theta waves 4-8 Hz

Alpha waves 8-12 Hz

Beta waves >10 Hz

Gamma waves (also called fast beta) 25-100 Hz

EEGs also contain short bursts of activity called spindles and very fast oscillations (VFOs). Spindles last for 1-2 seconds and contain rhythmic activity at 7-14 Hz. They are associated with the onset of sleep. The VFOs consist of short bursts at frequencies of over 80 Hz.

When the eyes are closed the amplitude of activity from most pairs of electrodes is increased compared with when the eyes are open. When subjects are awake the EEG consists mainly of alpha and beta activity with considerable low amplitude gamma when the eyes are open. In stage 1 sleep the EEG consists of theta waves, in stage 2 sleep of varied activity and spindles, in stage 4 sleep of delta and during REM sleep of beta and theta activity. In epileptic seizures there tends to be high amplitude activity with pronounced synchronisation between many pairs of electrodes.

The rhythmic electrical activity is due to cortical feeback loops, cortico-cortical synchronisation, thalamic pacemakers and thalamo-cortical synchronisation. VFOs have been attributed to the activity of electrical connections between cells (dendro-dendritic gap junctions) (Traub (2003)).

The gamma activity, centred on a frequency of 40 Hz appears to be related to activity in cortical interneurons that form electrical connections between their dendrites (Tamas et al 2000). These oscillations can be triggered by high frequency stimulation of the posterior intralaminar nuclei of the thalamus (Barth and MacDonald 1996, Sukov and Barth 2001)  and as a result of activation of the reticular system (Munk et al 1996). This suggests that stimulation of cortex by thalamic sensory relays triggers gamma band activity in the cortex. A shift from gamma to beta waves can occur in human event related potentials after about 0.2 secs (Pantev 1995, Traub et al 1999).

The alpha activity is related to thalamic pacemakers, perhaps as a result of intrinsic oscillatory activity in thalamic sensory relays (see Roy & Prichep 2005 for a brief review). Theta activity, which occurs during some cognitive tasks and mental arithmetic involves a loop from the cortex to the non-specific thalamic nuclei. Delta activity seems to be endogenous to cortex when input is suppressed during sleep. Beta activity is due to cortico-cortical interactions, often after a brief period of gamma activation. It should be noted that gamma and beta activity can be expressed as impulses in cortico-thalamic pathways and that when cortical and thalamic activity is correlated there is a conscious state. In other words gamma or beta waves in the cortex are not correlates of consciousness on their own - see for instance Laureys et al (2002).

Event related potentials
After a sudden event there are a characteristic set of changes in EEG activity known as event related potentials or ERPs. The time course of the ERP is shown in the diagram below.



ERPs occur in response to novel stimuli and are also produced by brief transcranial magnetic stimulation (TMS)(Iramina et al 2002). The slow component is known as the P3 or P300 phase of the ERP. It is due to activation of areas of the brain that are relatively remote from the primary sensory areas of brain.

Nieuwenhuis et al (2005) have reviewed the origin of the P300 ERP: "To summarize, convergent evidence suggests that P3-like activity can be recorded in several, widely separated brain areas. These include some medial temporal and subcortical structures (e.g., the hippocampal formation, amygdala, and thalamus), but these structures are unlikely to contribute directly to the scalp-recorded P3.". According to Nieuwenhuis et al (2005) the recorded P300 may be due to temporo-parietal and prefrontal cortical activity. Linden (2005) has also concluded that widespread, but specific, cortical activation is correlated with the recorded P300 ERP.

The generator of the P300 is still obscure. Nieuwenhuis et al (2005) consider that the Locus coeruleus, a nucleus in the pons that regulates task related attention and part of the sleep-wake cycle, may be responsible. In line with this, Mashour et al (2005) have discovered that TMS induced P300 activity is reduced in unconscious states.

Whether the P300 is related to Libet's 0.5 second delay is still obscure but the discovery that the P300 occurs in association with subliminal stimuli (stimuli that do not enter awareness)(Bernat et al 2001) suggests that it is associated with non-conscious cortical processing. Williams et al (2004), in an investigation of subliminal and supraliminal fear perception, found that "conscious fear perception was distinguished by a more prominent N4, peaking around 400 msec"; the N4 component follows the P300 component in the succession of phases of the ERP. Williams et al considered that the earlier phases in the ERP are probably related to non-conscious processing. In contrast Vogel et al (1998) found that suppression of the P300 was associated with suppression of awareness.

Global Workspace Theory
Global Workspace Theory is the idea that somewhere in the brain there is a facility that integrates the processes that occur in the various separate areas of the brain. The theory was first proposed by Descartes as the sensus communis, the common sense, but the modern form of the theory dispenses with the idea of a point soul looking at the brain. In modern Global Workspace theory it is proposed that an area of brain receives input from most of the cerebral cortex and provides integrative output that can solve problems across sensory and pre-motor modalities.

Modern Global Workspace Theory has been championed by Baars (1983, 1988).

There is increasing evidence for a Global Workspace or Global Workspaces in the brain. Much of this evidence comes from fMRI, single unit and magnetoencephalography studies in which it is shown that non-conscious or subliminal processing mainly occupies primary, sensory cortex whereas conscious processing occupies large areas of cerbral cortex.

In binocular rivalry the stimulus that is consciously perceived is responsible for relatively intense activation of large areas of brain whereas the non-conscious stimulus is often suppressed (see above and Sheinberg & Logothetis (1997), Tononi et al (1998)). The suppression is likely to occur in the Lateral Geniculate Nuclei which suggests a role for the Thalamic Reticular Nuclei, which modulate LGN activity, in the control of the percept.

Masking and visual awareness
Word masking has also been used to investigate the idea of a Global Workspace. When a word is presented on its own for a few tens of milliseconds it remains readable but if it is immediately succeeded by, or accompanied by, another word it becomes indistinct or invisible. This effect is known as "word masking". Vogel et al (1998) have investigated a version of word masking known as the "attentional blink". They found that when stimuli became invisible the P3 component of the Event Related Potential, which peaks at around 300-500 millisecs after a stimulus, was completely suppressed. The P3 component of the ERP has been related to the lodging of data in working memory and also to gamma band activity in the EEG. This strongly suggests the involvement of a cortico-thalamic loop in the "attentional blink". The delay of 0.3 to 0.5 secs is typical of the time required for conscious awareness (see above).

Word masking in conjunction with fMRI and Event Related Potential (ERP) recordings has been used by Dehaene et al (2001) to expose control by a central mechanism. It was found that masked words activate mainly the visual cortex and ventral stream (inferior temporal lobe) whereas visible words also activated distant parietal, prefrontal and cingulate sites.

Dehaene et al (2003) and found that the dynamics of the loss of visibility of words in an attentional blink experiment could be modelled by a simulated cortico-thalamic loop. In their simulation a distributed cortical process determined which events would receive attention and the system used the thalamic gating systems to exclude those that did not receive attention.

Tse et al (2005) have used purely visual stimuli in masking experiments and concluded that, in the case of purely visual stimuli, the neural correlates of awareness were limited to the occipital cortex:

"We suggest that there are both lower and upper bounds within the visual hierarchy for the processing of visual masking and the maintenance of visual awareness of simple unattended targets; the lower bound is at least as high as the border between V2 and V3, and the upper bound is within the occipital lobe, possibly somewhere downstream of V4."

This discovery would mean that activation of large areas of cortex are unnecessary for awareness.

Attention and the global workspace
Baars (2002) in his review of evidence for the Global Workspace Theory quotes many other experiments that show activation of larger areas of cortex in response to conscious stimuli compared with unconscious or subliminal stimuli.The effect is also seen in change blindness, learning and attention. Newman and Baars (1993) consider that the "workspace" is fairly global in the brain:

"This Neural Global Workspace (NGW) model views conscious processes in terms of a globally integrative brain system. The neural circuitry contributing to this system is not only widely distributed across the neocortex, but includes key corticothalamic and midbrain circuits as well. These cortico-subcortical circuits are hypothesized to be critical to understanding the mechanisms of attentional control that provide an essential basis for the conscious processing of information".

However they focus particularly on the role of the thalamic Reticular Nucleus and cortico-thalamic connectivity in the control of attention.

Other ideas for the location of the Global Workspace are the idea of Singer et al. that gamma synchrony controls access to the content of consciousness and Llinas et al. (1998) that the thalamus is the hub through which communication occurs between areas of cortex.

One of the problems with Global Workspace theory is that it suggests that attention, working memory, cognitive control and consciousness may all be in the same area of the brain. It is likely that the mechanisms of attention, working memory, and cognitive control may involve several, interlinked systems perhaps co-opting the basal ganglia in the process. In view of this Maia and Cleeremans (2005) propose that ".. attention, working memory, cognitive control and consciousness are not distinct functions implemented by separate brain systems. Attempting to find separate neural correlates for each may therefore be the wrong approach. Instead, we suggest that they should be understood in terms of the dynamics of global competition, with biasing from PFC (prefrontal cortex).". The inclusion by Maia and Cleeremans of consciousness with distributed attention, working memory and cognitive control is reminiscent of Zeki & Bartel's idea of microconsciousness.

It should be noted that, in common with Libet's data, the percept seems to be available to phenomenal consciousness some 0.3 to 0.5 secs after a stimulus; this suggests that whatever determines the content of phenomenal consciousness operates before events become part of phenomenal consciousness. This relegates phenomenal consciousness from being a controller of attention to being the recipient of content that is the subject of attention. This finding is consistent with the philosophical problem of the apparently epiphenomenal nature of phenomenal consciousness.

Given the data on the timing of conscious awareness it seems that there may be two "workspaces", an active workspace that models the world, discarding and suppressing data during rivalry, and a passive workspace that receives the final, edited product. The active workspace would correlate with the cortical systems stressed by Dehaene et al and Maia and Cleermans although, given the results of Tse et al., the workspace would be limited to small zones of cortex. The loading of the passive workspace with the output of the active workspace would correlate with thalamo-cortical activity during component P3 of the ERP in which data is transferred from the cortex to the thalamus. This workspace might constitute the source for reports of the content of phenomenal consciousness.

Llinas et al (1998) have proposed two parallel cortico-thalamic attentional systems, one of which is related to the thalamic specific nuclei and the other to the thalamic non-specific nuclei, especially the ILN. The non-specific system would be related to consciousness itself.