Reconciling the Roles of Orbitofrontal Cortex in Reversal Learning and the Encoding of Outcome Expectancies
GEOFFREY SCHOENBAUM,
MICHAEL P. SADDORIS,
THOMAS A. STALNAKER,
GEOFFREY SCHOENBAUM
Departments of Anatomy and Neurobiology and Psychiatry, University of Maryland School of Medicine, Baltimore, Maryland 21201, USA
Department of Psychology, University of Maryland Baltimore County, Baltimore, Maryland 21228, USA
Search for more papers by this authorMICHAEL P. SADDORIS
Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, Maryland 21218, USA
Search for more papers by this authorTHOMAS A. STALNAKER
Departments of Anatomy and Neurobiology and Psychiatry, University of Maryland School of Medicine, Baltimore, Maryland 21201, USA
Search for more papers by this authorGEOFFREY SCHOENBAUM,
MICHAEL P. SADDORIS,
THOMAS A. STALNAKER,
GEOFFREY SCHOENBAUM
Departments of Anatomy and Neurobiology and Psychiatry, University of Maryland School of Medicine, Baltimore, Maryland 21201, USA
Department of Psychology, University of Maryland Baltimore County, Baltimore, Maryland 21228, USA
Search for more papers by this authorMICHAEL P. SADDORIS
Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, Maryland 21218, USA
Search for more papers by this authorTHOMAS A. STALNAKER
Departments of Anatomy and Neurobiology and Psychiatry, University of Maryland School of Medicine, Baltimore, Maryland 21201, USA
Search for more papers by this authorAddress for correspondence: Geoffrey Schoenbaum, M.D., Ph.D., 20 Penn Street HSF-2, Rm S251, Baltimore, MD 21201. Voice: 410-706-3814; fax: 410-706-2512. [email protected]
Abstract
Abstract: Damage to orbitofrontal cortex (OFC) has long been associated with decision-making deficits. Such deficits are epitomized by impairments in reversal learning. Historically, reversal learning deficits have been linked to a response inhibition function or to the rapid reversal of associative encoding in OFC neurons. However here we will suggest that OFC supports reversal learning not because its encoding is particularly flexible—indeed it actually is not—but rather because output from OFC is critical for flexible associative encoding downstream in basolateral amygdala (ABL). Consistent with this argument, we will show that reversal performance is actually inversely related to the flexibility of associative encoding in OFC (i.e., the better the reversal performance, the less flexible the encoding). Further, we will demonstrate that associative correlates in ABL are more flexible during reversal learning than in OFC, become less flexible after damage to OFC, and are required for the expression of the reversal deficit caused by OFC lesions. We will propose that OFC facilitates associative flexibility in downstream regions, such as ABL, for the same reason that it is critical for outcome-guided behavior in a variety of setting—namely that processing in OFC signals the value of expected outcomes. In addition to their role in guiding behavior, these outcome expectancies permit the rapid recognition of unexpected outcomes, thereby driving new learning.
REFERENCES
- 1
Jones, B. &
M. Mishkin. 1972. Limbic lesions and the problem of stimulus-reinforcement associations.
Exp. Neurol.
36: 362–377.
- 2
Mishkin, M.
1964. Perseveration of central sets after frontal lesions in monkeys.
In
The Frontal Granular Cortex and Behavior. J.M. Warren &
K. Akert, Eds.: 219–241. McGraw-Hill.
New York
.
- 3
Schoenbaum, G.
et al.
2002. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations.
Neuroreport
13: 885–890.
- 4
Bohn, I.,
C. Giertler &
W. Hauber. 2003. Orbital prefrontal cortex and guidance of instrumental behavior in rats under reversal conditions.
Behav. Brain Res.
143: 49–56.
- 5
Dias, R.,
T.W. Robbins &
A.C. Roberts. 1996. Dissociation in prefrontal cortex of affective and attentional shifts.
Nature
380: 69–72.
- 6
Schoenbaum, G.
et al.
2003. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals.
Learn. Mem.
10: 129–140.
- 7
Rolls, E.T.
et al.
1994. Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage.
J. Neurol. Neurosurg. Psychiatry
57: 1518–1524.
- 8
Izquierdo, A.D.,
R.K. Suda &
E.A. Murray. 2004. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency.
J. Neurosci.
24: 7540–7548.
- 9
Meunier, M.,
J. Bachevalier &
M. Mishkin. 1997. Effects of orbital frontal and anterior cingulate lesions on object and spatial memory in rhesus monkeys.
Neuropsychologia
35: 999–1015.
- 10
Chudasama, Y. &
T.W. Robbins. 2003. Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex.
J. Neurosci.
23: 8771–8780.
- 11
Kim, J. &
K.E. Ragozzino. 2005. The involvement of the orbitofrontal cortex in learning under changing task contingencies.
Neurobiol. Learn. Mem.
83: 125–133.
- 12
Brown, V.J. &
K. McAlonan. 2003. Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat.
Behav. Brain Res.
146: 97–130.
- 13
Teitelbaum, H.
1964. A comparison of effects of orbitofrontal and hippocampal lesions upon discrimination learning and reversal in the cat.
Exp. Neurol.
9: 452–462.
- 14
Gallagher, M.,
R.W. McMahan &
G. Schoenbaum. 1999. Orbitofrontal cortex and representation of incentive value in associative learning.
J. Neurosci.
19: 6610–6614.
- 15
Pickens, C.L.
et al.
2003. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task.
J. Neurosci.
23: 11078–11084.
- 16
Ostlund, S.B. &
B.W. Balleine. 2007. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental learning.
J. Neurosci.
27: 4819–4825.
- 17
Chudasama, Y.,
J.D. Kralik &
E.A. Murray. 2006. Rhesus monkeys with orbital prefrontal cortex lesions can learn to inhibit prepotent responses in the reversed reward contingency task.
Cereb. Cortex
17: 1154–1159.
- 18
Schoenbaum, G.,
A.A. Chiba &
M. Gallagher. 1999. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning.
J. Neurosci.
19: 1876–1884.
- 19
Thorpe, S.J.,
E.T. Rolls &
S. Maddison. 1983. The orbitofrontal cortex: neuronal activity in the behaving monkey.
Exp. Brain Res.
49: 93–115.
- 20
Rolls, E.T.
et al.
1996. Orbitofrontal cortex neurons: role in olfactory and visual association learning.
J. Neurophysiol.
75: 1970–1981.
- 21
Critchley, H.D. &
E.T. Rolls. 1996. Olfactory neuronal responses in the primate orbitofrontal cortex: analysis in an olfactory discrimination task.
J. Neurophysiol.
75: 1659–1672.
- 22
Stalnaker, T.A.
et al.
2006. Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making.
Euro. J. Neurosci.
24: 2643–2653.
- 23
Nobre, A.C.
et al.
1999. Orbitofrontal cortex is activated during breaches of expectation in tasks of visual attention.
Nat. Neurosci.
2: 11–12.
- 24
Tobler, P.N.
et al.
2006. Human neural learning depends on reward prediction errors in the blocking paradigm.
J. Neurophysiol.
95: 301–310.
- 25
Feierstein, C.E.
et al.
2006. Representation of spatial goals in rat orbitofrontal cortex.
Neuron
51: 495–507.
- 26
Krettek, J.E. &
J.L. Price. 1977. Projections from the amygdaloid complex to the cerebral cortex and thalamus in the rat and cat.
J. Comp. Neurol.
172: 225–254.
- 27
Kita, H. &
S.T. Kitai. 1990. Amygdaloid projections to the frontal cortex and the striatum in the rat.
J. Comp. Neurol.
298: 40–49.
- 28
Shi, C.J. &
M.D. Cassell. 1998. Cortical, thalamic, and amygdaloid connections of the anterior and posterior insular cortices.
J. Comp. Neurol.
399: 440–468.
- 29
Weiskrantz, L.
1956. Behavioral changes associated with ablations of the amygdaloid complex in monkeys.
J. Comp. Physiol. Psychology
9: 381–391.
- 30
Davis, M.
2000. The role of the amygdala in conditioned and unconditioned fear and anxiety.
In
The Amygdala: a Functional Analysis. J.P. Aggleton, Ed.: 213–287. Oxford University Press.
Oxford
.
- 31
Gallagher, M.
2000. The amygdala and associative learning.
In
The Amygdala: a Functional Analysis. J.P. Aggleton, Ed.: 311–330. Oxford University Press.
Oxford
.
- 32
Everitt, B.J.
et al.
2000. Differential involvement of amygdala subsystems in appetitive conditioning and drug addiction.
In
The Amygdala: a Functional Analysis. J.P. Aggleton, Ed.: 353–390. Oxford University Press.
New York
.
- 33
Quirk, G.J.,
J.L. Armony &
J.E. LeDoux. 1997. Fear conditioning enhances different temporal components of tone-evoked spike trains in auditory cortex and lateral amygdala.
Neuron
19: 613–624.
- 34
Saddoris, M.P.,
M. Gallagher &
G. Schoenbaum. 2005. Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex.
Neuron
46: 321–331.
- 35
Quirk, G.J.,
J.C. Repa &
J.E. LeDoux. 1995. Fear conditioning enhances short-latency auditory responses of lateral amygdala neurons: parallel recordings in the freely behaving rat.
Neuron
15: 1029–1039.
- 36
Stalnaker, T.A.
et al.
2007. Cocaine-induced decision-making deficits are mediated by miscoding in basolateral amygdala.
Nat. Neurosci.
10: 949–951.
- 37
Patton, J.J.
et al.
2006. The primate amygdala represents the positive and negative value of visual stimuli during learning.
Nature
439: 865–870.
- 38
Schoenbaum, G.,
A.A. Chiba &
M. Gallagher. 1998. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning.
Nat. Neurosci.
1: 155–159.
- 39
Padoa-Schioppa, C. &
J.A. Assad. 2006. Neurons in orbitofrontal cortex encode economic value.
Nature
441: 223–226.
- 40
Schoenbaum, G.
et al.
2003. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala.
Neuron
39: 855–867.
- 41
Schoenbaum, G.
et al.
2006. Encoding changes in orbitofrontal cortex in reversal-impaired aged rats.
J. Neurophysiol.
95: 1509–1517.
- 42
Hikosaka, K. &
M. Watanabe. 2000. Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards.
Cereb. Cortex
10: 263–271.
- 43
Roesch, M.R.,
A.R. Taylor &
G. Schoenbaum. 2006. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation.
Neuron
51: 509–520.
- 44
Hikosaka, K. &
M. Watanabe. 2004. Long- and short-range reward expectancy in the primate orbitofrontal cortex.
Eur. J. Neurosci.
19: 1046–1054.
- 45
Tremblay, L. &
W. Schultz. 2000. Reward-related neuronal activity during go-no go task performance in primate orbitofrontal cortex.
J. Neurophysiol.
83: 1864–1876.
- 46
Roesch, M.R. &
C.R. Olson. 2004. Neuronal activity related to reward value and motivation in primate frontal cortex.
Science
304: 307–310.
- 47
O'Doherty, J.
et al.
2002. Neural responses during anticipation of a primary taste reward.
Neuron
33: 815–826.
- 48
Gottfried, J.A.,
J. O'Doherty &
R.J. Dolan. 2003. Encoding predictive reward value in human amygdala and orbitofrontal cortex.
Science
301: 1104–1107.
- 49
Roesch, M.R. &
C.R. Olson. 2005. Neuronal activity in primate orbitofrontal cortex reflects the value of time.
J. Neurophysiol.
94: 2457–2471.
- 50
Tremblay, L. &
W. Schultz. 1999. Relative reward preference in primate orbitofrontal cortex.
Nature
398: 704–708.
- 51
Blair, K.
et al.
2006. Choosing the lesser of two evils, the better of two goods: specifying the roles of ventromedial prefrontal cortex and dorsal anterior cingulate in object choice.
J. Neurosci.
26: 11379–11386.
- 52
Calu, D.J.,
M.R. Roesch &
G. Schoenbaum. 2007. Orbitofrontal cortex does not signal reward prediction errors.
Society for Neuroscience Abstracts. 749.16.
- 53
Frank, M.J. &
E.D. Claus. 2006. Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal.
Psychol. Rev.
113: 300–326.
- 54
Schoenbaum, G.,
M.R. Roesch &
T.A. Stalnaker. 2006. Orbitofrontal cortex, decision-making, and drug addiction.
Trends Neurosci.
29: 116–124.
- 55
Maren, S. &
K.A. Goosens. 2001. Contextual and auditory fear conditioning are mediated by the lateral, basal, and central amygdaloid nuclei in rats.
Learn. Mem.
8: 148–155.
- 56
Amorapanth, P.,
J.E. Ledoux &
M.A. Nader. 2000. Different lateral amygdala outputs mediate reactions and actions elicited by a fear-arousing stimulus.
Nat. Neurosci.
3: 74–79.
- 57
LeDoux, J.E.
et al.
1990. The lateral amygdaloid nucleus: sensory interface of the amygdala in fear conditioning.
J. Neurosci.
10: 1062–1069.
- 58
Morgan, M.M. &
J.E. Ledoux. 1999. Contribution of ventrolateral prefrontal cortex to the acquisition and extinction of conditioned fear in rats.
Neurobiol. Learn. Mem.
72: 244–251.
- 59
Petrovich, G.D.
et al.
2002. Amygdalo-hypothalamic circuit allows learned cues to override satiety and promote eating.
J. Neurosci.
22: 8748–8753.
- 60
Holland, P.C. &
M. Gallagher. 2003. Double dissociation of the effects of lesions of basolateral and central amygdala on conditioned stimulus-potentiated feeding and Pavlovian-instrumental transfer.
E. J. Neurosci.
17: 1680–1694.
- 61
McDannald, M.A.
et al.
2005. Lesions of orbitofrontal cortex impair rats' differential outcome expectancy learning but not conditioned stimulus-potentiated feeding.
J. Neurosci.
25: 4626–4632.
- 62
Stalnaker, T.A.
et al.
2007. Basolateral amygdala lesions abolish orbitofrontal-dependent reversal impairments.
Neuron
54: 51–58.
- 63
Baxter, M.G. &
E.A. Murray. 2002. The amygdala and reward.
Nat. Rev. Neurosci.
3: 563–573.
