Basal Ganglia Mechanisms of Reward-Oriented Eye Movement
OKIHIDE HIKOSAKA
Laboratory of Sensorimotor Research, National Eye Institute, National Institute of Health, Bethesda, Maryland, USA
Search for more papers by this authorOKIHIDE HIKOSAKA
Laboratory of Sensorimotor Research, National Eye Institute, National Institute of Health, Bethesda, Maryland, USA
Search for more papers by this authorAbstract
Abstract: Expectation of reward facilitates motor behaviors that enable the animal to approach a location in space where the reward is expected. It is now known that the same expectation of reward profoundly modifies sensory, motor, and cognitive information processing in the brain. However, it is still unclear which brain regions are responsible for causing the reward-approaching behavior. One candidate is the dorsal striatum where cortical and dopaminergic inputs converge. We tested this hypothesis by injecting dopamine antagonists into the caudate nucleus (CD) while the monkey was performing a saccade task with a position-dependent asymmetric reward schedule. We previously had shown that: (1) serial GABAergic connections from the CD to the superior colliculus (SC) via the substantia nigra pars reticulata (SNr) exert powerful control over the initiation of saccadic eye movement and (2) these GABAergic neurons encode target position and are strongly influenced by expected reward, while dopaminergic neurons in the substantia nigra pars compacta (SNc) encode only reward-related information. Before injections of dopamine antagonists the latencies of saccades to a given target were shorter when the saccades were followed by a large reward than when they were followed by a small reward. After injections of dopamine D1 receptor antagonist the reward-dependent latency bias became smaller. This was due to an increase in saccade latency on large-reward trials. After injections of D2 antagonist the latency bias became larger, largely due to an increase in saccade latency on small-reward trials. These results indicate that: (1) dopamine-dependent information processing in the CD is necessary for the reward-dependent modulation of saccadic eye movement and (2) D1 and D2 receptors play differential roles depending on the positive and negative reward outcomes.
REFERENCES
- 1 Inoue, M. et al. 1985. Reward related neuronal activity in monkey dorsolateral prefrontal cortex during feeding behavior. Brain Res. 326: 307–312.
- 2 Watanabe, M. 1996. Reward expectancy in primate prefrontal neurons. Nature 382: 629–632.
- 3 Leon, M.I. & M.N. Shadlen. 1999. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24: 415–425.
- 4 Kobayashi, S. et al. 2002. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87: 1488–1498.
- 5 Watanabe, M. et al. 2002. Coding and monitoring of motivational context in the primate prefrontal cortex. J. Neurosci. 22: 2391–2400.
- 6 Barraclough, D.J., M.L. Conroy & D. Lee. 2004. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7: 404–410.
- 7 Platt, M.L. & P.W. Glimcher. 1999. Neural correlates of decision variables in parietal cortex. Nature 400: 233–238.
- 8 Glimcher, P.W. 2001. Making choices: the neurophysiology of visual-saccadic decision making. Trends Neurosci. 4: 654–659.
- 9 Sugrue, L.P., G.S. Corrado & W.T. Newsome. 2004. Matching behavior and the representation of value in the parietal cortex. Science 304: 1782–1787.
- 10 Roesch, M.R. & C.R. Olson. 2003. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90: 1766–1789.
- 11 Roesch, M.R. & C.R. Olson. 2004. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304: 307–310.
- 12 Kawagoe, R., Y. Takikawa & O. Hikosaka. 1998. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1: 411–416.
- 13 Tremblay, L., J.R. Hollerman & W. Schultz. 1998. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J. Neurophysiol. 80: 964–977.
- 14 Hollerman, J.R., L. Tremblay & W. Schultz. 1998. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J. Neurophysiol. 80: 947–963.
- 15 Takikawa, Y., R. Kawagoe & O. Hikosaka. 2002. Reward-dependent spatial selectivity of anticipatory activity in monkey caudate neurons. J. Neurophysiol. 87: 508–515.
- 16 Lauwereyns, J. et al. 2002. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33: 463–473.
- 17 Lauwereyns, J. et al. 2002. A neural correlate of response bias in monkey caudate nucleus. Nature 418: 413–417.
- 18 Watanabe, K., J. Lauwereyns & O. Hikosaka. 2003. Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J. Neurosci. 23: 10052–10057.
- 19 Cromwell, H.C. & W. Schultz. 2003. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J. Neurophysiol. 89: 2823–2838.
- 20 Dayan, P. & B. Balleine. 2002. Reward, motivation, and reinforcement learning. Neuron 36: 285–298.
- 21 Baxter, M.G. & E.A. Murray. 2002. The amygdala and reward. Nat. Rev. Neurosci. 3: 563–573.
- 22 Maunsell, J.H. 2004. Neuronal representations of cognitive state: reward or attention? Trends Cogn. Sci. 8: 261–265.
- 23 Delgado, M.R. et al. 2000. Tracking the hemodynamic responses to reward and punishment in the striatum. J. Neurophysiol. 84: 3072–3077.
- 24 Knutson, B. et al. 2000. FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 12: 20–27.
- 25 O'Doherty, J. et al. 2001. Representation of pleasant and aversive taste in the human brain. J. Neurophysiol. 85: 1315–1321.
- 26 Denny-Brown, D. 1962. The Basal Ganglia and Their Relation to Disorders of Movement. Oxford University Press. London .
- 27
Graybiel, A.M. &
C.W. Ragsdale. 1979. Fiber connections of the basal ganglia.
In
: Development of Chemical Specificity of Neurons. M. Cuenod,
G.W. Kreutzberg &
F.E. Bloom, Eds.: 239–283. Elsevier.
Amsterdam
.
10.1016/S0079-6123(08)61309-6 Google Scholar
- 28 DeLong, M.R. & A.P. Georgopoulos. 1981. Motor functions of the basal ganglia. In : The Nervous System. Vol. sect.1, part 2, vol. II, chapt. 19. V.B. Brooks, Ed.: 1017–1061. American Physiological Society. Bethesda , MD .
- 29 Mogenson, G.J., D.L. Jones & C.Y. Yim. 1980. From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14: 69–97.
- 30 Haber, S.N. & N.R. McFarland. 1999. The concept of the ventral striatum in nonhuman primates. Ann. N. Y. Acad. Sci. 877: 33–48.
- 31 Swanson, L.W. 2000. Cerebral hemisphere regulation of motivated behavior. Brain Res. 886: 113–164.
- 32 Fudge, J.L. & S.N. Haber. 2000. The central nucleus of the amygdala projection to dopamine subpopulations in primates. Neuroscience 97: 479–494.
- 33 Schultz, W. 1998. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80: 1–27.
- 34 Hikosaka, O. et al. 1993. Role of basal ganglia in initiation and suppression of saccadic eye movements. In : Role of the Cerebellum and Basal Ganglia in Voluntary Movement. N. Mano, I. Hamada & M.R. Delong, Eds.: 213–219. Elsevier. Amsterdam .
- 35 Mink, J.W. 1996. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50: 381–425.
- 36 Salmon, D.P. & N. Butters. 1995. Neurobiology of skill and habit learning. Curr. Opin. Neurobiol. 5: 184–190.
- 37 Graybiel, A.M. 1998. The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70: 119–136.
- 38 Packard, M.G. & B.J. Knowlton. 2002. Learning and memory functions of the Basal Ganglia. Annu. Rev. Neurosci. 25: 563–593.
- 39 Alexander, G.E., M.R. DeLong & P.L. Strick. 1986. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9: 357–381.
- 40 Hikosaka, O., Y. Takikawa & R. Kawagoe. 2000. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80: 953–978.
- 41 Sparks, D.L. 2002. The brainstem control of saccadic eye movements. Nat. Rev. Neurosci. 3: 952–964.
- 42 Hayhoe, M. & D. Ballard. 2005. Eye movements in natural behavior. Trends Cogn. Sci. 9: 188–194.
- 43
Ewert, J.-P.
1980. Neuroethology. Springer.
Berlin
.
10.1007/978-3-642-67500-3 Google Scholar
- 44 Johansson, R.S. et al. 2001. Eye-hand coordination in object manipulation. J. Neurosci. 21: 6917–6932.
- 45 Triesch, J. et al. 2003. What you see is what you need. J. Vis. 3: 86–94.
- 46 Miyashita, K. et al. 1996. Anticipatory saccades in sequential procedural learning in monkeys. J. Neurophysiol. 76: 1361–1366.
- 47 Miyashita, N., O. Hikosaka & M. Kato. 1995. Visual hemineglect induced by unilateral striatal dopamine dificiency in monkeys. Neuroreport 6: 1257–1260.
- 48 Hikosaka, O. & R.H. Wurtz. 1983. Visual and oculomotor functions of monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J. Neurophysiol. 49: 1230–1253.
- 49 Joseph, J.P. & D. Boussaoud. 1985. Role of the cat substantia nigra pars reticulata in eye and head movements. I. Neural activity. Exp. Brain Res. 57: 286–296.
- 50 Handel, A. & P.W. Glimcher. 1999. Quantitative analysis of substantia nigra pars reticulata activity during a visually guided saccade task. J. Neurophysiol. 82: 3458–3475.
- 51 Hikosaka, O., M. Sakamoto & S. Usui. 1989. Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J. Neurophysiol. 61: 780–798.
- 52 Matsumura, M. et al. 1992. Visual and oculomotor functions of monkey subthalamic nucleus. J. Neurophysiol. 67: 1615–1632.
- 53 Kato, M. & O. Hikosaka. 1995. Function of the indirect pathway in the basal ganglia oculomotor system: visuo-oculomotor activities of external pallidum neurons. In : Age-Related Dopamine-Deficient Disorders. Vol. 14. M. Segawa & Y. Nomura, Eds.: 178–187. Karger. Basal .
- 54 Parthasarathy, H.B., J.D. Schall & A.M. Graybiel. 1992. Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J. Neurosci. 12: 4468–4488.
- 55 Hikosaka, O. & R.H. Wurtz. 1983. Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus. J. Neurophysiol. 49: 1285–1301.
- 56 Kita, H. 1993. GABAergic circuits of the striatum. In : Chemical Signalling in the Basal Ganglia. G.W. Arbuthnott & P.C. Emson, Eds.: 51–72. Elsevier. Amsterdam .
- 57 Nambu, A., H. Tokuno & M. Takada. 2002. Functional significance of the cortico-subthalamo-pallidal ‘hyperdirect’ pathway. Neurosci. Res. 43: 111–117.
- 58
Ungerstedt, U.
1971. Stereotaxic mapping of the monoamine pathways in the rat brain.
Acta Physiologica Scandinavica
Suppl 367: 1–48.
10.1111/j.1365-201X.1971.tb10998.x Google Scholar
- 59 Hikosaka, O. & R.H. Wurtz. 1985. Modification of saccadic eye movements by GABA-related substances. II. Effects of muscimol in monkey substantia nigra pars reticulata. J. Neurophysiol. 53: 292–308.
- 60 Boussaoud, D. & J.P. Joseph. 1985. Role of the cat substantia nigra pars reticulata in eye and head movements. II. Effects of local pharmacological injections. Exp. Brain Res. 57: 297–304.
- 61 Sakamoto, M. & O. Hikosaka. 1989. Eye movements induced by microinjection of GABA agonist in the rat substantia nigra pars reticulata. Neurosci. Res. 6: 216–233.
- 62 Hikosaka, O. & R.H. Wurtz. 1989. The basal ganglia. In : The Neurobiology of Saccadic Eye Movements. Vol. 3. R.H. Wurtz & M.E. Goldberg, Eds.: 257–281. Elsevier. Amsterdam .
- 63 Hikosaka, O., M. Sakamoto & N. Miyashita. 1993. Effects of caudate nucleus stimulation on substantia nigra cell activity in monkey. Exp. Brain Res. 95: 457–472.
- 64 Karabelas, A.B. & A.K. Moschovakis. 1985. Nigral inhibitory termination on efferent neurons of the superior colliculus: an intracellular horseradish peroxidase study in the cat. J. Comp. Neurol. 239: 309–329.
- 65 Sommer, M.A. & R.H. Wurtz. 2000. Composition and topographic organization of signals sent from the frontal eye field to the superior colliculus. J. Neurophysiol. 83: 1979–2001.
- 66 Tehovnik, E.J. et al. 2000. Eye fields in the frontal lobes of primates. Brain Res. Rev. 32: 413–448.
- 67 Paré, M. & R.H. Wurtz. 2001. Progression in neuronal processing for saccadic eye movements from parietal cortex area LIP to superior colliculus. J. Neurophysiol. 85: 2545–2562.
- 68 Hikosaka, O., M. Sakamoto & S. Usui. 1989. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J. Neurophysiol. 61: 814–832.
- 69 Carr, G.D. & N.M. White. 1983. Conditioned place preference from intra-accumbens but not intra-caudate amphetamine injections. Life Sci. 33: 2551–2557.
- 70 Watanabe, K. & O. Hikosaka. 2005. Immediate changes in anticipatory activity of caudate neurons associated with reversal of position-reward contingency. J. Neurophysiol. 94: 1879–1887.
- 71 Hikosaka, O., M. Sakamoto & S. Usui. 1989. Functional properties of monkey caudate neurons. II. Visual and auditory responses. J. Neurophysiol. 61: 799–813.
- 72 Sato, M. & O. Hikosaka. 2002. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J. Neurosci. 22: 2363–2373.
- 73 Amador, N., M. Schlag-Rey & J. Schlag. 2000. Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J. Neurophysiol. 84: 2166–2170.
- 74 Stuphorn, V., T.L. Taylor & J.D. Schall. 2000. Performance monitoring by the supplementary eye field. Nature 408: 857–860.
- 75 Middleton, F.A. & P.L. Strick. 2000. Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res. Rev. 31: 236–250.
- 76 Pasupathy, A. & E.K. Miller. 2005. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433: 873–876.
- 77 Schultz, W., P. Apicella & T. Ljungberg. 1993. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13: 900–913.
- 78 Mirenowicz, J. & W. Schultz. 1994. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72: 1024–1027.
- 79 Hollerman, J.R. & W. Schultz. 1998. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1: 304–309.
- 80 Rescorla, R.A. & A.R. Wagner. 1972. A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In : Classical Conditioning II: Current Research and Theory. A.H. Black & W.F. Prokasy, Eds.: 64–99. Appleton-Century-Croft. New York .
- 81 Barto, A.G. 1995. Adaptive critics and the basal ganglia. In : Models of Information Processing in the Basal Ganglia. J.C. Houk, J.L. Davis & D.G. Beiser, Eds.: 215–232. MIT Press. Cambridge , MA .
- 82 Houk, J.C., J.L. Adams & A. Barto. 1995. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In : Models of Information Processing in the Basal Ganglia. J.C. Houk, J.L. Davis & D.G. Beiser, Eds.: 249–270. MIT Press. Cambridge , MA .
- 83 Schultz, W. & A. Dickinson. 2000. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23: 473–500.
- 84 Kawagoe, R., Y. Takikawa & O. Hikosaka. 2004. Reward-predicting activity of dopamine and caudate neurons – a possible mechanism of motivational control of saccadic eye movement. J. Neurophysiol. 91: 1013–1024.
- 85 Tobler, P.N., A. Dickinson & W. Schultz. 2003. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J. Neurosci. 23: 10402–10410.
- 86 Takikawa, Y., R. Kawagoe & O. Hikosaka. 2004. A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping. J. Neurophysiol. 92: 2520–2529.
- 87 Nakamura, K. & O. Hikosaka. 2006. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci. 26: 5360–5369.
- 88 Gerfen, C.R. et al. 1990. D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250: 1429–1432.
- 89 Surmeier, D.J., W.-J. Song & Z. Yan. 1996. Coordinated expression of dopamine receptors in neostriatal medium spiny neurons. J. Neurosci. 16: 6579–6591.
- 90 Gonon, F. 1997. Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. J. Neurosci. 17: 5972–5978.
- 91 West, A.R. & A.A. Grace. 2002. Opposite influences of endogenous dopamine D1 and D2 receptor activation on activity states and electrophysiological properties of striatal neurons: studies combining in vivo intracellular recordings and reverse microdialysis. J. Neurosci. 22: 294–304.
- 92 Richfield, E.K., J.B. Penney & A.B. Young. 1989. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience 30: 767–777.
- 93 Calabresi, P. et al. 1996. The corticostriatal projection: from synaptic plasticity to dysfunctions of the basal ganglia. Trends Neurosci. 19: 19–24.
- 94 Reynolds, J.N. & J.R. Wickens. 2002. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15: 507–521.
- 95 Lovinger, D.M., J.G. Partridge & K.C. Tang. 2003. Plastic control of striatal glutamatergic transmission by ensemble actions of several neurotransmitters and targets for drugs of abuse. Ann. N. Y. Acad. Sci. 1003: 226–240.
- 96 Mahon, S., J.M. Deniau & S. Charpier. 2004. Corticostriatal plasticity: life after the depression. Trends Neurosci. 27: 460–467.
- 97 Nakahara, H. et al. 2004. Dopamine neurons can represent context-dependent prediction error. Neuron 41: 269–280.
- 98 Haber, S.N., J.L. Fudge & N.R. McFarland. 2000. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20: 2369–2382.
- 99 Iribe, Y. et al. 1999. Subthalamic stimulation-induced synaptic responses in substantia nigra pars compacta dopaminergic neurons in vitro. J. Neurophysiol. 82: 925–933.
- 100 Kobayashi, Y. et al. 2002. Contribution of pedunculopontine tegmental nucleus neurons to performance of visually guided saccade tasks in monkeys. J. Neurophysiol. 88: 715–731.
- 101 Dommett, E. et al. 2005. How visual stimuli activate dopaminergic neurons at short latency. Science 307: 1476–1479.
- 102 Sutherland, R.J. 1982. The dorsal diencephalic conduction system: a review of the anatomy and functions of the habenular complex. Neurosci. Biobehav. Rev. 6: 1–13.