Massachusetts Institute of Technology
Sign in | Create Account

Computational Models of Basal Ganglia Function

05/07/2009 1:30 PM 46"3002
Kenji Doya, Okinawa Institute of Science and Technology

Description: As a mathematical engineer, Kenji Doya approaches the goal of describing the most intricate brain mechanisms from a computational perspective. He constructs models of reinforcement learning involving the networked structures of the basal ganglia. His efforts are captured and expressed quantitatively as probabilities, regressions, and algorithms.

In this presentation, Doya covers basic concepts of reinforcement learning, then surveys the last decade of inquiry into the components of the basal ganglia circuit governing voluntary motion. Among the topics: action values, action candidates, and reward prediction involving the neurotransmitter dopamine; model"free versus model"based learning strategies; and the essential role of serotonin as modulator in the complex information loop.

Doya's recent research is carried out via robots he calls "cyber rodents." His dream as an undergraduate was to "build a robot that learns the variety of behaviors on its own." That is, the computer, not the human engineer, teaches the robot to move. He accomplished this in designing a machine"creature exhibiting emotion"like attributes characterized as "depression," "impulsivity," "greed," and "patience."

Doya believes the "metaparameters" of reinforcement learning must be "tuned appropriatelyOtherwise the performance of your learning is very, very poor." The iterative process involves three terms -- the reward itself, the expected reward for a new state based on choice of action, and memory of the reward gained in the previous state. In the comparison, any differential greater than zero can be exploited for learning. The tradeoff: "No pain, no gain."

As research advanced to increasing levels of structural specificity, Doya posited that "there seems to be spatial segregation in the function" of basal ganglia components. Specialization in aspects of reinforcement learning is now seen, for instance, in ventral versus dorsal areas of the striatum.

Differentiation is also found in the cortico"basal ganglia information network: not a simple closed loop, but parallel electrical pathways conducting distinct neural operations. Further, the neuromodulators each have their respective missions. Dopamine encodes the temporal difference error -- the reward learning signal. Acetylcholine affects learning rate through memory updates of actions and rewards. Noradrenaline controls width or randomness of exploration. Serotonin is implicated in "temporal discounting," evaluating if a given action is worth the expected reward. Doya reminds us that clinically "it is well known that the serotonin function is impaired in the depression patient."

The system of basal ganglia components and neuromodulators requires dynamic balancing. A delicate interplay determines outcomes for learning, actions, and affective states. Doya's synthetic models are proxies for human behavior, and his computational framework describing the moving parts ultimately has therapeutic implications for psychiatric and neurological disorders.

About the Speaker(s): Kenji Doya received B.S. and M.S. degrees from the University of Tokyo. His studies there culminated in a Ph.D. in Mathematical Engineering in 1991. He is Principal Investigator at the Okinawa Institute of Science and Technology in Japan, and is affiliated with the Advanced Telecommunications Research Institute International, heading the Computational Neuroscience Labs.

Doya has concentrated on computational neurobiology to discover and describe through algorithms the molecular mechanisms of the mind. His research examines reinforcement learning, metalearning, sequence learning, neuromodulators, and specialization and integration of brain structures. His laboratory subjects have been birds, monkeys, rats, and robots he calls "cyber rodents." The past twenty years of Doya's research activities are documented in more than 100 academic papers. He serves as co"editor"in"chief of Neural Networks, as well as guest editor for other international journals of current neuroscience research.

Host(s): School of Science, McGovern Institute for Brain Research at MIT

Comments (0)

It looks like no one has posted a comment yet. You can be the first!

You need to log in, in order to post comments. If you don’t have an account yet, sign up now!

MIT World — special events and lectures

MIT World — special events and lectures

Category: Events | Updated 15 days ago

Created
December 16, 2011 13:22
Category
Tags
License
All Rights Reserved (What is this?)
Additional Files


Viewed
3202 times

More from MIT World — special events and lectures

Vegetation Dynamics and the Earth System

Vegetation Dynamics and the Earth S...

Added almost 3 years ago | 01:06:00 | 2096 views

Bioengineering at MIT: Building Bridges Between the Sciences, Engineering and Health Care  (Part One)

Bioengineering at MIT: Building Bri...

Added almost 3 years ago | 01:51:00 | 2483 views

Space Shuttle Discovery Mission to the International Space Station

Space Shuttle Discovery Mission to ...

Added almost 3 years ago | 00:52:43 | 2379 views

How Much Do We Differ From Others and When Do We Know it?

How Much Do We Differ From Others a...

Added almost 3 years ago | 00:56:13 | 1638 views

The Future of Science and Technology in Europe

The Future of Science and Technolog...

Added almost 3 years ago | 01:34:00 | 2661 views

Transportation, the Built Environment and Greenhouse Gas Emissions in Developing Cities

Transportation, the Built Environme...

Added almost 3 years ago | 00:59:46 | 2975 views