CNS*2020 Keynote 1: Deep reinforcement learning and its neuroscientific implications - Matthew Botvinick

Institutions OCNS
,
1

K1: Deep reinforcement learning and its neuroscientific implications - Matthew Botvinick

The last few years have seen some dramatic developments in artificial intelligence research. What implications might these have for neuroscience? Investigations of this question have, to date, focused largely on deep neural networks trained using supervised learning, in tasks such as image classification. However, there is another area of recent AI work which has so far received less attention from neuroscientists, but which may have more profound neuroscientific implications: Deep reinforcement learning. Deep RL offers a rich framework for studying the interplay among learning, representation and decision-making, offering to the brain sciences a new set of research tools and a wide range of novel hypotheses. I’ll provide a high level introduction to deep RL, discuss some recent neuroscience-oriented investigations from my group at DeepMind, and survey some wider implications for research on brain and behavior.

2

Some Questions:

  1. Do vector-like distributed dopamine signals relate to successor representation (predictive coding) based reinforcement learning?
  2. Are successor representation based reinforcement learning models possibly better (in terms of artificial intelligence applications or modeling animal behaviors) and more biologically plausible than model free and model based reinforcement learning models?
3

I am wondering how far we can take the parallels between modern deep RL systems and how brains work. It is quite clear that on a fundamental level they aren’t the same: DL uses real valued states and ReLu transfer functions but neurons in the brain are spiking and implement different, potentially complex “transfer functions”. On the other hand, Matt pointed to some quite detailed correspondences. Is there some more fundamental principles in the background that they both are different implementations / approximations of? What would that be?

1 Like
4

Are you coming to the discussion session at 10pm (Berlin) today? Would you be happy to ask the questions on screen?