Compulsion and the mechanisms of model-based reinforcement learning

Decisions and neural correlates of decision variables both indicate that humans and animals make decisions taking into account task structure. Such deliberative, "model-based" choice is thought to be important for overcoming habits and various sorts of compulsions, but there is still little evidence about the algorithmic or neural mechanisms that support it. I discuss recent studies attempting to address these questions.

How to spot confidence in the brain

Decision confidence is a forecast about the correctness of one’s decision. It is often regarded as a higher-order function of the brain requiring a capacity for metacognition that may be unique to humans. If confidence manifests itself to us as a feeling, how can then one identify it amongst the brain’s electrical signals in an animal? We tackle this issue by using mathematical models to gain traction on the problem of confidence, allowing us to identify neural correlates and mechanisms.

Understanding trained recurrent neural networks

Recurrent neural networks are an important class of models for explaining neural computations. Recently, there has been progress both in training these networks to perform various tasks, and in relating their activity to that recorded in the brain. Despite this progress, there are many fundamental gaps towards a theory of these networks. Neither the conditions for successful learning, nor the dynamics of trained networks are fully understood.

Flexible sensorimotor routing in the rat

We trained rats in a task in which two cued sensorimotor associations could be rapidly reversed, from one trial to the next (Duan et al., Neuron 2015). I will describe behavioral, electrophysiological, optogenetic, and modeling data which suggest that the superior colliculus appears to play a surprisingly cognitive role in enabling the top-down executive control required to perform sensorimotor reversals in the task.

Spiking neurons can discover predictive features by aggregate-label learning

The brain routinely discovers sensory clues that predict opportunitiesor dangers. However, it is unclear how neural learning processes canbridge the typically long delays between sensory clues and behavioraloutcomes. Here, I introduce a learning concept, aggregate-labellearning, that enables biologically plausible model neurons to solvethis temporal credit assignment problem. Aggregate-label learningmatches a neuron’s number of output spikes to a feedback signal that isproportional to the number of clues but carries no information abouttheir timing.

The interaction of plasticity processes across time scales and their role in memory dynamics

Many experiments provide evidences that, after learning, human and animal memories are very dynamic and changeable. Amongst others, one intriguing and counterintuitive effect is the destabilization of memories by recalling them. In addition, some of these destabilized memories can be ‘rescued’ by sleep-induced consolidation while others not. Up to now, the basic principles underlying these effects are widely unknown.