Basic Biological RL: “Asking computational questions with associative tasks”

Melissa Sharpe

Testing theories of computational neuroscience often requires behavioural experiments. It is becoming more and more common that computational neuroscientists have the resources and expertise to test their theories in their own labs. This is hugely beneficial for neuroscience. The field of associative learning theory, which comprises a study of the content of associations that underlie our cognitive experience, provides a wealth of experimental designs that can be used to test computational theories. These procedures have been extensively validated across many decades, providing detailed accounts of the learned constructs underlying particular behavioural phenomenon. A carefully tailored experimental design allows for a qualitative analysis of decision-making behaviour and the neural substrates underlying particular behavioural constructs. To demonstrate the utility of associative paradigms in this context, I will discuss the use of associative tasks in testing the computational theory of the dopamine prediction error.

Basic Computational RL

Csaba Szepesvári

Csaba Szepesvári is the team leader of the Foundations group at DeepMind, a Professor of Computing Science of the University of Alberta and a Principal Investigator of the Alberta Machine Intelligence Institute. With over 30 years of experience in machine learning including 10 years of industry experience, he has published two books, a third one nearly finished, as well as over 200 peer-refereed journal and conference papers on machine learning, reinforcement learning, learning theory, bandit algorithms, winning paper awards at ICML, UAI, ALT, IEEE ADPRL and ECML. He is best known as the co-inventor of UCT, a tree-search algorithm that has become a canonical search method in Artificial Intelligence, serving, among other things, as the core search algorithm in DeepMind’s AlphaGo, which defeated top human Go players. Prof. Szepesvári currently serves as an action editor for the Journal of Machine Learning Research. Previously, he served as a co-chair for the top Conference on Learning Theory in 2014, and for the Algorithmic Learning Theory Conference in 2011, as well as a senior PC member for NIPS, ICML, AISTATS, AAAI and IJCAI for many years. Of the 34 trainees (13 PhD, 14 MSc, 7 Post-doctoral) he graduated during his 10+ year tenure at the University of Alberta, 7 won prestigious awards in Artificial Intelligence with their thesis work. Of these trainees, 20 are now professors or are employed at Google, Deepmind, Yahoo, Apple, Mitsubishi Research, Adobe, Huawei, while others continue their training.

Advanced Computational RL: “Counterfactuals and Reinforcement Learning.”

Emma Brunskill

Advanced topic in Biological RL: “Dynamic Decisions in Humans”

Cleotide Gonzalez

Decisions we make every day may range from simple to highly complex. For example, during driving we make many decisions that seem effortless and routine, such as judging the distance to the front car or the speed regulation; while other decisions such as making route planning navigation, or allocating our attention over multiple demands and distractions may seem very complex. However, these and many of the decisions we make are in fact very similar in some respects: they are made in the presence of environmental change and in the absence of explicit information regarding possible outcomes and their probabilities. The experiences of the decision makers and how those experiences are acquired and used in context, could make some decisions simple and others complex. The way humans make dynamic decisions depend on individualized experience, cognitive abilities, and their interaction with the particular conditions of the decision environment.

In this session, we will discuss how decisions are made from experience, in different dynamic situations, and how our cognitive processes (e.g., attention, memory, biases, and other factors) influence the way those decisions are made. Participants will be introduced to different topics of dynamic decision processes and dynamic systems by analyzing the sources of error in complex problems, such as cases of accidents and disasters.  Participants will be introduced to simulations of dynamic systems (e.g., microworlds/decision games) that are used to understand how humans learn and adapt to changing conditions of choice; they will learn to construct graphical models of dynamic systems and simulations that can help in understanding complex systems.  Participants will be introduced to dynamic decision models and to simulations of different model scenarios and the interpretation of simulation results to provide decision recommendations.