Conscious processing, inductive biases and generalization in deep learning (Bengio)

Yoshua Bengio, MILA, Université de Montréal 17 feb 2023

VIDEO

Abstract: Humans are very good at “out-of-distribution” generalization (compared to current AI systems). It would be useful to determine the inductive biases they exploit and translate them into machine-language architectures, training frameworks and experiments. I will discuss several of these hypothesized inductive biases. Many exploit notions in causality and connect abstractions in representation learning (perception and interpretation) with reinforcement learning (abstract actions). Systematic generalizations may arise from efficient factorization of knowledge into recomposable pieces. This is partly related to symbolic AI (aas seen in the errors and limitations of reasoning in humans, as well as in our ability to learn to do this at scale, with distributed representations and efficient search). Sparsity of the causal graph and locality of interventions — observable in the structure of sentences — may reduce the computational complexity of both inference (including planning) and learning. This may be why evolution incorporated this as “consciousness.” I will also suggest some open research questions to stimulate further research and collaborations.

Yoshua Bengio, Professor, University of Montreal, founder and scientific director of Mila – Institut québécois d’AI, and co-director CIFAR’s Machine Learning, Biological Learning program as a Senior Fellow. He also serves as scientific director of IVADO.

Butlin, P., Long, R., Elmoznino, E., Bengio, Y., Birch, J., Constant, A., … & VanRullen, R. (2023). Consciousness in artificial intelligence: insights from the science of consciousness. arXiv preprint arXiv:2308.08708.

Zador, A., Escola, S., Richards, B., Ölveczky, B., Bengio, Y., Boahen, K., … & Tsao, D. (2023). Catalyzing next-generation artificial intelligence through neuroai. Nature Communications, 14(1), 1597.