Oral History of Juergen Schmidhuber, part 1 of 2

Video Description

Jürgen Schmidhuber looks back on the discoveries, collaborations, and mentorships that defined his path in AI, including the 1990–1991 period in which he and his students developed concepts that became central to modern neural network research. Interviewed by Hansen Hsu on 2025-06-24 © Computer History Museum Jürgen Schmidhuber was born in Munich in 1963. Young Schmidhuber was interested in math, physics, art, and science fiction. His first experience with a computer was a Sinclair ZX81. Schmidhuber attended the Technical University of Munich, completing a diploma thesis in 1987, a doctoral thesis on artificial neural networks in 1991, and a postdoctoral thesis in 1993 before starting a faculty position at Munich. During his studies, Schmidhuber worked with LISP machines, hidden Markov models, genetic algorithms, meta-learning, as well as neural networks, and considers symbolic and sub-symbolic AI parts along a continuum. Schmidhuber considers 1990-1991 his “miraculous year,” in which he and collaborators came up with many ideas in deep learning: very deep recurrent neural networks, pre-training (the ‘P’ in ChatGPT), distillation, generative adversarial networks, artificial curiosity, planning with recurrent neural world models, hierarchical reinforcement learning, and fast weight programmers, some of which are now called "unnormalized linear transformers” (the ‘T’ in ChatGPT). He supervised the diploma thesis of his student Sepp Hochreiter which led to the invention of the long short-term memory (LSTM) type of recurrent neural network and to the most cited computer science paper of the 20th century, as well as insights leading to the creation of extremely deep feed-forward networks called “highway” networks (May 2015), a variant of which was later popularized by Microsoft as “ResNets.” Schmidhuber became co-director of the Dalle Molle Institute for Artificial Intelligence (IDSIA) in Lugano, Switzerland, in 1995. He continued to hold that position until 2021, while also holding a professorship in cognitive robotics at the Tech University of Munich from 2004 to 2009, and a professorship at the University of Lugano from 2009-2021. In 2021, he accepted his current position at King Abdullah University of Science and Technology in Saudi Arabia. While at IDSIA, Schmidhuber mentored many students and postdocs, including Dan Ciresan, a Romanian who with others created DanNet, the first GPU-trained convolutional neural network to win image recognition competitions in 2011, including street signs, Chinese handwriting, and medical imaging, prior to the similar AlexNet’s ImageNet win in 2012. Other students include Daan Wierstra, the first technical employee at DeepMind. DeepMind cofounder Shane Legg was also a student in Schmidhuber’s group at IDSIA. * Note: Transcripts represent what was said in the interview. However, to enhance meaning or add clarification, interviewees have the opportunity to modify this text afterward. This may result in discrepancies between the transcript and the video. Please refer to the transcript for further information - http://www.computerhistory.org/collections/catalog/300000122 Visit computerhistory.org/collections/oralhistories/ for more information about the Computer History Museum's Oral History Collection. Catalog number: 300000123 Acquisition number: 2025.0087