TheanoLab @ NCAP CIFAR Summer School 2011

Aug 2-6, 2011, Toronto, Canada.

Theano is python software for evaluating complicated array expressions.

What does it do?

  • aggressive expression optimizations,

  • automatic GPU use, and

  • symbolic differentiation.

It complements the Python numeric/scientific software stack (e.g. NumPy, SciPy, scikits, matplotlib, PIL.)

Design and feature set has been driven by machine learning research at the University of Montreal (groups of Yoshua Bengio, Pascal Vincent, Douglas Eck). The result is a very good library for doing research in deep learning and neural network training, and a flexible framework for many other models and algorithms in machine learning more generally.

It has proven to be useful for implementing:

  • linear and nonlinear neural network classifiers

  • convolutional models

  • Energy models: RBM, DBN, GRBM, ssRBM, AIS

  • Auto-encoders: DAE, CAE

  • GP regression

  • sparse coding

  • recurrent neural networks, echo state, (HMM?)

  • online and batch learning and optimization

As people’s needs change this list will grow, but Theano is built around vector, matrix, and tensor expressions; there is little reason to use it for calculations on other data structures.

Contents

The structured part of these lab sessions will be a walk-through of the following material. Interleaved with this structured part will be blocks of time for individual or group work. The idea is that you can try out Theano and get help from gurus on hand if you get stuck.