Getting the Power of GPU for Deep Learning

install EPD
install bleeding edge theano

On my macbook pro it says my graphics card is Intel HD Graphics 4000 1024 MB. I was very disappointed that Apple just left an intel graphics card on a $2000 laptop. They could definitely do better. So I gave up on CUDA. Recently I found that two graphics cards on board!

Visualizing Optimization Algorithms

Here we look at a visualization of optimization algorithms:
- SGD (Stochastic Gradient Descent)
- Momentum
- NAG  (Numerical Algorithms Group)
- Adagrad (Adaptive Gradient Algorithm)
- Adadelta
- RMSProp (Divide the gradient by a running average of its recent magnitude)

What we are missing is the L-BFGS (Limited-memory BFGS)...