January 28, 2015

Deep Learning: Theano vs Torch vs Caffe vs deeplearning4j?

I am using deeplearning for hyperspectral data and I had to decide between available frameworks.

  Theano vs Torch vs Caffe vs deeplearning4j



There really isn't a single one that stands out from others.

Theano offers a compiler that automatically calculates derivatives for you. but s hard to modify and debug, a lot of tutorials online and lots and lots of active community. bunch of kaggle winners used theano

Torch is being used by the big guys: facebook, google, purdue, etc. kinda like matlab but being dependent on Lua is a draw back. there seems to be a bunch of activity on its github but its mostly from big guys and normal users aren't there

caffe seems more popular on github developed by berkeley. itiis very abstract (define a network with json) and high level dedicated to convolution networks, hard to tweak

DL4j can give you the backone of hadoop and spark but it needs more review.

--------------

I tried to go with torch but getting up and running fast was not an option. No good tutorial out there. Contrary Theano has plenty of resources to learn and videos to view. So for now I have created a single layer MNIST logistic layer NN. We'll see how it goes from there

Some benchmark and comparison link
https://github.com/soumith/convnet-benchmarks
http://www.quora.com/Deep-Learning/How-fast-is-Theano-compared-to-other-DBN-implementations
http://www.reddit.com/r/MachineLearning/comments/2c9x0s/best_framework_for_deep_neural_nets/

https://news.ycombinator.com/item?id=7928738
http://deeplearning.net/blog/

5 comments:

  1. Of those, only Deeplearning4j is commercially supported. It has a very active Google Group and responsive community members. DL4J integrates with Hadoop and Spark, and integrates with distributed CPUs and GPUs. Its linear-algebra engine ND4J has Java and Scala APIs.

    ReplyDelete
    Replies
    1. What do you mean by commercially supported?

      Delete
  2. which one did you chose at last?

    ReplyDelete
  3. I actually stuck with theano to begin with and get some proof of concept by myself. I kissed away the ease and abstraction of caffe and put aside the cumbersomeness of Torch for now. Later on if I need better speed up and can verify other systems perform better I might switch to those, but for now Theano does the job,

    ReplyDelete