Visualizing Optimization Algorithms

Here we look at a visualization of optimization algorithms:
- SGD (Stochastic Gradient Descent)
- Momentum
- NAG  (Numerical Algorithms Group)
- Adagrad (Adaptive Gradient Algorithm)
- Adadelta
- RMSProp (Divide the gradient by a running average of its recent magnitude)

What we are missing is the L-BFGS (Limited-memory BFGS)...

Deep Learning: Theano vs Torch vs Caffe vs deeplearning4j?

I am using deeplearning for hyperspectral data and I had to decide between available frameworks.

  Theano vs Torch vs Caffe vs deeplearning4j

Create Authentication for REST API

Authentication for REST API like done in amazon, facebook, dropbox they most probably conform to HMAC pattern. See here, here, here, and here.

alternatively

Semantic Web in Action - Structured Data

So Google has been building a knowledge graph of things. To better facilitate your website to be included in Google knowledge graph and enhance its SEO you have to embed semantic web (SW) vocabulary to your website.

NoSQL vs Lucene as Database

Today I was looking at the products of SindiceTech specifically SIREn which is schemaless search engine for JSON. They seem to be using it for their knowledge graph representation. Now the question boils down to why even bother with Lucene? MongoDB is already out there with scalability and fast nested schemaless indexing and search, even if you want to consider using it for a knowledge graph or semantic web context. Here is a good comparison of Lucene vs MongoDB as schemaless data stores and search engine. At the end of the day, if lucene done right it can do anything MongoDB can.

Matlab libsvm Mac Yosemite

After upgrading to Mac Yosemite things don't work as expected and Matlab doesn't run any more. So here is the fix:

Get Started with MEAN stack for Dummies

It's always tricky to get on track with new technologies. Here is how to get your first helloworld MEAN app up and running in a breeze!

Fucking SCP

Fucking scp didn't work because I printed out some statistics on the machine and its name bla bla and it took me some time to realize that scp depends on any returns from the server EVEVN ECHO prints!!!

So I had to disable all my status welcome messages and machine status printouts to console. Fucking dummy

Fix for Arabic Typing in Microsoft Word in Mac

How to fix sepratated/segragated arabic alphabet in microsoft word in mac:

Quit word.

download http://www.ghyoom.net/flv/Normal.dotm

copy and replace it with existing file located in

~/Library/Application Support/Microsoft/Office/User Templates

Source https://www.youtube.com/watch?v=hwUY31d7Wr4

Genetics and Inheritance for Idiots

Each person's cell is composed of 23 pairs of chromosomes (therefore a total of 46). Each location in a chromosome is called a Gene that is about a specific feature of a person e.g. hair color, how big teeth are, eye color, etc. Each pair of chromosomes contains different Genes than other pairs.

How to Pass Turing Test?

Recently a chatbot fooled human judges in thinking it is a 13 year old boy for whom English is the second language. Here is the basic details of the structure of this system: it uses a hierarchy of patterns specified in XML structure for state estimation, and a set of rules that activate based on state, with knowledge that is stored in the database. Watch the original conference Chatbots 3.0 conference held in 2010:

Curse of Dimentionality

Classifiers that tend to model non-linear decision boundaries very accurately (e.g. neural networks, KNN classifiers, decision trees) do not generalize well and are prone to overfitting. Therefore, the dimensionality should be kept relatively low when these classifiers are used. If a classifier is used that generalizes easily (e.g. naive Bayesian, linear classifier), then the number of used features can be higher since the classifier itself is less expressive.

which features should be used?