One of the desired features of a classifier is that it generates probabilities for us. So we know with what probability a sample belongs to class A or class B (in the case of a binary classification problem). However it has been shown that the probabilities that the classifiers produce are not always correct.