 |
Business | Industries | Finance | Tax |
| Index: > A B C D E F G H I J K L M N O P Q R S T U V W X Y Z |
|
|||||
| First Prev [ 1 2 3 ] Next Last |
Abstractly, the probability model for a classifier is a conditional model
over a dependent class variable with a small number of outcomes or classes, conditional on several feature variables through . The problem is that if the number of features is large or when a feature can take on a large number of values, then basing such a model on probability tables is infeasible. We therefore reformulate the model to make it more tractable.
Using Bayes' theorem, we write
In practice we are only interested in the numerator of that fraction, since the denominator does not depend on and the values of the features are given, so that the denominator is effectively constant. The numerator is equivalent to the joint probability model
which can be rewritten as follows, using repeated applications of the definition of conditional probability:
and so forth. Now the "naive" conditional independence assumptions come into play: assume that each feature is conditionally independent of every other feature for . This means that
and so the joint model can be expressed as
This means that under the above independence assumptions, the conditional distribution over the class variable can be expressed like this:
where is a scaling factor dependent only on , i.e., a constant if the values of the feature variables are known.
Models of this form are much more manageable, since they factor into a so-called class prior and independent probability distributions . If there are classes and if a model for can be expressed in terms of parameters, then the corresponding naive Bayes model has (k - 1) + n r k parameters. In practice, often (binary classification) and ( Bernoulli variable s as features) are common, and so the total number of parameters of the naive Bayes model is , where is the number of binary features used for prediction.
In a supervised learning setting, one wants to estimate the parameters of the probability model. Because of the independent feature assumption, it suffices to estimate the class prior and the conditional feature models independently, using the method of maximum likelihood, Bayesian inference or other parameter estimation procedures.
The discussion so far has derived the independent feature model, that is, the naive Bayes probability model. The naive Bayes classifier combines this model with a decision rule . One common rule is to pick the hypothesis that is most probable; this is known as the maximum a posteriori or MAP decision rule. The corresponding classifier is the function defined as follows:
The naive Bayes classifier has several properties that make it surprisingly useful in practice, despite the fact that the far-reaching independence assumptions are often violated. Like all probabilistic classifiers under the MAP decision rule, it arrives at the correct classification as long as the correct class is more probable than any other class; class probabilities do not have to be estimated very well. In other words, the overall classifier is robust to serious deficiencies of its underlying naive probability model. Other reasons for the observed success of the naive Bayes classifier are discussed in the literature cited below.
In real life, the naive Bayes approach is more powerful than might be expected from the extreme simplicity of its model; in particular, it is fairly robust in the presence of non-independent attributes wi. Recent theoretical analysis has shown why the naive Bayes classifier is so robust.
| Politics | Business | Finance |
 |
 |
 |
|