What is Naive Bayes?

Naive Bayes is a simple but surprisingly powerful predictive modeling algorithm.

The model consists of two types of probabilities that can be calculated directly from your training data:

  1. Probability of each class
  2. Give the conditional probability of each class for each x value.

Once calculated, the probabilistic model can be used to predict new data using Bayes' theorem. When your data is real, you usually assume a Gaussian distribution (bell curve) so that you can easily estimate these probabilities.

Naive Bayesian theorem

Naive Bayes is called simplicity because it assumes that each input variable is independent. This is a powerful assumption that is unrealistic for actual data, however, the technique is very effective for a wide range of complex problems.


Advantages and disadvantages of Naive Bayes


  • The naive Bayesian model originated from classical mathematical theory and has a solid mathematical foundation and stable classification efficiency.
  • It has a higher speed for large numbers of training and queries.Even with very large training sets, there is usually only a relatively small number of features for each project, and the training and classification of the project is only a mathematical operation of the feature probability;
  • It works well for small-scale data, can handle multi-category tasks, and is suitable for incremental training (that is, it can train new samples in real time);
  • Less sensitive to missing data, the algorithm is also relatively simple, often used for text classification;
  • Naïve Bayes explains the results easily.


Things to note:

  • Need to calculate the prior probability;
  • There is an error rate in the classification decision;
  • Very sensitive to the form of input data;
  • due toThe assumption of sample attribute independence is used, so if the sample attributes are related, the effect is not good.



Baidu Encyclopedia version

The naive Bayesian method is a classification method based on Bayes' theorem and the independent assumption of feature conditions. The two broadest classification models are the Decision Tree Model and the Naive Bayesian Model (NBM).

Compared with the decision tree model, the Naive Bayes Classifier (NBC) originates from classical mathematical theory and has a solid mathematical foundation and stable classification efficiency. At the same time, the NBC model requires few parameters to estimate, is less sensitive to missing data, and the algorithm is relatively simple. In theory, the NBC model has the smallest error rate compared to other classification methods. But this is not always the case. This is because the NBC model assumes that the attributes are independent of each other. This assumption is often not true in practical applications, which has a certain impact on the correct classification of the NBC model.

Read More


Wikipedia version

In machine learning, the naive Bayes classifier is a series of simple "probabilistic classifiers" that apply strong (naive) independent assumptions between features based on Bayes' theorem.

Since the 20 century 50 era, Naïve Bayes has been extensively studied. It was introduced to the text retrieval community under different names in the 20 century 60 era, and is still a popular (baseline) method of text categorization, which is to judge documents as belonging to one category or another (eg spam or legal, sports or Politics, etc.) are characterized by word frequency. With proper pre-processing, it has more advanced methods in this field, including support vector machines, which are competitive. It is also suitable for automatic medical diagnosis.

The Naive Bayes classifier is highly scalable, requiring linear parameters for the number of variables (features/predictors) in learning problems. Maximum likelihood training can be done by evaluation to complete the expression of the closed form, which requires linear time, rather than being used by many other types of classifiers by expensive iterative approximations.

In the statistics and computer science literature, the naive Bayesian model is known by various names, including simple Bayes and independent Bayes. All of these names refer to the use of Bayes' theorem in classifier decision rules, but naive Bayes (necessarily) is not a Bayesian method.

Read More