Now that you're familiar with Bayes' theorem and foundational concepts of Bayesian statistics, you'll take a look at how to implement some of these ideas for machine learning. Classification tasks can be a natural application of Bayes' theorem since you are looking to predict some label given other information, which can be conceptualized through conditional probability.
You will be able to:
- Explain the assumption that leads to Naive Bayes being "naive"
- Explain how to use the probabilities generated by Naive Bayes to make a classification
Naive Bayes algorithms extend Bayes' formula to multiple variables by assuming that these features are independent of one another. This then allows you to estimate an overall probability by multiplying the conditional probabilities for each of the independent features.
For example, extending the previous medical examples of Bayes' theorem, a researcher might examine multiple patient measurements to better predict whether or not an individual has a given disease. Provided that these measurements are independent (and uncorrelated from one another), one can then examine the conditional probability of each of these metrics and apply Bayes' theorem to determine a relative probability of having the disease or not. Combining these probabilities can then give an overall confidence of a patient having the disease given all the information. From this, one can then make a prediction for whether or not you believe an individual has the disease or not based on which probability is higher.
Mathematically, if
Again, note that multiplying the conditional probabilities is based on the assumption that these probabilities (and their underlying features) are independent -- and it is this assumption that the Naive Bayes algorithm is considered naive, or simple, because this is almost never true. However, Naives Bayes can prove to be quite efficient given the right circumstances, as you will see in the upcoming lessons.
In practice, calculating the denominator,
In this lesson, you briefly explored how Bayes' theorem can be used to build classification algorithms. In the upcoming lessons and labs you'll investigate particular implementations of Naive Bayes classifiers which differ in how the individual conditional probabilities themselves are constructed. As you will see, Naive Bayes can be extremely effective or trivially useful depending on the context and implementation.