Bayes' Theorem

Bayes’ theorem describes the probability of occurrence of an event related to any condition. It is also considered for the case of conditional probability. Bayes theorem is also known as the formula for the probability of “causes”. For example: if we have to calculate the probability of taking a blue ball from the second bag out of three different bags of balls, where each bag contains three different colour balls viz. red, blue, black. In this case, the probability of occurrence of an event is calculated depending on other conditions is known as conditional probability. In this article, let us discuss the statement and proof for Bayes theorem, its derivation, formula, and many solved examples.

Table of Contents:

• Statement
• Proof
• Formula
• Derivation
• Examples
• Applications
• Practice problems
• FAQs

Bayes Theorem Statement

Let E₁, E₂,…, E_n be a set of events associated with a sample space S, where all the events E₁, E₂,…, E_n have nonzero probability of occurrence and they form a partition of S. Let A be any event associated with S, then according to Bayes theorem,

for any k = 1, 2, 3, …., n

Bayes Theorem Proof

According to the conditional probability formula,

Using the multiplication rule of probability,

Using total probability theorem,

Putting the values from equations (2) and (3) in equation 1, we get

Note:

The following terminologies are also used when the Bayes theorem is applied:

Hypotheses: The events E₁, E₂,… E_n is called the hypotheses

Priori Probability: The probability P(E_i) is considered as the priori probability of hypothesis E_i

Posteriori Probability: The probability P(E_i|A) is considered as the posteriori probability of hypothesis E_i
Bayes’ theorem is also called the formula for the probability of “causes”. Since the E_i‘s are a partition of the sample space S, one and only one of the events E_i occurs (i.e. one of the events E_i must occur and the only one can occur). Hence, the above formula gives us the probability of a particular E_i (i.e. a “Cause”), given that the event A has occurred.

Bayes Theorem Formula

If A and B are two events, then the formula for the Bayes theorem is given by:

Where P(A|B) is the probability of condition when event A is occurring while event B has already occurred.
Also, get the Bayes Theorem Calculator here.

Bayes Theorem Derivation

Bayes Theorem can be derived for events and random variables separately using the definition of conditional probability and density.

From the definition of conditional probability, Bayes theorem can be derived for events as given below:

P(A|B) = P(A ⋂ B)/ P(B), where P(B) ≠ 0

P(B|A) = P(B ⋂ A)/ P(A), where P(A) ≠ 0

Here, the joint probability P(A ⋂ B) of both events A and B being true such that,

P(B ⋂ A) = P(A ⋂ B)

P(A ⋂ B) = P(A | B) P(B) = P(B | A) P(A)

P(A|B) = [P(B|A) P(A)]/ P(B), where P(B) ≠ 0

Similarly, from the definition of conditional density, Bayes theorem can be derived for two continuous random variables namely X and Y as given below:

Therefore, 

Examples and Solutions

Some illustrations will improve the understanding of the concept.

Example 1:

A bag I contains 4 white and 6 black balls while another Bag II contains 4 white and 3 black balls. One ball is drawn at random from one of the bags, and it is found to be black. Find the probability that it was drawn from Bag I.

Solution:

Let E₁ be the event of choosing bag I, E₂ the event of choosing bag II, and A be the event of drawing a black ball.

Then,

Also, P(A|E₁) = P(drawing a black ball from Bag I) = 6/10 = 3/5

P(A|E₂) = P(drawing a black ball from Bag II) = 3/7

By using Bayes’ theorem, the probability of drawing a black ball from bag I out of two bags,

Example 2:

A man is known to speak the truth 2 out of 3 times. He throws a die and reports that the number obtained is a four. Find the probability that the number obtained is actually a four.

Solution:

Let A be the event that the man reports that number four is obtained.

Let E₁ be the event that four is obtained and E₂ be its complementary event.

Then, P(E₁) = Probability that four occurs = 1/6.

P(E₂) = Probability that four does not occur = 1- P(E₁) = 1 – (1/6) = 5/6.

Also, P(A|E₁)= Probability that man reports four and it is actually a four = 2/3

P(A|E₂) = Probability that man reports four and it is not a four = 1/3.

By using Bayes’ theorem, probability that number obtained is actually a four, P(E₁|A)

Bayes Theorem Applications

One of the many applications of Bayes’ theorem is Bayesian inference, a particular approach to statistical inference. Bayesian inference has found application in various activities, including medicine, science, philosophy, engineering, sports, law, etc. For example, we can use Bayes’ theorem to define the accuracy of medical test results by considering how likely any given person is to have a disease and the test’s overall accuracy. Bayes’ theorem relies on consolidating prior probability distributions to generate posterior probabilities. In Bayesian statistical inference, prior probability is the probability of an event before new data is collected.

Practice Problems

Solve the following problems using Bayes Theorem.

1. A bag contains 5 red and 5 black balls. A ball is drawn at random, its colour is noted, and again the ball is returned to the bag. Also, 2 additional balls of the colour drawn are put in the bag. After that, the ball is drawn at random from the bag. What is the probability that the second ball drawn from the bag is red?
2. Of the students in the college, 60% of the students reside in the hostel and 40% of the students are day scholars. Previous year results report that 30% of all students who stay in the hostel scored A Grade and 20% of day scholars scored A grade. At the end of the year, one student is chosen at random and found that he/she has an A grade. What is the probability that the student is a hosteler? 
3. From the pack of 52 cards, one card is lost. From the remaining cards of a pack, two cards are drawn and both are found to be diamond cards. What is the probability that the lost card is a diamond?

Students, are you struggling to find a solution to a specific question from Bayes theorem? We will make it easy for you. For a detailed discussion on the concept of Bayes’ theorem, download BYJU’S – The Learning App.