Bayes' Theorem

Bayes’ theorem describes the probability of occurrence of an event related to any condition. It is also considered for the case of conditional probability. Bayes theorem is also known as the formula for the probability of “causes”. For example: if we have to calculate the probability of taking a blue ball from the second bag out of three different bags of balls, where each bag contains three different colour balls viz. red, blue, black. In this case, the probability of occurrence of an event is calculated depending on other conditions is known as conditional probability. In this article, let us discuss the statement and proof for Bayes theorem, its derivation, formula, and many solved examples.

Table of Contents:

Bayes Theorem

Bayes Theorem Statement

Let E1, E2,…,En be a set of events associated with a sample space S, where all the events E1, E2,…, En have nonzero probability of occurrence and they form a partition of S. Let A be any event associated with S, then according to Bayes theorem,

\(P(E_i│A)~=~\frac{P(E_i)P(A│E_i)}{\sum\limits_{k=1}^{n}P(E_k)P(A| E_k)}\)

for any k = 1, 2, 3, …., n

Bayes Theorem Proof

According to the conditional probability formula,

\(P(E_i│A)~=~\frac{P(E_i ∩ A)}{P(A)}\) ⋯⋯⋯⋯⋯⋯⋯⋯(1)

Using the multiplication rule of probability,
\(P(E_i ∩ A)~= ~P(E_i)P(A │E_i)\)⋯⋯⋯⋯⋯⋯⋯⋯(2)

Using total probability theorem,
\(P(A)~=~\sum\limits_{k=1}^{n}~P(E_k)P(A| E_k)\)⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯(3)

Putting the values from equations (2) and (3) in equation 1, we get

\(P(E_i│A)~=~\frac{P(E_i)P(A│E_i)}{\sum\limits_{k=1}^n~P(E_k)P(A| E_k)}\)

Note:

The following terminologies are also used when the Bayes theorem is applied:

Hypotheses: The events E1, E2,… En is called the hypotheses

Priori Probability: The probability P(Ei) is considered as the priori probability of hypothesis Ei

Posteriori Probability: The probability P(Ei|A) is considered as the posteriori probability of hypothesis Ei

Bayes Theorem Formula

If A and B are two events, then the formula for Bayes theorem is given by:

P(A|B) = P(A∩B)/P(B)

Where P(A|B) is the probability of condition when event A is occurring while event B has already occurred.

P(A ∩ B) is the probability of event A and event B

P(B) is the probability of event B

Bayes Theorem Derivation

Bayes Theorem can be derived for events and random variables separately using the definition of conditional probability and density.

From the definition of conditional probability, Bayes theorem can be derived for events as given below:

P(A|B) = P(A ⋂ B)/ P(B), where P(B) ≠ 0

P(B|A) = P(B ⋂ A)/ P(A), where P(A) ≠ 0

Here, the joint probability P(A ⋂ B) of both events A and B being true such that,

P(B ⋂ A) = P(A ⋂ B)

P(A ⋂ B) = P(A | B) P(B) = P(B | A) P(A)

P(A|B) = [P(B|A) P(A)]/ P(B), where P(B) ≠ 0

Similarly, from the definition of conditional density, Bayes theorem can be derived for two continuous random variables namely X and Y as given below:

\(f_{X|Y=y}(x)=\frac{f_{X,Y(x,y)}}{f_Y(y)}\\f_{Y|X=x}(y)=\frac{f_{X,Y(x,y)}}{f_X(x)}\)

Therefore, 

\(f_{X|Y=y}(x)=\frac{f_{Y|X=x}(y)f_X(x)}{f_Y(y)}\)

Examples and Solutions

Some illustrations will improve the understanding of the concept.

Example 1:

A bag I contain 4 white and 6 black balls while another Bag II contains 4 white and 3 black balls. One ball is drawn at random from one of the bags, and it is found to be black. Find the probability that it was drawn from Bag I.

Solution:

Let \(E_1\) be the event of choosing the bag I, \(E_2\) the event of choosing the bag II, and A be the event of drawing a black ball.

Then,\(P(E_1)~ = ~P(E_2)~ =~\frac{1}{2}\)

Also,\(P(A|E_1) ~= ~P\)(drawing a black ball from Bag I) = \(\frac{6}{10}~ = ~\frac{3}{5}\)

\(P(A|E_2) ~=~ P\)(drawing a black ball from Bag II) = \(\frac{3}{7}\)

By using Bayes’ theorem, the probability of drawing a black ball from bag I out of two bags,

\(P(E_1 |A)~ =~\frac{P(E_1)P(A|E_1)}{P(E_1 )P(A│E_1 )+ P(E_2)P(A|E_2)}\)

=\(\large\frac{\frac{1}{2}~\times~\frac{3}{5}}{\frac{1}{2}~\times~\frac{3}{5}~+~\frac{1}{2}~ ×~\frac{3}{7}}\) = \(\frac{7}{12}\)

Example 2:

A man is known to speak truth 2 out of 3 times. He throws a die and reports that the number obtained is a four. Find the probability that the number obtained is actually a four.

Solution:

Let \(A\) be the event that the man reports that number four is obtained.

Let \(E_1\) be the event that four is obtained and \(E_2\) be its complementary event.

Then, \(P(E_1)\) = Probability that four occurs = \(\frac{1}{6}\)

\(P(E_2)\) = Probability that four does not occurs = \(1 ~–~ P(E_1) ~=~ 1~-\frac{1}{6}~ =~\frac{5}{6}\)

Also, \(P(A|E_1)\) = Probability that man reports four and it is actually a four = \(\frac{2}{3}\)

\(P(A|E_2)\) = Probability that man reports four and it is not a four = \(\frac{1}{3}\)

By using Bayes’ theorem, probability that number obtained is actually a four,

\(P(E_1 |A)~ \) \(= \large \frac{P(E_1)P(A|E_1)}{P(E_1 )P(A│E_1 )~+~ P(E_2)P(A|E_2)}~
=~\frac{\frac{1}{6} ~ ×~ \frac{2}{3}}{\frac{1}{6} ~×~ \frac{2}{3}~ +~ \frac{5}{6}~ ×~\frac{1}{3}}\)
= \(\frac{2}{7}\)

Practice Problems

Solve the following problems using Bayes Theorem.

  1. A bag contains 5 red and 5 black balls. A ball is drawn at random, its color is noted, and again the ball is returned to the bag. Also, 2 additional balls of the color drawn are put in the bag. After that, the ball is drawn at random from the bag. What is the probability that the second ball drawn from the bag is red?
  2. Of the students in the college, 60% of the students reside in the hostel and 40% of the students are day scholars. Previous year result reports that 30% of all students who stay in the hostel scored A Grade and 20% of day scholars scored A grade. At the end of the year, one student is chosen at random and found that he/she has an A grade. What is the probability that the student is a hostlier? 
  3. From the pack of 52 cards, one card is lost. From the remaining cards of a pack, two cards are drawn and both are found to be the diamond cards. What is the probability that the lost card being a diamond?

Frequently Asked Questions on Bayes Theorem

What is meant by Bayes theorem in probability?

In Probability, Bayes theorem is a mathematical formula, which is used to determine the conditional probability of the given event. Conditional probability is defined as the likelihood that an event will occur, based on the occurrence of a previous outcome.

How is Bayes theorem different from conditional probability?

As we know, Bayes theorem defines the probability of an event based on the prior knowledge of the conditions related to the event. In case, if we know the conditional probability, we can easily find the reverse probabilities using the Bayes theorem.

When can we use Bayes theorem?

Bayes theorem is used to find the reverse probabilities if we know the conditional probability of an event.

What is the formula for Bayes theorem?

The formula for Bayes theorem is:
P(A|B)= [P(B|A). P(A)]/P(B)
Where P(A) and P(B) are the probabilities of events A and B.
P(A|B) is the probability of event A given B
P(B|A) is the probability of event B given A.

Where can we use Bayes theorem?

Bayes rule can be used in the condition while answering the probabilistic queries conditioned on the piece of evidence.

Students, are you struggling to find a solution to a specific question from Bayes theorem? We will make it easy for you. For a detailed discussion on the concept of Bayes’ theorem, download BYJU’S – The Learning App.

2 Comments

  1. Superb explanation. Thanks to bujy’s🙂

    1. very nicely explained.

Leave a Comment

Your email address will not be published. Required fields are marked *

BOOK

Free Class

by India's top teacher with
live doubt solving