Bayes Theorem

Bayes' Theorem is one of the fundamental concepts in probability theory, named after the English statistician Thomas Bayes. It provides a powerful method for calculating conditional probabilities, which are the likelihoods of events based on prior knowledge or evidence. This theorem is particularly useful when new information becomes available, allowing us to update our predictions or beliefs accordingly.

In simple terms, Bayes' Theorem allows us to revise our initial assumptions about the probability of an event as new data is introduced. It connects prior probability, the likelihood of current evidence, and the overall probability of the observed data to give a more accurate estimation of the event in question.

1.0What is Bayes' Theorem?

Let E₁, E₂, …, E_n represent mutually exclusive and collectively exhaustive events in a random experiment, and let E be any event that occurs with some E_i. Then,  

$P\left(E_i|E\right)=\frac{P\left(E|E_i\right)\cdot P\left(E_i\right)}{{\sum }_{i=1}^{n}P\left(E\mid E_i\right)\cdot P\left(E_i\right)};i=1,2,3,\dots$, n

Proof: 

By the theorem of total probability, we have

$P(E)={\sum }_{i=1}^{n}P\left(E\mid E_i\right)\cdot P\left(E_i\right)$ …. (1)

$\therefore P\left(E_i\mid E\right)=\frac{P\left(E\cap E_i\right)}{P(E)}$

[by multiplication theorem]

$=\frac{P\left(E\mid E_i\right)\cdot P\left(E_i\right)}{P(E)}$

$\left[\therefore \mathrm{P}\left(\mathrm{E}\mid {\mathrm{E}}_{\mathrm{i}}\right)=\frac{\mathrm{P}\left(\mathrm{E}\cap {\mathrm{E}}_{\mathrm{i}}\right)}{\mathrm{P}\left({\mathrm{E}}_{\mathrm{i}}\right)}\right]$

$\begin{array}{rl}=\frac{P\left(E\mid E_i\right)\cdot P\left(E_i\right)}{{\sum }_{i=1}^{n}P\left(E\mid E_i\right)\cdot P\left(E_i\right)}& \\ & \text{ [using (i)] }\end{array}$

Hence, 

$P\left(E_i\mid E\right)=\frac{P\left(E\mid E_i\right)\cdot P\left(E_i\right)}{{\sum }_{i=1}^{n}P\left(E\mid E_i\right)\cdot P\left(E_i\right)}$

Bayes' Theorem describes the probability of an event based on prior knowledge of conditions related to the event. It combines prior probability with the likelihood of new evidence to produce a revised probability.

2.0Bayes' Theorem Formula: 

$P\left(E_i\mid E\right)=\frac{P\left(E\mid E_i\right)\cdot P\left(E_i\right)}{{\sum }_{i=1}^{n}P\left(E\mid E_i\right)\cdot P\left(E_i\right)}OrP\left(E_i\mid E\right)=\frac{P\left(E\mid E_i\right)\cdot P\left(E_i\right)}{P(E)}$

where:

• P(E_i |E) is the posterior probability (the probability of event E_i given that E is true).
• P(E | E_i) is the likelihood (the probability of event E given that E_i is true).
• P(E_i) is the prior probability (the initial probability of event E_i).
• P(E) is the marginal likelihood (the total probability of event E).

3.0Solved Examples of Bayes Theorem

Example 1: A factory operates three machines—X, Y, and Z—which produce 1000, 2000, and 3000 bolts daily, respectively. Machine X produces defective bolts at a rate of 1%, Y at 1.5%, and Z at 2%. At the end of the day, if a randomly selected bolt is found to be defective, what is the probability that it was produced by one of these machines?

Solution: 

Total number of bolts produced in a day.

= (1000 + 2000 + 3000) = 6000.

Let E₁, E₂, and E₃ be the events of drawing a bolt produced by machines X, Y, and Z respectively. Then,

Let E be the event of drawing a defective bolt. Then,

P(E |E₁) = probability of drawing a defective bolt, given that it is produced by machine X.

$=\frac{1}{100}$

P(E |E₂) = probability of drawing a defective bolt, given that it is produced by the machine Y

$=\frac{1.5}{100}=\frac{15}{1000}=\frac{3}{200}$

P(E |E₃) = probability of drawing a defective bolt, given that it is produced by the machine Z

$=\frac{2}{100}=\frac{1}{50}.$

Required Probability

= P(E₁ |E)

= probability that the bolt drawn is produced by X, given that it is defective

= $\frac{P(E_i)\cdot P\left(E\mid E_i\right)}{P\left(E_1\right)\cdot P\left(E\mid E_1\right)+P\left(E_2\right)\cdot P\left(E\mid E_2\right)+P\left(E_3\right)\cdot P\left(E\mid E_3\right)}$

$=\frac{\left(\frac{1}{6}\times \frac{1}{100}\right)}{\left(\frac{1}{6}\times \frac{1}{100}\right)+\left(\frac{1}{3}\times \frac{3}{200}\right)+\left(\frac{1}{2}\times \frac{1}{50}\right)}$

$=\left(\frac{1}{600}\times \frac{600}{10}\right)=\frac{1}{10}=0.1$

Hence, the required probability is 0.1.

Example 2: Bag A contains 2 white balls and 3 red balls, while bag B contains 4 white balls and 5 red balls. A ball is randomly drawn from one of the bags, and it turns out to be red. What is the probability that this red ball was drawn from bag B?

Solution:  

Let

E₁ = The event of ball being drawn from bag A

E2 = The event of the ball being drawn from bag B.

E = The event of the ball being red.

Since, both the bags are equally likely to be selected, therefore

$P\left(E_1\right)=P\left(E_2\right)=\frac{1}{2}$

$P\left(E\mid E_1\right)=\frac{3}{5}$

$P\left(E\mid E_1\right)=\frac{3}{5}$

$P\left(E\mid E_2\right)=\frac{5}{9}$

∴ Required probability  

$P\left(E_2\mid E\right)=\frac{P\left(E_2\right)P\left(E\mid E_2\right)}{P\left(E_1\right)\cdot P\left(E\mid E_1\right)+P\left(E_2\right)P\left(E\mid E_2\right)}$

$P\left(E_2\mid E\right)=\frac{\left(\frac{1}{2}\times \frac{5}{9}\right)}{\left(\frac{1}{2}\times \frac{3}{5}\right)+\left(\frac{1}{2}\times \frac{5}{9}\right)}$

$P\left(E_2\mid E\right)=\frac{25}{52}$

Hence, the required probability is $\frac{25}{52}$ .

Example 3:

9 balls are transferred from A to B & then 9 balls are transferred from B to A then find the probability that the red ball is still in Bag A.

Solution: 

n(F): red ball in Bag A. 

Initially transferred from A to B: 1R + 8G or 9G

Now, number of balls in bag B: 1R + 18G or Zero R + 19G

Later: Balls transferred from B to A: 1R + 8G or 9G

$P=\frac{{}^9{C}_9}{{}^{10}{C}_9}\times \frac{{}^{19}{C}_9}{{}^{19}{C}_9}$   ...(1)

or

$P=\frac{{}^9{C}_8\times {}^1{C}_1}{{}^{10}{C}_9}\times \frac{{}^{18}{C}_8\times {}^1{C}_1}{{}^{19}{C}_9}$  ...(2)

Equation (1) + (2)

$=\frac{1}{10}+\frac{9}{10}\times \frac{9}{19}=\frac{100}{190}=\frac{10}{19}$

Example 4: There are 3 boxes containing respectively 1 white, 2 red and 3 black balls, 2 white, 3 red and 1 black balls, 3 white, 1 red and 2 black balls. One box is chosen at random, and then two balls are drawn at random from the selected box. The two balls are one red and one white. Find the probability that these came from the first box. 

Solution: 

If let E₁, E₂ and E₃ be the events of the boxes containing balls in Box I, Box II and Box III respectively then,

$\mathrm{P}\left({\mathrm{E}}_1\right)=\frac{1}{3}$, 

$\mathrm{P}\left({\mathrm{E}}_2\right)=\frac{1}{3}$,

$\mathrm{P}\left({\mathrm{E}}_3\right)=\frac{1}{3}$

Let E be the event that two balls picked are one red and one white.

$\mathrm{P}\left(\mathrm{E}\mid {\mathrm{E}}_1\right)=\frac{{}^1{\mathrm{C}}_1{}^2{\mathrm{C}}_1}{{}^6{\mathrm{C}}_1}=\frac{2}{15}$

$P\left(E\mid E_2\right)=\frac{{}^2{C}_1{}^3{C}_1}{{}^6{C}_2}=\frac{6}{15}$

$\mathrm{P}\left(\mathrm{E}\mid {\mathrm{E}}_3\right)=\frac{{}^3{\mathrm{C}}_1{}^1{\mathrm{C}}_1}{{}^6{\mathrm{C}}_2}=\frac{3}{15}$

Required probability = P(E₁|E)

P(E₁|E) = Probability that balls are one red and one white, and it is picked from Box I.

$=\frac{P\left(E_1\right)\cdot P\left(E\mid E_1\right)}{P\left(E_1\right)\cdot P\left(E\mid E_1\right)+P\left(E_2\right)\cdot P\left(E\mid E_2\right)+P\left(E_3\right)\cdot P\left(E\mid E_3\right)}$

$=\frac{\left(\frac{1}{3}\times \frac{2}{15}\right)}{\left(\frac{1}{3}\times \frac{2}{15}\right)+\left(\frac{1}{3}\times \frac{6}{15}\right)+\left(\frac{1}{3}\times \frac{6}{15}\right)}$

$=\frac{2}{2+6+3}=\frac{2}{11}$

Hence the required probability is 0.1.

4.0Practice Questions Based on Bayes Theorem

1. A car manufacturing company operates two plants, X and Y. Plant X produces 70% of the cars, while Plant Y produces 30%. Out of the cars manufactured, 80% at Plant X and 90% at Plant Y are rated as standard quality. If a car is selected at random and found to be of standard quality, what is the probability that it was produced by Plant X?
2. Bag I contains 3 red balls and 5 black balls, while Bag II contains 4 red balls and 6 black balls. A ball is randomly selected from one of the bags and is found to be red. What is the probability that this red ball was drawn from Bag II?
3. Consider three identical boxes—Box I, Box II, and Box III—each containing two coins. Box I contains two gold coins, Box II contains two silver coins, and Box III contains one gold coin and one silver coin. A person randomly selects a box and draws a coin, which turns out to be gold. What is the probability that the other coin in the same box is also gold?

5.0Sample Question on Bayes Theorem

1. What is the formula for Bayes' Theorem?

Ans: The formula is: $P(A\mid B)=\frac{P(B\mid A)\cdot P(A)}{P(B)}$