What is Bayes' Theorem?

Bayes' Theorem

However, just for the sake of argument, let's say that you want to know what Bayes' formula is.

Let's use the same example, but shorten each event to its one letter initial, ie: A, B, C, and D instead of Aberations, Brochmailians, Chompieliens, and Defective.

P(D|B) is not a Bayes problem. This is given in the problem. Bayes' formula finds the reverse conditional probability P(B|D).

It is based that the Given (D) is made of three parts, the part of D in A, the part of D in B, and the part of D in C.

                       P(B and D)
   P(B|D) =  -----------------------------------------
              P(A and D)  + P(B and D)  + P(C and D)

Inserting the multiplication rule for each of these joint probabilities gives

                          P(D|B)*P(B)
   P(B|D) =  -----------------------------------------
              P(D|A)*P(A) + P(D|B)*P(B) + P(D|C)*P(C)

However, and I hope you agree, it is much easier to take the joint probability divided by the marginal probability. The table does the adding for you and makes the problems doable without having to memorize the formulas.

What is Conditional Probability?

Conditional Probability

Recall that the probability of an event occurring given that another event has already occurred is called a conditional probability.

The probability that event B occurs, given that event A has already occurred is

P(B|A) = P(A and B) / P(A)
This formula comes from the general multiplication principle and a little bit of algebra.

Since we are given that event A has occurred, we have a reduced sample space. Instead of the entire sample space S, we now have a sample space of A since we know A has occurred. So the old rule about being the number in the event divided by the number in the sample space still applies. It is the number in A and B (must be in A since A has occurred) divided by the number in A. If you then divided numerator and denominator of the right hand side by the number in the sample space S, then you have the probability of A and B divided by the probability of A.



Example 1:

The question, "Do you smoke?" was asked of 100 people. Results are shown in the table.
.

What is the probability of a randomly selected individual being a male who smokes? This is just a joint probability. The number of "Male and Smoke" divided by the total = 19/100 = 0.19
What is the probability of a randomly selected individual being a male? This is the total for male divided by the total = 60/100 = 0.60. Since no mention is made of smoking or not smoking, it includes all the cases.
What is the probability of a randomly selected individual smoking? Again, since no mention is made of gender, this is a marginal probability, the total who smoke divided by the total = 31/100 = 0.31.
What is the probability of a randomly selected male smoking? This time, you're told that you have a male - think of stratified sampling. What is the probability that the male smokes? Well, 19 males smoke out of 60 males, so 19/60 = 0.31666...
What is the probability that a randomly selected smoker is male? This time, you're told that you have a smoker and asked to find the probability that the smoker is also male. There are 19 male smokers out of 31 total smokers, so 19/31 = 0.6129 (approx)
After that last part, you have just worked a Bayes' Theorem problem. I know you didn't realize it - that's the beauty of it. A Bayes' problem can be set up so it appears to be just another conditional probability. In this class we will treat Bayes' problems as another conditional probability and not involve the large messy formula given in the text (and every other text).

Example 2:

There are three major manufacturing companies that make a product: Aberations, Brochmailians, and Chompielians. Aberations has a 50% market share, and Brochmailians has a 30% market share. 5% of Aberations' product is defective, 7% of Brochmailians' product is defective, and 10% of Chompieliens' product is defective.

This information can be placed into a joint probability distribution

The percent of the market share for Chompieliens wasn't given, but since the marginals must add to be 1.00, they have a 20% market share.

Notice that the 5%, 7%, and 10% defective rates don't go into the table directly. This is because they are conditional probabilities and the table is a joint probability table. These defective probabilities are conditional upon which company was given. That is, the 7% is not P(Defective), but P(Defective|Brochmailians). The joint probability P(Defective and Brochmailians) = P(Defective|Brochmailians) * P(Brochmailians).

The "good" probabilities can be found by subtraction as shown above, or by multiplication using conditional probabilities. If 7% of Brochmailians' product is defective, then 93% is good. 0.93(0.30)=0.279.

What is the probability a randomly selected product is defective? P(Defective) = 0.066
What is the probability that a defective product came from Brochmailians? P(Brochmailian|Defective) = P(Brochmailian and Defective) / P(Defective) = 0.021/0.066 = 7/22 = 0.318 (approx).
Are these events independent? No. If they were, then P(Brochmailians|Defective)=0.318 would have to equal the P(Brochmailians)=0.30, but it doesn't. Also, the P(Aberations and Defective)=0.025 would have to be P(Aberations)*P(Defective) = 0.50*0.066=0.033, and it doesn't.

What is Independence Revisited?

Independence Revisited

The following four statements are equivalent

• A and B are independent events
• P(A and B) = P(A) * P(B)
• P(A|B) = P(A)
• P(B|A) = P(B)

The last two are because if two events are independent, the occurrence of one doesn't change the probability of the occurrence of the other. This means that the probability of B occurring, whether A has happened or not, is simply the probability of B occurring.

What is General Multiplication Rule?

General Multiplication Rule

Always works.

P(A and B) = P(A) * P(B|A)
Example 4:

P(A) = 0.20, P(B) = 0.70, P(B|A) = 0.40

A good way to think of P(B|A) is that 40% of A is B. 40% of the 20% which was in event A is 8%, thus the intersection is 0.08.

What is Conditional Probability?

Conditional Probability

The probability of event B occurring that event A has already occurred is read "the probability of B given A" and is written: P(B|A)

What is Dependent Events?

Dependent Events

If the occurrence of one event does affect the probability of the other occurring, then the events are dependent.

What is Specific Multiplication Rule?

Specific Multiplication Rule

Only valid for independent events

P(A and B) = P(A) * P(B)
Example 3:

P(A) = 0.20, P(B) = 0.70, A and B are independent.

The 0.14 is because the probability of A and B is the probability of A times the probability of B or 0.20 * 0.70 = 0.14.

What is Independent Events?

Independent Events

Two events are independent if the occurrence of one does not change the probability of the other occurring.

An example would be rolling a 2 on a die and flipping a head on a coin. Rolling the 2 does not affect the probability of flipping the head.

If events are independent, then the probability of them both occurring is the product of the probabilities of each occurring.

What is Interpreting the table?

Interpreting the table

Certain things can be determined from the joint probability distribution. Mutually exclusive events will have a probability of zero. All inclusive events will have a zero opposite the intersection. All inclusive means that there is nothing outside of those two events: P(A or B) = 1.

What is General Addition Rule?

General Addition Rule

Always valid.

P(A or B) = P(A) + P(B) - P(A and B)
Example 2:

Given P(A) = 0.20, P(B) = 0.70, P(A and B) = 0.15

What is Non-Mutually Exclusive Events?

Non-Mutually Exclusive Events

In events which aren't mutually exclusive, there is some overlap. When P(A) and P(B) are added, the probability of the intersection (and) is added twice. To compensate for that double addition, the intersection needs to be subtracted.

What is Specific Addition Rule?

Specific Addition Rule

Only valid when the events are mutually exclusive.

P(A or B) = P(A) + P(B)
Example 1:

Given: P(A) = 0.20, P(B) = 0.70, A and B are disjoint

I like to use what's called a joint probability distribution. (Since disjoint means nothing in common, joint is what they have in common -- so the values that go on the inside portion of the table are the intersections or "and"s of each pair of events). "Marginal" is another word for totals -- it's called marginal because they appear in the margins.

The values in red are given in the problem. The grand total is always 1.00. The rest of the values are obtained by addition and subtraction.

What is Mutually Exclusive Events?

Mutually Exclusive Events

Two events are mutually exclusive if they cannot occur at the same time. Another word that means mutually exclusive is disjoint.

If two events are disjoint, then the probability of them both occurring at the same time is 0.

Disjoint: P(A and B) = 0
If two events are mutually exclusive, then the probability of either occurring is the sum of the probabilities of each occurring.

What is Probability Rules?

Probability Rules

There are two rules which are very important.

• All probabilities are between 0 and 1 inclusive   0 <= P(E) <= 1
• The sum of all the probabilities in the sample space is 1
• There are some other rules which are also important.
• The probability of an event which cannot occur is 0.
• The probability of any event which is not in the sample space is zero.
• The probability of an event which must occur is 1.
• The probability of the sample space is 1.
• The probability of an event not occurring is one minus the probability of it occurring.

P(E') = 1 - P(E)

What is Empirical Probability?

Empirical Probability

Empirical probability is based on observation. The empirical probability of an event is the relative frequency of a frequency distribution based upon observation.

P(E) = f / n

What is Classical Probability?

Classical Probability

The above table lends itself to describing data another way -- using a probability distribution. Let's consider the frequency distribution for the above sums.

If just the first and last columns were written, we would have a probability distribution. The relative frequency of a frequency distribution is the probability of the event occurring. This is only true, however, if the events are equally likely.

This gives us the formula for classical probability. The probability of an event occurring is the number in the event divided by the number in the sample space. Again, this is only true when the events are equally likely. A classical probability is the relative frequency of each event in the sample space when each event is equally likely.

P(E) = n(E) / n(S)

What is Sample Spaces?

Sample Spaces

A sample space is the set of all possible outcomes. However, some sample spaces are better than others.

Consider the experiment of flipping two coins. It is possible to get 0 heads, 1 head, or 2 heads. Thus, the sample space could be {0, 1, 2}. Another way to look at it is flip { HH, HT, TH, TT }. The second way is better because each event is as equally likely to occur as any other.

When writing the sample space, it is highly desirable to have events which are equally likely.

Another example is rolling two dice. The sums are { 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 }. However, each of these aren't equally likely. The only way to get a sum 2 is to roll a 1 on both dice, but you can get a sum of 4 by rolling a 1-3, 2-2, or 3-1. The following table illustrates a better sample space for the sum obtain when rolling two dice.