Math 365, Elementary Statistics 

Lesson 3 : ProbabilityIntroductionProbability to a statistician is the probability of the occurrence of an event. To an ordinary person it is the quantified chance of occurrence of that event. Some of the early theory of probability originated in gambling and later theories developed in bioscience. We get very tempted when we see somebody win $1 million in a lottery, but lottery operators design their games and machines in such a way that they will make more money than they give, in the long run.

S=  (1,1)  (1,2)  (1,3)  (1,4)  (1,5)  (1,6)  
(2,1)  (2,2)  (2,3)  (2,4)  (2,5)  (2,6)  
(3,1)  (3,2)  (3,3)  (3,4)  (3,5)  (3,6)  
(4,1)  (4,2)  (4,3)  (4,4)  (4,5)  (4,6)  
(5,1)  (5,2)  (5,3)  (5,4)  (5,5)  (5,6)  
(6,1)  (6,2)  (6,3)  (6,4)  (6,5)  (6,6) 
In brace notation, we can write
S = {(i,j) : i = 1,2,3,4,5,6 and j = 1,2,3,4,5,6}.
5. Suppose your experiment is to determine the number of road accidents
in Lawrence on a particular day. So, the sample space is S = {0,1,2,3
... }.
6. Suppose the experiment is to determine the sex of an unborn chlid.
Then the sample space is S = {Female, Male}.
7. Suppose your experiment is to determine the blood group of a patient
in a lab. Then the sample space is S = {O,A,B,AB}.
8. Suppose your experiment is to observe the annual wheat production
in Kansas. Then the sample space is S={x :
x is a nonnegative Number} = {x ∈ R
: x ≥ 0} =[0, ∞).
Definition. The sample space S is called a finite sample space if S has only a finite number of outcomes. If S has infinite elements, it is called an infinite sample space. Note that examples 1, 2, 3, 4, 6, and 7 above have finite sample spaces, and 5 and 8 have infinite sample space.
Events
Definitions. Given an experiment and its sample space S, the following are important definitions.
E ⊆ S.
Remark. Often, we will describe events in "English," and we may have to identify them as a subset of the sample space and also conversely.
Examples. The following are some examples
of events.
The Theory of Probability
Given a sample space S, in the MATHEMATICS of probability we have rules for how to compute the probability of an event E. Although the MATHEMATICS of probability was inspired by the empirical concept of probability, we do not derive anything from our intuitive ideas. We are guided by the precise rules and laws that we set up.
For now we will be dealing with finite sample spaces.
Definition. Let
S = { e_{1}, e_{2}, ... ,e_{n} }.
be a finite sample space. The probability of a simple event {e} is a number (possibly given) denoted by P({e}) which has the following properties:
P(E)=  ∑  P({e}) 
_{e ∈ E} 
Remark. If we know the probabilities P({e})
of all the simple events {e}, we will be able to compute the probability
of any event E using 3. The probabilities of the simple events will
Probability with Equally Likely Outcomes
One of the most frequently used models to compute probabilities of
simple events is called EQUALLY LIKELY OUTCOMES.
Definition. Let S
= {e_{1}, ... , e_{N}} be a finite sample space. We
say that all the outcomes are equally likely
if all the outcomes have the same probability. So, in this case, we
have
P({e_{1}}) = P({e_{2}}) = … = P({e_{N}}) = 1/N.
Also, in this case, for an event E
P(E) =  ∑  P({e}) =  ∑  1/N 
_{e ∈ E}  _{e ∈ E} 
=(Number of Outcomes in E)/(Number of Outcomes in S)
If n(E) denotes the number of outcomes in E then
P(E) =  n(E)
n(S) 
. 
Problems on 3.2
Probability of Simple Events Given in a Table
Exercise 3.2.1. The following table gives the blood group distribution of a certain population.
Blood Group Distribution  
Blood Group  O  A  B  AB 

Percentage of Population 
47  42  8  3 
Find the probability that a random sample of blood will be of Blood
Group A or B or AB. (Here S={O, A, B, AB}
and we want to compute the probability P(E) of the event E={A, B, AB}.
Solution
Exercise 3.2.2. A student wants to pick a school based on its grade distribution. Following is the most recent grade distribution in a school:
Grade Distribution Unreal Data 

Grades  A  B  C  D  F 

Percentage of Students 
19  33  31  14  3 
Find the probability that a randomly picked student will have at least
a B average.
Solution
Exercise 3.2.3. The following table gives the probability distribution of a loaded die.
Probability Distribution for a Die  
Face  1  2  3  4  5  6 

Probability  0.20  0.15  0.15  0.10  0.05  0.35 
Find the probability that the face 2 or 3 or 6 will show up when you
roll the die.
Solution
Exercise 3.2.4. An urn contains 7 apples and 3 oranges and 5 pears. One piece of fruit is picked at random. Find the probability that
Exercise 3.2.5. A die is rolled twice. Find the probability that
Exercise 3.2.6. A letter is chosen at random from the letters of the English alphabet. Find the probability that
Notations from Set Theory
Following are a few notations from the set theory, which we will be using in the context of sample spaces and events.
Notations. Let S be a set and E, F be two subsets of S.
E ∪ F = {x ∈ S : x ∈ E
or x ∈ F}.
So, if you put together the elements of E and F in a single collection, you get the union E ∪ F.
E ∩ F = {x ∈ S : both x ∈ E and x ∈ F}.
So, if you take all the elements common to both E and F, you get the intersection of E and F.
E^{c} = {x ∈ S : x ∉ E}.
So, the complement E^{c} of E is the collection of all the elements in S that are not in E.
Remark. If we can understand and interpret
the above definitions in our context of sample spaces and events, that
is adequate. For us, S will be a fixed sample space and E,F will be
events.
E ∪ F = E or F.
E ∩ F = E and F.
E ^{c} = (not E)
Laws of Probability
Following are some of the laws of probability.
First, probability behaves like area and the laws of probability are like that of area.
Some formulas and definitions: Let S be sample space and let E and F be two events.
P(E ∪ F) = P( E or F) = P(E)+P(F)
P(E ∩ F)
= P(E) + P(F) P(E and F)
We subtract P(E ∩ F) because we counted it twice: once in
P(E) and once in P(F).
if E and F are mutually exclusive then
P(E ∪ F) = P(E) + P(F)
P(E^{c}) = 1  P(E).
P(E) = a/(a+b)
Remark: This concept of ODDS is used often in gambling. When the odds in favor of a horse are 2 to 3, essentially this means that the probability the horse will win is 2/5. We say "essentially" because in actual betting, the probability is actually slightly less than 2/3, so that in the long run the gambling establishment makes more money than it gives. (This instructor is not particularly experienced in such betting or horse races.)
Problems on 3.3: Laws of Probability
Exercise 3.3.1. Let E, F, G be three events. It is given
P(E)=0.3
P(F)=0.7 P(G)=0.6
P( E ∩ F) = 0.2 P( E ∪ G) = 0.7
Exercise 3.3.2. Let E, F, G be events .
Exercise 3.3.3. The probability that a Christmas
tree is taller than 6 feet is .30; the probability that a Christmas
tree weighs more than sixty pounds is 0.25; and the probability that
a Christmas tree is either taller than 6 feet or more than sixty pounds
is .4.
Exercise 3.3.4. The probability that a student majors in liberal arts is .44; the probability that a student majors in business is .33; and the probability that a student majors in either liberal arts or business is .65. Find the probabilities
Counting techniques are important and useful to learn. You might like to know, for example,
Notations. Let n be a positive integer. Then the n! (read as factorial n) is defined as
n!= 1 . 2 . … (n2) . (n1)
n
0!=1.
Factorial n is the product of all integers from 1 up to n.
One of the main tools for such counting is the following principle:
The Basic Counting Principle. Suppose we have an experiment that is a combination of r subexperiments, performed one after the other, such that
Then our original experiment will have n_{1}n_{2} ... n_{r} outcomes.
Remark. Here we have used the word "experiment"
in a slightly different sense than the statistical experiments. The
basic counting principle will be used to count the number of outcomes
in sample spaces and events.
Examples.
3.4.1. Count the number of words of length four that you can construct from the English alphabet. Answer: 26x 26x26x26
We use the counting principle by splitting this experiment into four subexperiments:
Stage  Job to do  Number of Ways 
1.  Pick the first letter  26 
2.  Pick the 2nd letter  26 
3.  Pick the 3rd letter  26 
4.  Pick the 4th letter  26 
Answer = Product =  456976 
3.4.2. Count the number of ways you can assign the 11 seats in the first
row in a concert hall to 231 guests.
Stage  Job to do  Number of Ways 
1.  Assign seat 1  231 
2.  Assign seat 2  230 
3.  Assign seat 3  229 
4.  Assign seat 4  228 
5.  Assign seat 5  227 
6.  Assign seat 6  226 
7.  Assign seat 7  225 
8.  Assign seat 8  224 
9.  Assign seat 9  223 
10.  Assign seat 10  222 
11.  Assign seat 11  221 
Answer = Product =  221*222*...*230*231 
3.4.3. Contrast: How many ways can you form
a committee of 11 members from a group of 231 people? Unlike assigning
seats, here the order of selection of the members will be ignored. The
11 members, when permuted around, will have different seat assignments
but in the same committee. Forming the committee is a "combination"
problem that comes below.
Remark. The difference between assigning 11
seats in a row and forming a committee of 11 is that in the first case
the order of assignment is important. Assigning
the first row to the same 11 guests in two different ways will count
as two different outcomes. When we form a committee, the order in which
we pick 11 members does not make any difference.
Definition. Suppose we have n objects. We
pick r of them one by one (without ever puttting them back) and arrange
them in a row. Such an ordered arrangement will be called a permutation
of n objects taken r at a time. The number of permutations
of n objects taken r at a time is denoted by _{n}P_{r}.
It follows from the basic counting principle that
_{n}P_{r}
= n (n1) (n2) ... (nr+1) = n!/(n r)!
Number of permutations _{n}P_{r}
= product of r integers starting from n
downward.
In contrast, we can pick r objects from a collection of n objects one by one but place the object back in the collection before the next pick, and arrange all of them in a row. Such selection and arrangement is called picking with replacment. Constructing a formal word of length 4 is an experiment of picking with replacement.
Remark: Example 3.1 is a problem on picking with replacement because a letter can be selected more that once. Example 3.2 is a permutation problem.
Definition. Suppose we have n objects in a
container. We pick r of them all at a time. In this case the order of
selection does not come into consideration. Such a selection is called
a combination of n
objects taken r at a time. The number of
combinations of n objects taken r at a time is denoted by _{n}C_{r}
and is given by
_{n}C_{r} =  n! 
(r! (nr)!) 
Examples. 1. Count the number of ways you
can form a committee of 11 from a group of 231 people. Answer: 231C_{11}
2. Count the number of ways you can deal a hand of 13 cards from a deck
of 52 cards. Answer: 52C_{13}.
Problems on 3.4: Counting Techniques and Probability
Exercise 3.4.1. Find 5!
Solution
Exercise 3.4.2. A homeowner would like to
install a new storm door. The local store offers 2 brand names; each
brand has 4 different styles and 3 colors. How many choices does the
homeowner have?
Solution
Exercise 3.4.3. Suppose in the World Cup soccer tournament, group A has 8 teams. Each team of group A has to play all the other teams in the group. How many games will be played among the group A teams. Answer: 8^{C}_{2}
Exercise 3.4.4. How many ways can you deal a hand of 13 cards from a deck of 52 cards? Answer: 52^{C}_{13}
Exercise 3.4.5. How many ways can you deal
a hand of 4 spades, 3 hearts, 3 diamonds, and 3 clubs?
Solution
Solutionvariation
Exercise 3.4.6. We have 13 students in a class. How many ways can we assign the 4 seats in the first row? Solution
Exercise 3.4.7. Programming languages sometimes use a hexadecimal system (also called "hex") of numbers. In this system, 16 digits are used and denoted by 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F. Suppose you form a 6digit number in a hexadecimal system.
Solution
Here the sample space is the collection of all the 5digit hex numbers.
Exercise 3.4.8. You are playing Bridge and you are dealt a hand of 13 cards.
Exercise 3.4.9. A committee of 9 is selected at random from a group of 11 students, 17 mothers and 13 fathers.
Exercise 3.4.10. Three scholarships of unequal
value will be awarded from a group of 35 applicants. How many ways can
such a selection be made?
Solution
Sometimes when new information becomes available, the probability
of an event may have to be reevaluated in light of this new information.
Suppose we have a sample space S and an event E. Now suppose we have
new information that an event C has occurred. We will have to reevaluate
the conditional probability of E given that C has
occurred. The conditional probability of E given that C has occurred
is denoted by P(EC). Clearly, P(EC) may
be different from P(E). In fact, now that C has occurred, our old sample
space is no longer relevant. And C assumes the role of the new sample
space.
Example. Suppose we pick a KU student at random
and let E be the event that the student is taller than 6 feet. Then
we have the following observations.
P(E) =  number of KU students who are taller
than 6 feet
Total number of KU students 
=  n(E)
n(S) 
. 
= P(EC) =  number of MALE students who are taller
than 6 feet
Total number of male KU students 
=  n(E∩C)
n(C) 
. 
P(EC) =  n(E∩C)/n(S)
n(C)/n(S) 
=  P(E∩C)
P(C) 
. 
Based on the above example, we give the following definition and formula.
Definition. Let S be a sample space and E, C be two events.
P(EC) =  P(E∩C)
P(C) 
if P(C) ≠ 0.
P(E∩C) = P(EC)P(C). 
Independent Events
If the conditional probability P(EF) = P(E) the "simple" probability, then we say that E and F are independent. In this case,
P(E∩F) = P(E)P(F).
Definition. We say that two events E and F are independent if
P(E∩F) = P(E)P(F).
If two events are not independent, then they are said to be dependent.
Remark. Let us also describe what we mean by independence of 3 or more events. For events E_{1},E_{2}, … , E_{n}, we say they are independent if the "multiplication rule" applies. For example E,F,G,H are independent if all of the following holds:
2 events
P(E∩F) = P(E)P(F),
P(E∩G) = P(E)P(G),
P(E∩H) = P(E)P(H), P(F∩G) = P(F)P(G),
P(F∩H) = P(F)P(H), P(G∩H) = P(G)P(H)
3 events
P(E∩F∩G)
= P(E)P(F)P(G), P(E∩F∩H) = P(E)P(F)P(H),
P(E∩G∩H) = P(E)P(G)P(H), P(F∩G∩H) = P(F)P(G)P(H)
4 events
P(E∩F∩G∩H) = P(E)P(F)P(G)P(H)
Problems on 3.5: Conditional Probability and Independent Events
Exercise 3.5.1. Let A, B be two events. Given that
P(A) = .66  P(A ∩ B) = .11 
Find P(BA).
Solution
Exercise 3.5.2. Given
P(AB) = .8  P(B) = .1 
Find P(A∩B).
Solution
Exercise 3.5.3. In a certain county, the
probability that a person took a flu shot is .45 and the probability
that a person will get flu, given that he/she took a flu shot is .06.
What is the probability that a randomly selected person took a flu shot
and will get flu?
Solution
Exercise 3.5.4. Consider the following two
circuit diagrams:
Circuit 1 
Circuit 2 
For each of the two circuits do the following:
As you can see, current flows through two switches A and B to the radio
and back to the battery. It is given that the probability that the switch
A is closed is 0.91 and the probability that the switch B is closed
is 0.83. Assume that the two switches function independently. Find the
probability that the radio is playing.
Solution
Exercise 3.5.5. An airplane has two engines. The probability that engine 1 fails is 0.023 and the probability that engine 2 fails is 0.06. Assume that the engines function independently.
Exercise 3.5.6. Following are data from a hospital emergency room:
What is the conditional probability that a patient in the emergency
room will survive, given that he/she has health insurance.
Solution
Exercise 3.5.7. The probability that you
will receive a wrong number call this week is 0.3; the probability that
you will receive a sales call this week is 0.8; and that the probability
that you will receive a survey call this week is 0.5. What is the probability
that you will receive one of each this week? (Assume that all these
calls are independent.)
Solution