The Multinomial and Poisson Distributions

The Multinomial and Poisson Distributions

As you and your friend travel one thousand miles to watch a series of Barclays Premier League football, the two of you deliberate the probability distributions for the possible results over the five game series. Conversely, it hits you right in the face, at MDM4U in school, you learnt calculating probability distributions for binomial circumstances, but football always encompasses a draw. You could manipulate the probabilities and count a “draw” as a loss, but would that really be a representative probability distribution? Undoubtedly not, unless you or your friend were rooting for one team so badly, that any result against them counted as a loss.

In class, Bernoulli trials in connotation with binomial distributions were conferred. As a reminder, let us take the time to revisit Bernoulli trials and binomial distributions. Bernoulli trials are recurrent, independent, indistinguishable trials measured in terms of success or failure. Several examples of independent trials include tossing a coin one hundred times, rolling a pair of dice, or pass and fail situations. Binomial distributions are probability distributions where a definite outcome is anticipated, and where there are two possible results; success or failure. The probability of success is denoted as p, while the probability of failure is denoted as q. The sum of these two probabilities, must, in all situations be equal to one. Let us take an example; what is the probability of tossing exactly three heads out of five tosses? The first step is to determine the success and failure, along with the probabilities and desired number of successes and failures, respectively. Therefore, the probability of achieving exactly three heads in five tosses is 0.3125 (10/32).

Multinomial Formula

Going back to the football example, the situation of a draw is not covered by a binomial distribution, so, what do we do? Instead of using the binomial distribution, where there are two outcomes (success and failure), we proceed to using the formula for multinomial distribution, which allows infinite (n) outcomes, as long as the probabilities for (n-1) outcomes are defined, and the number of desired outcomes are also defined. Suppose you were attending the game series between Chelsea and Manchester United, the probability of Chelsea winning is 60%, while the probability of Manchester United winning is 25%. The probability of a draw is the remaining 15%. If the series was five games long, what is the probability of three Chelsea wins, one Manchester United win and one draw? And what is the number of expected Chelsea wins, Manchester United wins and draws in the five game series?

The first step towards solving this issue is to gather the formula, and identify all the values associated with it. The formula for multinomial distributions is similar to that of the binomial distributions, with the only difference being an added outcome, denoted as “r” or “n3”. In the football instance, n equals to five, because the game series is five matches long. The desired outcomes are three Chelsea wins (probability of Chelsea winning is 60%), one Manchester United win (probability of Manchester United winning is 25%),and one draw (probability of one match going for a draw is 15%). Subsequently, the values are to be entered into the formula and calculated based “on the order of operations”. After doing the above mentioned, the probability of Chelsea winning three games, Manchester United winning one game and one game stalemated, is attained, and is found to be 0.162 (or 16.2%).

However, only parts of your queries are answered. As a football fanatic, you still do not know how many games your team is expected to win, or how many draws are expected to be part of the series? Recall that the formula for obtaining the expected number of successes in a binomial distribution was simply n x p (where n was the number of times the experiment/event occurred, and p was the probability of success). Similarly, the expected value for the number of desired outcomes can be obtained by multiplying the number of events/experiments by the probability of the desired outcome.

In the situation of the football series, the probabilities are 60%, 25% and 15% respectively, while the number of games occurring within the series is five. As a result, the expected values for the series are three Chelsea wins, one Manchester United win, and one draw. In summary, situations where the number of outcomes are greater than two require more than a binomial distribution, hence the multinomial distribution is an effective manner in which the probabilities for situations that include more than two possible outcomes. Similarly, the expected value can also be calculated for these situations. The multinomial distribution can easily be used in situations like chess, stock prices, sporting events, and any other situation involving more than two possible outcomes. However, what about situations in which rates are involved?

Expected ValueOver a century ago, French mathematician Simeon-Denis Poisson developed, what is known today as the Poisson distribution. There are several uses for the Poisson distribution. For instance, this style of distribution can be used to approximate the binomial distribution if the value of p is miniscule, but the value of n is extremely large. However, the more common use of the Poisson distribution is when an average rate of occurrence is given. It is especially used when the mean number of successes (predetermined outcome) is given, and the probabilities of various numbers of successes are required. By definition, the Poisson distribution is a discrete probability distribution for the counts of events that occur randomly in a given interval of time (or space). The Poisson distribution only works if the events are independent. The formula for the Poisson distribution is shown below:

Poisson Distribution Formula

With regards to the Poisson distribution, the mean is equal to λ, while the standard deviation is equal to the square root of the mean. As an example, you were tasked with calculating the probability of observing 10 births in a given hour at the hospital, with the average rate of births per hour at the hospital being only 3. The first step is to provide a let statement, that x equals the number of births in a given hour. Consequently, a Poisson distribution statement must be written. The general formula for that is X – Po(λ). Therefore, the general statement for this instance is X – Po(3). The mean of the births per hour at the hospital is 3, while x is 10. After inputting all the information into the formula, you would realize that the probability of the desired outcome is 0.08%.

Poisson Distribution Example

In summation, where the binomial distribution is unable to provide probabilities for situations, the multinomial distribution can be used when the number of outcomes exceeds two, and the Poisson distribution can be used when rates are provided, or the probability is too small and the sample is too big to use the binomial distribution.

The Chi-Square Distribution

 The normal distribution was looked at when assessing situations, hypotheses and probabilities. However, the X2 distribution, known as the chi-square distribution is just as effective as the normal distribution. The X2 (chi square, pronounced kai-square) distribution is a continuous probability distribution that is widely used in statistical inference. In fact, it is closely related to the standard normal distribution; in that if a random variable Y has the standard normal distribution then Y2 has an X2  distribution with exactly one degree of freedom. Similarly, if Z1, Z2, …, Zk are independent standard normal variables, thenZ12 +  Z22 …+ Zk2  has an X2 distribution with exactly k degrees of freedom. For the X2 distribution, the mean is equal to the degrees of freedom, while the variance is twice the degrees of freedom. As you see a visual representation, you will notice that as the degrees of freedom increase, the X2distribution will start looking like the normal distribution. Furthermore, the X2 function with one degree of freedom is largely positively skewed, but as the degrees of freedom increases, the skew-ness of the graph begins decreasing. The chi-square is a statistical test, which is commonly used to substantiate the null hypothesis, or to affirm the alternate hypothesis. It is often used when trying to determine the “goodness of fit” between the observed and recorded pieces of data and the expected outcomes (empirical vs. theoretical). Should there be a large difference between the expected and actual data; a large Chi-square value will result, indicating that there is a significant difference between the original hypothesis and the actual data. Before I lose you as a reader in this annihilation of words, (I hope I still have your undivided attention at this point), let me show you an example pertaining to the goodness of fit for a binomial distribution.

In several instances, we can use anX2  test to verify or disprove the null hypothesis that the data comes from a specific parametric distribution (e.g. Binomial, Poisson, normal). As an example, suppose that a sports journalist claims that Michael Jordan’s free throw successes follow a binomial distribution with a success rate of 80%. Assume that the observed data over two seasons is as shown in the table below (take into consideration that the total number of free throw pairs is 338, and that each attempt consists of two free throws).

Table 1

In words, on five occasions Michael Jordan missed both free throws during the two seasons, while scoring one out of two free throws on 82 occasions. He did also score both free throws on 251 occasions. In this instance, the H0 is that Michael Jordan’s number of successes on two free throws follows a binomial distribution with a success rate of 80%; while the alternative hypothesis claims that the distribution differs from the null hypothesis in some manner. There are two situations in which the null hypothesis could be discredited; a) binomial distribution is reasonable, but the probability is wrong, or b) the binomial distribution is incorrect because free throws are not independent, but the probability of success is corrected. Now, let us calculate the expected proportion and the expected numbers if the null hypothesis was true.

Table 2

Through the table shown above, we can see that there are several occasions where the expected value differs from the observed value. Based on the hypothesis, it was expected that Michael Jordan would only make 216.32 pairs of free throws; however, in reality he made 251. Similarly, the values between the expected number (based on the hypothesis) and the theoretical values differ. Through the naked eye, it is easy to point out that differences are witnessed in the two sets of data. However, is the difference statistically different? Can we reject the null hypothesis? Instead of using the normal distribution, we proceed to test the hypothesis using the chi-square method.

The formula for determining the chi-squared is as follows: the values that are observed are subtracted by the values that are expected, and then squared. The expected number then divides that expression. The formula is applied over all cells (in this case, 0, 1, 2 makes).

Through this, we can see that the chi-squared test statistic is equal to 17.26. Subsequently, the “p” value must be obtained. To begin with, the degrees of freedom are required and are obtained by subtracting one from the total number of cells (in this case, 3). That makes the division of freedom equal two. There are two ways in which we can calculate the “p” value; we can either draw the chi-square distribution with a degree of freedom of two, or then look for the values based on our chi-squared test statistic, or we can use an online calculator. Take into consideration that the larger the value of the test statistic, the greater the evidence against the null hypothesis. The “p” value is the probability of attaining values greater than the chi-squared test statistic (to the right of the value, on the graph). The larger the value, the harder to prove the hypothesis insignificant and inaccurate. In this instance, the “p” value generated by an online calculator is 0.00017866; it indicates a strong rejection towards the null hypothesis. Because we have obtained incriminating evidence that the hypothesis is not true, our work remains unfinished. As stated above, there are two possibilities in which the hypothesis can be incorrect, and it is necessary to test which of the possibilities is the underlying reason why the null hypothesis was rejected.

In practical situations, the ambiguous nature of the parameters is prevalent, and often, the available data is required to obtain these parameters. So for this case, let us assume that the parameters are not provided, and that we have to form them. For now, the null hypothesis can be reworded to state, “Michael Jordan’s number of successes on two free throws follows a binomial distribution” (notice that the statement does not contain any success values). Let us then, estimate the probability that he made free throws.

Table 3

After calculating the expected numbers based on the estimated probability that he made sets of free throws, we can immediately notice the close nature of the expected and observed count of Michael Jordan’s free throw shooting over two seasons. Similar to the previous situation, we must calculate the chi-squared test statistic based on the new set of values. In this instance, the value of the test statistic is 0.34. Once again, the “p” value is required to determine whether the null hypothesis can be accepted or not. Therefore, there is a need to determine the degrees of freedom for the data set. While it may be quite enticing to say that the degree of freedom is two, there is a slight catch. Because we estimated the initial parameter for the data (finding p = 0.864), we have lost an extra degree of freedom. Therefore, the degree of freedom for this instance is one. After using the online calculator to find “p”, or plotting the chi-square distribution with one degree of freedom, you will find that the value of p (area under the curve to the right of 0.34) is approximately 0.56. Since this is a large p-value, it indicates a strong approval for the null hypothesis. In conclusion, the null hypothesis was tested and approved as noticed through the large p-value because there is no evidence that Jordan’s true distribution of successful free throws differs from the binomial distribution.

In this way, the chi-squared method of either accepting or rejecting hypotheses is quite useful, given a set of data and a null hypothesis. The nature of the chi-squared method is similar to the other methods of verifying hypotheses, and offers an alternative manner to prove the same point.