Fisher's exact test^{[1]}^{[2]}^{[3]} is a statistical significance test used in the analysis of contingency tables. Although in practice it is employed when sample sizes are small, it is valid for all sample sizes. It is named after its inventor, Ronald Fisher, and is one of a class of exact tests, so called because the significance of the deviation from a null hypothesis (e.g., Pvalue) can be calculated exactly, rather than relying on an approximation that becomes exact in the limit as the sample size grows to infinity, as with many statistical tests. Fisher is said to have devised the test following a comment from Dr Muriel Bristol, who claimed to be able to detect whether the tea or the milk was added first to her cup; see lady tasting tea.
Contents

Purpose and scope 1

Example 2

Controversies 3

Alternatives 4

See also 5

References 6

External links 7
Purpose and scope
The test is useful for categorical data that result from classifying objects in two different ways; it is used to examine the significance of the association (contingency) between the two kinds of classification. So in Fisher's original example, one criterion of classification could be whether milk or tea was put in the cup first; the other could be whether Dr Bristol thinks that the milk or tea was put in first. We want to know whether these two classifications are associated – that is, whether Dr Bristol really can tell whether milk or tea was poured in first. Most uses of the Fisher test involve, like this example, a 2 × 2 contingency table. The pvalue from the test is computed as if the margins of the table are fixed, i.e. as if, in the teatasting example, Dr Bristol knows the number of cups with each treatment (milk or tea first) and will therefore provide guesses with the correct number in each category. As pointed out by Fisher, this leads under a null hypothesis of independence to a hypergeometric distribution of the numbers in the cells of the table.
With large samples, a chisquared test can be used in this situation. However, the significance value it provides is only an approximation, because the sampling distribution of the test statistic that is calculated is only approximately equal to the theoretical chisquared distribution. The approximation is inadequate when sample sizes are small, or the data are very unequally distributed among the cells of the table, resulting in the cell counts predicted on the null hypothesis (the "expected values") being low. The usual rule of thumb for deciding whether the chisquared approximation is good enough is that the chisquared test is not suitable when the expected values in any of the cells of a contingency table are below 5, or below 10 when there is only one degree of freedom (this rule is now known to be overly conservative^{[4]}). In fact, for small, sparse, or unbalanced data, the exact and asymptotic pvalues can be quite different and may lead to opposite conclusions concerning the hypothesis of interest.^{[5]}^{[6]} In contrast the Fisher test is, as its name states, exact as long as the experimental procedure keeps the row and column totals fixed, and it can therefore be used regardless of the sample characteristics. It becomes difficult to calculate with large samples or wellbalanced tables, but fortunately these are exactly the conditions where the chisquared test is appropriate.
For hand calculations, the test is only feasible in the case of a 2 × 2 contingency table. However the principle of the test can be extended to the general case of an m × n table,^{[7]}^{[8]} and some statistical packages provide a calculation (sometimes using a Monte Carlo method to obtain an approximation) for the more general case.^{[9]}
Example
For example, a sample of teenagers might be divided into male and female on the one hand, and those that are and are not currently dieting on the other. We hypothesize, for example, that the proportion of dieting individuals is higher among the women than among the men, and we want to test whether any difference of proportions that we observe is significant. The data might look like this:

Men

Women

Row total

Dieting

1

9

10

Nondieting

11

3

14

Column total

12

12

24

The question we ask about these data is: knowing that 10 of these 24 teenagers are dieters, and that 12 of the 24 are female, and assuming the null hypothesis that men and women are equally likely to diet, what is the probability that these 10 dieters would be so unevenly distributed between the women and the men? If we were to choose 10 of the teenagers at random, what is the probability that 9 or more of them would be among the 12 women, and only 1 or fewer from among the 12 men?
Before we proceed with the Fisher test, we first introduce some notation. We represent the cells by the letters a, b, c and d, call the totals across rows and columns marginal totals, and represent the grand total by n. So the table now looks like this:

Men

Women

Row Total

Dieting

a

b

a + b

Nondieting

c

d

c + d

Column Total

a + c

b + d

a + b + c + d (=n)

Fisher showed that the probability of obtaining any such set of values was given by the hypergeometric distribution:
p = \frac{ \displaystyle \displaystyle }{ \displaystyle } = \frac{(a+b)!~(c+d)!~(a+c)!~(b+d)!}{a!~~b!~~c!~~d!~~n!}
where \tbinom nk is the binomial coefficient and the symbol ! indicates the factorial operator. With the data above, this gives:
p = { {\tbinom{10}{1}} {\tbinom{14}{11}} }/{ {\tbinom{24}{12}} } = \tfrac{10!~14!~12!~12!}{1!~9!~11!~3!~24!} \approx 0.001346076
The formula above gives the exact hypergeometric probability of observing this particular arrangement of the data, assuming the given marginal totals, on the null hypothesis that men and women are equally likely to be dieters. To put it another way, if we assume that the probability that a man is a dieter is P, the probability that a woman is a dieter is p, and we assume that both men and women enter our sample independently of whether or not they are dieters, then this hypergeometric formula gives the conditional probability of observing the values a, b, c, d in the four cells, conditionally on the observed marginals (i.e., assuming the row and column totals shown in the margins of the table are given). This remains true even if men enter our sample with different probabilities than women. The requirement is merely that the two classification characteristics—gender, and dieter (or not)  are not associated.
For example, suppose we knew probabilities P,Q,p,q with P+Q=p+q=1 such that (male dieter, male nondieter, female dieter, female nondieter) had respective probabilities (Pp,Pq,Qp,Qq) for each individual encountered under our sampling procedure. Then still, were we to calculate the distribution of cell entries conditional given marginals, we would obtain the above formula in which neither p nor P occurs. Thus, we can calculate the exact probability of any arrangement of the 24 teenagers into the four cells of the table, but Fisher showed that to generate a significance level, we need consider only the cases where the marginal totals are the same as in the observed table, and among those, only the cases where the arrangement is as extreme as the observed arrangement, or more so. (Barnard's test relaxes this constraint on one set of the marginal totals.) In the example, there are 11 such cases. Of these only one is more extreme in the same direction as our data; it looks like this:

Men

Women

Row Total

Dieting

0

10

10

Nondieting

12

2

14

Column Total

12

12

24

For this table (with extremely unequal dieting proportions) the probability is {p = {\tbinom{10}{0}} {\tbinom{14}{12}} }/{ {\tbinom{24}{12}} } \approx 0.000033652.
In order to calculate the significance of the observed data, i.e. the total probability of observing data as extreme or more extreme if the null hypothesis is true, we have to calculate the values of p for both these tables, and add them together. This gives a onetailed test, with p approximately 0.001346076 + 0.000033652 = 0.001379728. (For example, in the R statistical computing environment, this value can be obtained as fisher.test(rbind(c(1,9),c(11,3)), alternative="less")$p.value
. This value can be interpreted as the sum of evidence provided by the observed data—or any more extreme table—for the null hypothesis (that there is no difference in the proportions of dieters between men and women). The smaller the value of p, the greater the evidence for rejecting the null hypothesis; so here the evidence is strong that men and women are not equally likely to be dieters.
For a twotailed test we must also consider tables that are equally extreme, but in the opposite direction. Unfortunately, classification of the tables according to whether or not they are 'as extreme' is problematic. An approach used by the fisher.test function in R is to compute the pvalue by summing the probabilities for all tables with probabilities less than or equal to that of the observed table. In the example here, the 2sided pvalue is twice the 1sided value—but in general these can differ substantially for tables with small counts, unlike the case with test statistics that have a symmetric sampling distribution.
An example of the Fisher Exact Test used on a 2x3 matrix is provided here. This fictitious example varies high, middle, and low income with owning or not owning at least one dog. The example contains a pvalue calculator for a 2x3 matrix in which all of the work is shown. The formulas and rules used are the same as are used for the 2x2 matrix example. All possible matrices, while keeping the row and column sums the same as the original matrix, are calculated in the second sheet of the example. The pvalues for those matrices were calculated using the pvalue calculator. Finally, all pvalues less than or equal to the pvalue cutoff (the pvalue of the original matrix) are summed to create the final pvalue. Since the pvalue is smaller than the 0.05, the null hypothesis can be rejected and it can be decided that high, middle, and low income households do not have the same dogowning trends.
As noted above, most modern statistical packages will calculate the significance of Fisher tests, in some cases even where the chisquared approximation would also be acceptable. The actual computations as performed by statistical software packages will as a rule differ from those described above, because numerical difficulties may result from the large values taken by the factorials. A simple, somewhat better computational approach relies on a gamma function or loggamma function, but methods for accurate computation of hypergeometric and binomial probabilities remains an active research area.
Controversies
Despite the fact that Fisher's test gives exact pvalues, some authors have argued that it is conservative, i.e. that its actual rejection rate is below the nominal significance level.^{[10]}^{[11]}^{[12]} The apparent contradiction stems from the combination of a discrete statistic with fixed significance levels.^{[13]}^{[14]} To be more precise, consider the following proposal for a significance test at the 5%level: reject the null hypothesis for each table to which Fisher's test assigns a pvalue equal to or smaller than 5%. Because the set of all tables is discrete, there may not be a table for which equality is achieved. If \alpha_e is the largest pvalue smaller than 5% which can actually occur for some table, then the proposed test effectively tests at the \alpha_elevel. For small sample sizes, \alpha_e might be significantly lower than 5%.^{[10]}^{[11]}^{[12]} While this effect occurs for any discrete statistic (not just in contingency tables, or for Fisher's test), it has been argued that the problem is compounded by the fact that Fisher's test conditions on the marginals.^{[15]} To avoid the problem, many authors discourage the use of fixed significance levels when dealing with discrete problems.^{[13]}^{[14]}
Another early discussion revolved around the necessity to condition on the marginals.^{[16]}^{[17]} Fisher's test gives exact pvalues both for fixed and for random marginals. Other tests, most prominently Barnard's, require random marginals. Some authors^{[13]}^{[14]}^{[17]} (including, later, Barnard himself^{[13]}) have criticized Barnard's test based on this property. They argue that the marginal totals are an (almost^{[14]}) ancillary statistic, containing (almost) no information about the tested property.
Alternatives
An alternative exact test, Barnard's exact test, has been developed and proponents of it suggest that this method is more powerful, particularly in 2 × 2 tables. Another alternative is to use maximum likelihood estimates to calculate a pvalue from the exact binomial or multinomial distributions and reject or fail to reject based on the pvalue.
See also
References

^

^ Fisher, R.A. (1954).

^ Agresti, Alan (1992). "A Survey of Exact Inference for Contingency Tables". Statistical Science 7 (1): 131–153.

^ Larntz, Kinley (1978). "Smallsample comparisons of exact levels for chisquared goodnessoffit statistics". Journal of the American Statistical Association 73 (362): 253–263.

^ Mehta, Cyrus R; Patel, Nitin R; Tsiatis, Anastasios A (1984). "Exact significance testing to establish treatment equivalence with ordered categorical data". Biometrics 40 (3): 819–825.

^ Mehta, C. R. 1995. SPSS 6.1 Exact test for Windows. Englewood Cliffs, NJ: Prentice Hall.

^ Mehta C.R., Patel N.R. (1983). "A Network Algorithm for Performing Fisher's Exact Test in r Xc Contingency Tables". Journal of the American Statistical Association 78 (382): 427–434.

^ mathworld.wolfram.com Page giving the formula for the general form of Fisher's exact test for m × n contingency tables

^ Cyrus R. Mehta and Nitin R. Patel (1986). "ALGORITHM 643: FEXACT: a FORTRAN subroutine for Fisher's exact test on unordered r×c contingency tables". ACM Trans. Math. Softw. 12 (2): 154–161.

^ ^{a} ^{b} Liddell, Douglas (1976). "Practical tests of 2x2 contingency tables". The Statistician 25 (4): 295–304.

^ ^{a} ^{b} Berkson, Joseph (1978). "In dispraise of the exact test". Journal of Statistic Planning and Inference 2: 27–42.

^ ^{a} ^{b} D'Agostino, R. B., Chase, W., and Belanger, A. (1988). "The Appropriateness of Some Common Procedures for Testing Equality of Two Independent Binomial Proportions". The American Statistician 42 (3): 198–202.

^ ^{a} ^{b} ^{c} ^{d} Yates, F. (1984). "Tests of Significance for 2 x 2 Contingency Tables (with discussion)". Journal of the Royal Statistical Society, Series A 147 (3): 426–463.

^ ^{a} ^{b} ^{c} ^{d} Roderick J. A. Little (1989). "Testing the Equality of Two Independent Binomial Proportions". The American Statistician 43 (4): 283–288.

^ Cyrus R. Mehta and Pralay Senchaudhuri, "Conditional versus Unconditional Exact Tests for Comparing Two Binomials" (4 September 2003). Retrieved 20 November 2009 from http://www.cytel.com/Papers/twobinomials.pdf

^ Barnard, G.A (1945). "A New Test for 2×2 Tables". Nature 156 (3954): 177.

^ ^{a} ^{b} Fisher (1945). "A New Test for 2 × 2 Tables". Nature 156 (3961): 388.
External links

Calculate Fishers Exact Test Online

Fisher Exact Test calculator for Android devices
This article was sourced from Creative Commons AttributionShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, EGovernment Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a nonprofit organization.