World Library  
Flag as Inappropriate
Email this Article

Completeness (statistics)

Article Id: WHEBN0000194618
Reproduction Date:

Title: Completeness (statistics)  
Author: World Heritage Encyclopedia
Language: English
Subject: Sufficient statistic, Robert V. Hogg, Statistics, List of statistics articles, Statistical theory
Collection: Statistical Terminology, Statistical Theory
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Completeness (statistics)

In statistics, completeness is a property of a statistic in relation to a model for a set of observed data. In essence, it is a condition which ensures that the parameters of the probability distribution representing the model can all be estimated on the basis of the statistic: it ensures that the distributions corresponding to different values of the parameters are distinct.

It is closely related to the idea of identifiability, but in statistical theory it is often found as a condition imposed on a sufficient statistic from which certain optimality results are derived.

Contents

  • Definition 1
    • Example 1: Bernoulli model 1.1
    • Example 2: Sum of normals 1.2
  • Relation to sufficient statistics 2
  • Importance of completeness 3
    • Lehmann–Scheffé theorem 3.1
    • Basu's theorem 3.2
    • Bahadur's theorem 3.3
  • Notes 4
  • References 5

Definition

Consider a random variable X whose probability distribution belongs to a parametric family of probability distributions Pθ parametrized by θ.

Formally, a statistic s is a measurable function of X; thus, a statistic s is evaluated on a random variable X, taking the value s(X), which is itself a random variable. A given realization of the random variable X(ω) is a data-point (datum), on which the statistic s takes the value s(X(ω)).

The statistic s is said to be complete for the distribution of X if for every measurable function g (which must be independent of θ) the following implication holds:[1]

E(g(s(X))) = 0 for all θ implies that Pθ(g(s(X)) = 0) = 1 for all θ.

The statistic s is said to be boundedly complete if the implication holds for all bounded functions g.

Example 1: Bernoulli model

The Bernoulli model admits a complete statistic.[2] Let X be a random sample of size n such that each Xi has the same Bernoulli distribution with parameter p. Let T be the number of 1s observed in the sample. T is a statistic of X which has a binomial distribution with parameters (n,p). If the parameter space for p is (0,1), then T is a complete statistic. To see this, note that

\operatorname{E}(g(T)) = \sum_{t=0}^n {g(t){n \choose t}p^{t}(1-p)^{n-t}} = (1-p)^n \sum_{t=0}^n {g(t){n \choose t}\left(\frac{p}{1-p}\right)^t} .

Observe also that neither p nor 1 − p can be 0. Hence E(g(T)) = 0 if and only if:

\sum_{t=0}^n g(t){n \choose t}\left(\frac{p}{1-p}\right)^t = 0.

On denoting p/(1 − p) by r, one gets:

\sum_{t=0}^n g(t){n \choose t}r^t = 0 .

First, observe that the range of r is all positive reals. Also, E(g(T)) is a polynomial in r and, therefore, can only be identical to 0 if all coefficients are 0, that is, g(t) = 0 for all t.

It is important to notice that the result that all coefficients must be 0 was obtained because of the range of r. Had the parameter space been finite and with a number of elements smaller than n, it might be possible to solve the linear equations in g(t) obtained by substituting the values of r and get solutions different from 0. For example, if n = 1 and the parametric space is {0.5}, a single observation, T is not complete. Observe that, with the definition:

g(t) = 2(t-0.5), \,

then, E(g(T)) = 0 although g(t) is not 0 for t = 0 nor for t = 1.

Example 2: Sum of normals

This example will show that, in a sample of size 2 from a normal distribution with known variance, the statistic X1+X2 is complete and sufficient. Suppose (X1, X2) are independent, identically distributed random variables, normally distributed with expectation θ and variance 1. The sum

s((X_1, X_2)) = X_1 + X_2\,\!

is a complete statistic for θ.

To show this, it is sufficient to demonstrate that there is no non-zero function g such that the expectation of

g(s(X_1, X_2)) = g(X_1+X_2)\,\!

remains zero regardless of the value of θ.

That fact may be seen as follows. The probability distribution of X1 + X2 is normal with expectation 2θ and variance 2. Its probability density function in x is therefore proportional to

\exp\left(-(x-2\theta)^2/4\right).

The expectation of g above would therefore be a constant times

\int_{-\infty}^\infty g(x)\exp\left(-(x-2\theta)^2/4\right)\,dx.

A bit of algebra reduces this to

k(\theta) \int_{-\infty}^\infty h(x)e^{x\theta}\,dx\,\!

where k(θ) is nowhere zero and

h(x)=g(x)e^{-x^2/4}.\,\!

As a function of θ this is a two-sided Laplace transform of h(X), and cannot be identically zero unless h(x) is zero almost everywhere.[3] The exponential is not zero, so this can only happen if g(x) is zero almost everywhere.

Relation to sufficient statistics

For some parametric families, a complete sufficient statistic does not exist. Also, a minimal sufficient statistic need not exist. (A case in which there is no minimal sufficient statistic was shown by Bahadur in 1957. ) Under mild conditions, a minimal sufficient statistic does always exist. In particular, these conditions always hold if the random variables (associated with Pθ ) are all discrete or are all continuous.

Importance of completeness

The notion of completeness has many applications in statistics, particularly in the following two theorems of mathematical statistics.

Lehmann–Scheffé theorem

Completeness occurs in the Lehmann–Scheffé theorem, which states that if a statistic that is unbiased, complete and sufficient for some parameter θ, then it is the best mean-unbiased estimator for θ. In other words, this statistic has a smaller expected loss for any convex loss function; in many practical applications with the squared loss-function, it has a smaller mean squared error among any estimators with the same expected value.

See also minimum-variance unbiased estimator.

Basu's theorem

Bounded completeness occurs in Basu's theorem,[4] which states that a statistic which is both boundedly complete and sufficient is independent of any ancillary statistic.

Bahadur's theorem

Bounded completeness also occurs in Bahadur's theorem. If a statistic is sufficient and boundedly complete, then it is minimal sufficient.

Notes

  1. ^ Young, G. A. and Smith, R. L. (2005). Essentials of Statistical Inference. (p. 94). Cambridge University Press.
  2. ^ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 285-286). Duxbury Press.
  3. ^ Orloff, Jeremy. "Uniqueness of Laplace Transform". 
  4. ^ Casella, G. and Berger, R. L. (2001). Statistical Inference. (pp. 287). Duxbury Press.

References

  •  
  •  
  •  
  • Lehmann, E.L.;  
  • Lehmann, E.L.;  
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 



Copyright © World Library Foundation. All rights reserved. eBooks from World eBook Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.