World Library  
Flag as Inappropriate
Email this Article

Density estimation

Article Id: WHEBN0000554671
Reproduction Date:

Title: Density estimation  
Author: World Heritage Encyclopedia
Language: English
Subject: Histogram, Kernel density estimation, Discretization of continuous features, Vector quantization, Outline of statistics
Collection: Estimation of Densities, Nonparametric Statistics
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Density estimation

Demonstration of density estimation using kernel smoothing: The true density is mixture of two Gaussians centered around 0 and 3, shown with solid blue curve. In each frame, 100 samples are generated from the distribution, shown in red. Centered on each sample, a Gaussian kernel is drawn in gray. Averaging the Gaussians yields the density estimate shown in the dashed black curve.

In probability and statistics, density estimation is the construction of an estimate, based on observed data, of an unobservable underlying probability density function. The unobservable density function is thought of as the density according to which a large population is distributed; the data are usually thought of as a random sample from that population.

A variety of approaches to density estimation are used, including Parzen windows and a range of data clustering techniques, including vector quantization. The most basic form of density estimation is a rescaled histogram.

Contents

  • Example of density estimation 1
    • Script for example 1.1
  • See also 2
  • References 3
  • External links 4

Example of density estimation

We will consider records of the incidence of diabetes. The following is quoted verbatim from the data set description:

A population of women who were at least 21 years old, of [2][1]

In this example, we construct three density estimates for "glu" (plasma glucose concentration), one conditional on the presence of diabetes, the second conditional on the absence of diabetes, and the third not conditional on diabetes. The conditional density estimates are then used to construct the probability of diabetes conditional on "glu".

The "glu" data were obtained from the MASS package[3] of the R programming language. Within R, ?Pima.tr and ?Pima.te give a fuller account of the data.

The mean of "glu" in the diabetes cases is 143.1 and the standard deviation is 31.26. The mean of "glu" in the non-diabetes cases is 110.0 and the standard deviation is 24.29. From this we see that, in this data set, diabetes cases are associated with greater levels of "glu". This will be made clearer by plots of the estimated density functions.

The first figure shows density estimates of p(glu | diabetes=1), p(glu | diabetes=0), and p(glu). The density estimates are kernel density estimates using a Gaussian kernel. That is, a Gaussian density function is placed at each data point, and the sum of the density functions is computed over the range of the data.

Estimated density of p (glu | diabetes=1) (red), p (glu | diabetes=0) (blue), and p (glu) (black)

From the density of "glu" conditional on diabetes, we can obtain the probability of diabetes conditional on "glu" via Bayes' rule. For brevity, "diabetes" is abbreviated "db." in this formula.

p(\mbox{diabetes}=1|\mbox{glu}) = \frac{p(\mbox{glu}|\mbox{db.}=1)\,p(\mbox{db.}=1)}{p(\mbox{glu}|\mbox{db.}=1)\,p(\mbox{db.}=1) + p(\mbox{glu}|\mbox{db.}=0)\,p(\mbox{db.}=0)}

The second figure shows the estimated posterior probability p(diabetes=1 | glu). From these data, it appears that an increased level of "glu" is associated with diabetes.

Estimated probability of p(diabetes=1 | glu)

Script for example

The following R commands will create the figures shown above. These commands can be entered at the command prompt by using cut and paste.

library(MASS)
data(Pima.tr)
data(Pima.te)

Pima <- rbind (Pima.tr, Pima.te)
glu  <- Pima[, 'glu']

d0 <- Pima[, 'type'] == 'No'
d1 <- Pima[, 'type'] == 'Yes'
base.rate.d1 <- sum(d1) / (sum(d1) + sum(d0))

glu.density    <- density (glu)
glu.d0.density <- density (glu[d0])
glu.d1.density <- density (glu[d1])

approxfun(glu.d0.density$x, glu.d0.density$y) -> glu.d0.f
approxfun(glu.d1.density$x, glu.d1.density$y) -> glu.d1.f

p.d.given.glu <- function(glu, base.rate.d1)
{
    p1 <- glu.d1.f(glu) * base.rate.d1
    p0 <- glu.d0.f(glu) * (1 - base.rate.d1)
    p1 / (p0 + p1)
}

x <- 1:250
y <- p.d.given.glu (x, base.rate.d1)
plot(x, y, type='l', col='red', xlab='glu', ylab='estimated p(diabetes|glu)')

plot(density(glu[d0]), col='blue', xlab='glu', ylab='estimate p(glu), 
     p(glu|diabetes), p(glu|not diabetes)', main=NA)
lines(density(glu[d1]), col='red')

Note that the above conditional density estimator uses bandwidths that are optimal for unconditional densities. Alternatively, one could use the method of Hall, Racine and Li (2004)[4] and the R np package[5] for automatic (data-driven) bandwidth selection that is optimal for conditional density estimates; see the np vignette[6] for an introduction to the np package. The following R commands use the npcdens() function to deliver optimal smoothing. Note that the response "Yes"/"No" is a factor.

library(np)

fy.x <- npcdens(type~glu, nmulti=1, data=Pima)

Pima.eval <- data.frame(type=factor("Yes"),
                        glu=seq(min(Pima$glu), max(Pima$glu), length=250))
 
plot(x, y, type='l', lty=2, col='red', xlab='glu',
     ylab='estimated p(diabetes|glu)')
lines(Pima.eval$glu, predict(fy.x, newdata=Pima.eval), col="blue")
legend(0, 1, c("Unconditional bandwidth", "Conditional bandwidth"),
       col=c("red", "blue"), lty=c(2, 1))

The third figure uses optimal smoothing via the method of Hall, Racine, and Li[4] indicating that the unconditional density bandwidth used in the second figure above yields a conditional density estimate that may be somewhat undersmoothed.

Estimated probability of p (diabetes=1 | glu)

See also

References

  1. ^
  2. ^
  3. ^
  4. ^ a b
  5. ^
  6. ^

Sources

  • Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. New York: Springer, 2001. ISBN 0-387-95284-5. (See Chapter 6.)
  • Qi Li and Jeffrey S. Racine. Nonparametric Econometrics: Theory and Practice. Princeton University Press, 2007, ISBN 0-691-12161-3. (See Chapter 1.)
  • D.W. Scott. Multivariate Density Estimation. Theory, Practice and Visualization. New York: Wiley, 1992.
  • B.W. Silverman. Density Estimation. London: Chapman and Hall, 1986. ISBN 978-0-412-24620-3

External links

  • CREEM: Centre for Research Into Ecological and Environmental Modelling Downloads for free density estimation software packages Distance 4 (from Research Unit for Wildlife Population Assessment "RUWPA") and WiSP.
  • UCI Machine Learning Repository Content Summary (See "Pima Indians Diabetes Database" for the original data set of 732 records, and additional notes.)
  • Free MATLAB code for one and two dimensional density estimation
  • libAGF C++ software for variable kernel density estimation.
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 



Copyright © World Library Foundation. All rights reserved. eBooks from World eBook Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.