ch.4 Nonparametric Techniques

161p

4.1 Introduction

nonparametric procedures (with arbitrary distribution and without the assumption that the forms of the underlying densities are known)

1) estimating the density functions from sample patterns -> designing the classfier
2) directly estimating the a posteriori probability -> the nearest-neighbor rule -> decision functions

4.2 Density Estimation

http://en.wikipedia.org/wiki/Density_estimation
the construction of an estimate, based on observed data, of an unobservable underlying probability density function. The unobservable density function is thought of as the density according to which a large population is distributed; the data are usually thought of as a random sample from that population


i.i.d. = Independent and identically-distributed random variables
http://en.wikipedia.org/wiki/Iid
In probability theory, a sequence or other collection of random variables is independent and identically distributed (i.i.d.) if each has the same probability distribution as the others and all are mutually independent.


http://en.wikipedia.org/wiki/Binomial_coefficient
is the number of k-element subsets (the k-combinations) of an n-element set; that is, the number of ways that k things can be 'chosen' from a set of n things.


http://en.wikipedia.org/wiki/Probability_density_function
a function that represents a probability distribution in terms of integrals

kernel density estimation; Parzen window method
http://en.wikipedia.org/wiki/Parzen_window
a non-parametric way of estimating the probability density function of a random variable.
Given some data about a sample of a population, kernel density estimation makes it possible to extrapolate the data to the entire population.

http://en.wikipedia.org/wiki/Kernel_(statistics)
A kernel is a weighting function used in non-parametric estimation techniques. Kernels are used in kernel density estimation to estimate random variables' density functions, or in kernel regression to estimate the conditional expectation of a random variable.

http://en.wikipedia.org/wiki/Hypercube

http://en.wikipedia.org/wiki/Neural_networks