Chapter 6 Sums of Random Variables

central limit theorem
http://en.wikipedia.org/wiki/Central_limit_theorem


6.1 Expected Values of Sums

Theorem 6.1
The expected value of the sum equals the sum of the expected values whether or not each variables are independent.

Theorem 6.2
The variance of the sum is the sum of all the elements of the covariance matirx.


6.2 PDF of the Sum of Two Random Variables

Theorem 6.5
http://en.wikipedia.org/wiki/Convolution

linear system
http://en.wikipedia.org/wiki/Linear_system
 

6.3 Moment Generating Functions

In linear system theory, convolution in the time domain corresponds to multiplication in the frequency domain with time functions and frequency functions related by the Fourier transform.

In probability theory, we can use transform methods to replace the convolution of PDFs by multiplication of transforms.

moment generating function
: the transform of a PDF or a PMF

http://en.wikipedia.org/wiki/Moment_generating_function

http://en.wikipedia.org/wiki/Laplace_transform

region of convergence


6.4 MGF of the Sum of Independent Random Variables

Theorem 6.8
Moment generating functions provide a convenient way to study the properties of sums of independent finite discrete random variables.

Theorem 6.9
The sum of independent Poisson random variables is a Poisson random variable.

Theorem 6.10
The sum of independent Gaussian random variables is a Gaussian random variable.

In general, the sum of independent random variables in one family is a different kind of random variable.

Theorem 6.11
The Erlang random variable is the sum of n independent exponential random variables.


6.5 Random Sums of Independent Random Variables

random sum
: sum of iid random variables in which the number of terms in the sum is also a random variable

It is possible to express the probability model of R as a formula for the moment generating function.


The number of terms in the random sum cannot depend on the actual values of the terms in the sum.


6.6 Central Limit Theorem

Probability theory provides us with tools for interpreting observed data.

bell-shaped curve = normal distribution


So many practical phenomena produce data that can be modeled as Gaussian random variables.


http://en.wikipedia.org/wiki/Central_limit_theorem

Central Limit Theorem
The CDF of a sum of random variables more and more resembles a Gaussian CDF as the number of terms in the sum increases.
 
Central Limit Theorem Approximation = Gaussian approximation


6.7 Applications of the Central Limit Theorem

De Moivre-Laplace Formula

http://en.wikipedia.org/wiki/Theorem_of_de_Moivre%E2%80%93Laplace
normal approximation to the binomial distribution


6.8 The Chernoff Bound

Chernoff Bound

http://en.wikipedia.org/wiki/Chernoff_bound
exponentially decreasing bounds on tail distributions of sums of independent random variables