Cumulative Density Function
Probability Density Function
Probability Mass Function
Continuous Probability Distributions
Joint Probability Distributions
Mean of a Random Variable (Expected Value)
Let X be a random variable w/ probability distribution f(x). The mean/expected value of X is:
- $\mu = E(X) = \sum_x xf(x)$ if X is discrete
- $\mu = E(X) = \int_{-\infty}^{\infty}xf(x)dx$ if X is continuous
Difference between sample mean and expected value is sample mean is calculated using data; expected value is calculated using probability distribution.
Let X be a random variable w/ probability distribution f(x). The expected value of a random variable g(X) is:
- $\mu_{g(X)} = E[g(X)] = \sum_xg(x)f(x)$ if X is discrete
- $\mu_{g(X)} = E[g(X)] = \int_{-\infty}^{\infty}g(x)f(x)dx$ if X is continous
IMPORTANT: Mean/expected value of a random variable X is important because it describes where the probability distribution is centered.
Variance
Let X be a random ariable w/ probability distribution f(x) and mean $\mu$. The variance of X is:
- $\sigma^2 = E[(X - \mu)^2] = \sum_x(x - \mu)^2f(x)$ if X is discrete
- $\sigma^2 = $
Proof of formula for $Var(X_1) = E({X_1}^2) - {\mu_1}^2:$
$Var(X_1) = E[({X_1} - \mu_1)^2] = E({X_1}^2 - 2X_1\mu_1 + {\mu_1}^2) = E({X_1}^2) - 2\mu_1E(X_1) + E({\mu_1}^2)$
Then
$Var(X_1) = E({X_1}^2) - 2\mu_1(\mu_1) + {\mu_1}^2 = E({X_1}^2) - 2{\mu_1}^2 + {\mu_1}^2 = E({X_1}^2) - {\mu_1}^2$
The positive square root of the variance is the standard deviation of X.
Covariance
The covariance of two random variables X and Y w/ means $\mu_x$ and $\mu_y$, respecitvely, is given by: $\sigma_{xy} = E(XY) - {\mu_x}{\mu_y}$
Proof of formula for $Cov(X_1, X_2) = E({X_1}{X_2}) - {\mu_1}{\mu_2}$:
$Cov(X_1,X_2) = E[(X_1-\mu_1)(X_2-\mu_2)] = E({X_1}{X_2}-{X_1}{\mu_2}-{\mu_1}{X_2}+{\mu_1}{\mu_2})$ $= E(X_1X_2) - E(X_1\mu_2) - E(\mu_1X_2) + E(\mu_1\mu_2)$
Since $\mu_1$ and $\mu_2$ are non-random constants, we obtain: $Cov(X_1,X_2) = E(X_1X_2) - \mu_2E(X_1)-\mu_1E(X_2)+\mu_1\mu_2$
Since $E(X_1) = \mu_1$ and $E(X_2) = \mu_2$, we get:
$Cov(X_1,X_2) = E(X_1X_2) - \mu_2\mu_1 - \mu_1\mu_2 + \mu_1\mu_2 = E(X_1X_2) - \mu_1\mu_2$
Bernoulli Trial
It is an experiment that can result in one of two outcomes, “success” and “failure”, with:
$P(success) = p$ and $P(failure) = 1-p = q$ where $0 < p < 1$
Bernoulli and Binomial Distributions
The probability mass function (pmf) of a binomial random variable X is given by: $P(X=x) = {{n}\choose{x}}p^xq^{n-x}$ for x $\in \mathbb{Z}$
The cumulative distribution function (cdf) of a binomial random variable X w/ number of trials n and probability of success p is denoted by:
$P(X <= r) = \sum_{x=0}^r P(X=x) = \sum_{x=0}^r {{n}\choose{x}}p^xq^{n-x}$ for r $\in \mathbb{Z}$
If X ~ Bin(n, p), then the theoritical mean of X is: $\mu_x = E(X) = np$ and the theoritical variance is: $\sigma_{X}^2 = Var(x) = npq = np(1-p)$