Motivation for expected value
Lets say we have random variables $X$, $Y$ and $Z$ distributed with
In $X$ all values are equally possible. Our intuition tells us the expected value of $X$ would be somewhere around the middle point, since it’s more likely to get the middle than the edges. The expected value would be $2$.
For $Z$ we have the same reasoning, the outcomes are all equally possible. The middle point is, once again, number $2$. However, for $Y$ we have values with different possibility, with $3$ being the outcome with the highest chance. As a result, the expected value would be between $2$ and $3$, in contrast to middle point we had before.
In conclusion, the expected value of a random variable $X$ would be middle value of $X$..
Expected value
Let $X$ be a discrete random variable with distribution:
The expected value of $X$ is $$EX= \sum_{i} a_{i} p_{i}$$ if the said series converge absolutely. To be more precise, expected value is the long-run average value of repetitions of the same experiment it represents.
Expected value of a discrete random variable can also be defined as is the probability-weighted average of all possible values. In other words, each possible value the random variable can assume is multiplied by its probability of occurring, and the resulting products are summed to produce the expected value.
IMPORTANT!
Even if all the values of $X$ are integers, the expected value can be a real number. For example, it can be $3.87$ which means that most of the values will be $3$ or $4$, with bigger tendencies to $4$.
Example. Let $X$ be
If $\displaystyle{EX= \frac{1}{3}}$, find $a$.
Value of $X$ are $a_{1}=-1$, $a_{2}=0$ and $a_{3}=a$. The probabilities of given values are $p_{1}=\frac{1}{2}$, $p_{2}=\frac{1}{3}$ and $p_{3}=\frac{1}{6}$
The definition of $EX$ is $\sum_{i} a_{i} p_{i}$ $$EX=a_{1}\cdot p_{1} + a_{2}\cdot p_{2} + a_{2}\cdot p_{2}$$ When we put the values we know into the equation, we get $$\displaystyle{\frac{1}{3}=-1 \cdot \frac{1}{2} + 0 \cdot \frac{1}{3} + a \cdot \frac{1}{6}} $$
The step that’s left is solving an equation with one unknown variable. As a result, we get $$a=5$$
Proposition.
(a) $X=c$ almost always $\Leftrightarrow X\sim$ ${c}\choose{1}$ $\Rightarrow EX=c$
(b) $X\geq 0$ almost always $\Leftrightarrow a_{n} \geq 0, \forall n \in \mathbb{N} \Rightarrow EX \geq 0$
(c) $\alpha, \beta \in \mathbb{R}, E[\alpha X+\beta Y]=\alpha E[X] + \beta E[Y] $
(d) $X \geq Y$ almost always $\Rightarrow EX \geq EX$
(e) For
and function $g= \mathbb{R} \rightarrow \mathbb{R}$ $$E[g(X)]=\sum_{k=1}^{\infty} g(a_{k}) p_{k}$$
Example
Lets roll a symmetric die and let $X$ be the number on top. Find the expected value of $\frac{1}{X+1}$.
The distribution of $X$ is
Let random variable $Y$ be defined as $Y=g(X)$ with $g(X)$ being is the given expression $\displaystyle{g(X)=\frac{1}{X+1}}$.
To get the distribution of $Y$ we put the values of $X$ into the expression for $Y$.
The only thing that’s left is to calculate the expected value.
$\displaystyle{EY= \frac{1}{2} \cdot \frac{1}{6} + \frac{1}{3} \cdot \frac{1}{6} + … + \frac{1}{7} \cdot \frac{1}{6} = \frac{223}{840}} $
Motivation for variance
Lets look at the same example as before. $X$ and $Y$ are random variables that represent different events.
Firstly, lets calculate their expected value.
$\bullet$ $\displaystyle{EX= 1 \cdot \frac{1}{3} + 2\cdot \frac{1}{3} + 3 \cdot \frac{1}{3}= 2}$
$\bullet$ $\displaystyle{EY= 1 \cdot \frac{1}{6} + 2\cdot \frac{1}{6} + 3 \cdot \frac{2}{3}= \frac{5}{2}=2.5}$
We know that if we repeat the events enough times most outcomes will be around their expected value. However, how big is the interval in which most values will occur? How far are they spread?
In $X$ all values are equally possible, so we assume the interval will be $\pm \frac{1}{3}$ around expected value. On the other hand, random variable $Y$ isn’t equally distributed. How do we find its interval?
Variance
The variance is measure of spread for a distribution of a random variable. Moreover, it determines the degree to which the values of a random variable differ from the expected value. Generally, it shows how spread are the outcomes.
The variance of a random variable $X$ is the expected value of the squared deviation from the expected value of $X$. As a result, it’s defined with $$VarX=E[(X-EX)^{2}]$$
We have
(a) $VarX=E[X^{2}]-(EX)^{2}$
(b) $Var(aX+b)=a^{2}VarX$
For calculating variance in given problems we will mostly use (a).
Standard deviation
Standard deviation is also a measure of spread. As a matter of fact, it’s defined as a square root of variance and noted as $\sigma$.
$$\sigma(X)= \sqrt{Var(X)}$$
You may wonder why do we need standard deviation if we already have variance. Standard deviation is more useful in statistics and other areas of mathematics. It gives us a “standard” way of knowing whats normal and how deviated our values are from the norm.
Example
Lets throw two symmetric dice, and let random variable $X$ be absolute value of the differences between two numbers on dice. Find distribution, expected value. variance and standard deviation of $X$.
Solution
Firstly, let’s try to draw a table that represents all the outcomes of rolling dice. That won’t be a distribution of $X$ but it will useful for finding it. The first row will represent possible outcomes on the first die and the first column will represent the outcomes on the second one.
If the numbers we get are $(1,1)$ their difference is $0$. If we get $(2,1)$ their difference is $1$. Analogously, we repeat the process until we have the complete table that looks like this:
Looking at the table, all the possible values of $X$ are $\{0,1,2,3,4,5\}$. The number of all possible outcomes of rolling dice is $36$. As result, the probability of a single outcome is $\displaystyle{\frac{1}{36}}$. Let’s find the distribution.
To find the probability of $X$ being a certain value, we count how many times it’s occurring in the table. There are 6 zeros in the table, therefore probability of $X=0$ is $\displaystyle{\frac{6}{36}=\frac{1}{6}}$. By continuing the process we get the full distribution of $X$ which looks like this:
Secondly, lets calculate the expected value:
$$\displaystyle{E[X]=0 \cdot \frac{1}{6}+ 1 \cdot \frac{5}{18} + 2 \cdot \frac{2}{9} + 3\cdot \frac{1}{6} + 4 \cdot \frac{1}{9} + 5 \cdot \frac{1}{18}= \frac{35}{18} \approx 1.94}$$
Thirdly, lets calculate the variance using previous formula $VarX=E[X^{2}]-(EX)^{2}$
First we need to find $E[X^{2}]$ $$\displaystyle{E[X^{2}]=0^{2} \cdot \frac{1}{6}+ 1^{2} \cdot \frac{5}{18} + 2^{2} \cdot \frac{2}{9} + 3^{2}\cdot \frac{1}{6} + 4^{2} \cdot \frac{1}{9} + 5^{2} \cdot \frac{1}{18}= \frac{105}{18}}$$
As a result,
$$\displaystyle{VarX=\frac{105}{18} – (\frac{35}{18})^{2} \approx 2.052}$$
Finally, lets calculate standard deviation.
$$\sigma(X)= \sqrt{Var(X)} = \sqrt{2.052} \approx 1.432$$
Expected value and variance of special random variables
(a) Bernoulli random variable
As we’ve seen in previous lesson, Bernoulli random variable $X \sim B(1,p)$ has distribution 
Consequently, its expected value is:
$\bullet$ $EX=0 \cdot (1-p) + 1 \cdot p \Rightarrow EX=p$
Furthermore, variance is:
$\bullet$ $Var(X)=E[X^{2}]-(EX)^{2}=0^{2} \cdot (1-p) + 1^{2} \cdot p – p^{2} \Rightarrow Var(X)= p(1-p)$
(b) Binomial random variable
We can interpret Binomial random variable, $X \sim B(n,p)$, as $n$ Bernoulli variables. To be precise, $X=X_{1}+ X_{2}+…+X_{n}$, with $X_{i} \sim B(1,p)$ independent variables.
As a result, its expected value is: $EX=n \cdot p$
Furthermore variance is: $Var(X)=n \cdot p(1-p)$
(c) Geometric random variable
$X \sim G(P)$, $P(X=k)=q^{k-1} \cdot p, \forall k \in \mathbb{N}$. Remember $q=(1-p)$.
Consequently, its expected value is: $EX=\displaystyle{\frac{1}{p}}$
And variance is: $Var(X)=\displaystyle{\frac{q}{p^{2}}}$
(d) Poisson random variable
$X \sim G(P)$, $\displaystyle{P(X=k)=e^{- \lambda} \cdot \frac{\lambda^{k}}{k!}}, \forall k \in \mathbb{N_{o}}$.
As a result, its expected value is: $EX=\lambda $
Furthermore variance is: $Var(X)=\lambda $
