Motivation
Let $\Omega=\{\omega_{1}, \omega_{2},\omega_{3} \}$ be a space space and $P(\{\omega_{i}\}=\frac{1}{3}, \forall i=1,2,3$. Let $X:\Omega \rightarrow \mathbb{R}$ and $Y:\Omega \rightarrow \mathbb{R}$ be discrete random variables defined with:
$$X(\omega_{1})=1, X(\omega_{2})=2, X(\omega_{3})=3$$ $$Y(\omega_{1})=2, X(\omega_{2})=3, X(\omega_{3})=1$$ $X$ and $Y$ has the same distribution: 
Lets take a look at the distribution of $X+Y$. We know that $(X+Y)(\omega_{1})=X(\omega_{1})+Y(\omega_{1})=1+2=3$. Similarly, for $(X+Y)(\omega_{2})=4$ and $(X+Y)(\omega_{3})=5$. The distribution looks like this:
Naturally we would think the distribution of $X+X$ looks the same as $X+Y$ since $X \sim Y$. However, that’s not the case.
Generally, function $g:\mathbb{R}^{2} \rightarrow \mathbb{R}$, $g(X,Y)$ is random variable. However, for finding its distribution it’s not enough to know the the distribution of $X$ and $Y$. We need to find their combined distribution, i.e. the distribution of random vector $(X,Y)$.
Definition
For discrete random variables $X$ and $Y$ we define discrete random vector as a function $(X,Y): \Omega \rightarrow \mathbb{R}^{2}$.
Let $X$ have values $a_{i}$, $\forall i \in \mathbb{N}$ and $Y$ values $b_{j}$, $\forall j \in \mathbb{N}$. Pairs $(a_{i}, b_{j})$ and their probabilities $$p_{i,j}:=P(X=a_{i}, Y=b_{j})$$ are the distribution of discrete random vector $(X,Y)$.
The table of distribution looks like this:
For example,
$$q_{1}=\sum_{j=1}^{\infty} p_{1,j} = \sum_{j=1}^{\infty} P(X=a_{i}, Y=b_{j}) =P(\{X=a_{1}\} \cap (\cup_{j=1}^{\infty}\{Y=b_{j}\}))=P(X=a_{1})$$
because $(\cup_{j=1}^{\infty}\{Y=b_{j}\})=\Omega$.
From the table we can easily see that 
Proposition.
(i) $X$ and $Y$ are independent if and only if $$p_{i,j}=P(X=a_{i}, Y=b_{j})=P(X=a_{i}) P( Y=b_{j})=q_{i} r_{j}, \forall i,j \in \mathbb{N}$$
(ii) If $g:\mathbb{R}^{2} \rightarrow \mathbb{R}$ then for every $c \in \mathbb{R}$
$$\displaystyle{P(g(X,Y)=c)= \sum^{\infty} _{\substack {i,j=1\\ g(a_{i},b_{j})=c}} P(X=a_{i}, Y=b_{j}) }$$ Consequently, $$E[g(X,Y)] =\sum_{i,j=1}^{\infty} g(a_{i}, b_{j}) p_{i,j}$$
Covariance
Covariance is a measure of the strength of the correlation between two or more random variables. It shows how much the variables change together. In other words, it’s a measure of the joint variability of random variables.
Definition. Let $X$ and $Y$ be random variables and $Var(X), Var(Y) < \infty$. Covariance of $X$ and $Y$ is $$Cov(X,Y):=E[(X-E[X])(X-E[Y])]=E[XY]-E[X] E[Y]$$ If $Cov(X,Y)=0$ variables $X$ and $Y$ aren’t correlated, otherwise they are correlated.
Sometimes it’s difficult to estimate the strength of correlation solely on covariance. For example, does $Cov(X,Y)=65$ mean $X$ and $Y$ are strongly correlated or not?
For that reason we define correlation coefficient $$\rho (X,Y)= \frac{Cov(X,Y)}{\sqrt{Var(X)} \sqrt{Var(Y)}} \in [-1,1]$$
We have three possibilities:
- positive correlation, $\rho \in (0,1] \rightarrow$ both variables change into the same direction
- neutral correlation, $\rho=0 \rightarrow$ no relationship between variables
- negative correlation, $\rho \in [-1,0) \rightarrow$ variables change in the opposite direction
Intuitively, if $\rho (X,Y)$ is close to $-1$ or $1$ that indicates strong dependence between $X$ and $Y$. Analogously, if $X$ and $Y$ are independent, then $\rho (X,Y)=0$, i.e. they aren’t correlated. However, if $\rho (X,Y)=0$ that doesn’t indicate that $X$ and $Y$ are independent.
Example. Let $X$ be random variable with distribution 
and $Y=X^{2}$. In this case $X$ and $Y$ are dependent, however $$Cov(X,Y)=E[XY]-E[X][Y]=E[X^{3}]- 0 \cdot E[Y]=0$$ Consequently, $X$ and $Y$ aren’t correlated.
In conclusion, $Cov(X,Y)=0$ generally doesn’t imply that $X$ and $Y$ are independent. That’s because covariance measures only one form of dependence, i.e. “linear” dependence.
Example
Lets throw two symmetric dice. Let $X$ be the number of $6$s we get, and $Y$ the number of $1$s. Find:
(a) Distribution of random vector $(X,Y)$ and random variables $X$ and $Y$. Are they independent?
(b) $\rho (X,Y)$
(c) Distribution of $X+2Y$.
Solution
(a) Distribution of $(X,Y)$
Since we only have $2$ dice the number of $6$s and $1$s can only be $0, 1$ or $2$. Lets start from beginning.
Lets say we got zero $6$s and $1$ss. That means both dice have $4$ options left. Since they are independent, we use the multiplication rule. $$\displaystyle{P(X=0, Y=0)= \frac{4^{2}}{6^{2}}=\frac{16}{36}}$$
Furthermore, lets say we have zero $6$s and one $1$. That means one die has only one option $(1)$ while the other has $4$ – there can’t be $1$ or $6$. If $1$ rolled on the second die too, it would mean we had two $1$ and not only one. Moreover, we don’t know which die had rolled $1$, so we can have two options. Consequently, $$\displaystyle{P(X=0, Y=1)= 2 \cdot \frac{4}{6^{2}} =\frac{8}{36}}$$
Similarly for $\displaystyle{(X=1, Y=0)=\frac{8}{36}}$. We use the same way of thinking, we just switch the numbers.
Additionally, for one $1$ and one $6$ we only have an option when one die has $1$ and another has $6$. However, since we can switch dice, we have $2$ options. $$\displaystyle{P(X=1, Y=1)= 2 \cdot \frac{1}{6^{2}} =\frac{2}{36}}$$
For situation where we have $$\displaystyle{(X=0, Y=2)= \frac{1}{6^{2}}= \frac{}{36}}$$
Options where:
$(X=1, Y=2)$, $(X=2, Y=1)$ or $(X=2, Y=2)$ cannot happen because we only have 2 dice. Consequently, we cannot have rolled two $6$s and one $1$.
Finally, lets make a table of distribution 
The distribution of $X$ and $Y$ separately will be symmetric.
Are $X$ and $Y$ independent?
Random variables $X$ and $Y$ are independent if $$P(X=a_{i}, Y=b_{j})=P(X=a_{i}) \cdot P(Y=b_{j})$$ The best way to see if two variables are independent while using the distribution of random vector is to look for $0$s.
Lets see if $P(X=0, Y=0)=P(X=0) \cdot P(Y=0)$. Reading from the table we have:
$$P(X=0, Y=0)=\displaystyle{\frac{16}{36} \neq \frac{25}{36} \cdot \frac{25}{36} = P(X=0) \cdot P(Y=0)}$$
As a result, $X$ and $Y$ aren’t independent.
(b) $\rho (X,Y) = ?$
For starters lets look at the formula of correlation coefficient
$$\rho (X,Y)= \frac{Cov(X,Y)}{\sqrt{Var(X)} \sqrt{Var(Y)}}$$
Firstly, lets find $Cov(X,Y)$. $$Cov(X,Y)=E[XY]-E[X]E[Y]$$
$\displaystyle{E[X]=0 \cdot \frac{25}{36} + 1 \cdot \frac{10}{36} + 2 \cdot \frac{1}{36}=\frac{1}{3}}$
Similarly, $\displaystyle{E[Y]= \frac{1}{3}}$.
For $E[XY]$ we have to multiply every separate value of both $X$ and $Y$ and their combined probability. For starters:
$$X=0, Y=0, P= \frac{16}{36} \rightarrow 0 \cdot 0 \cdot \frac{16}{36}$$
We then sum up all the products:
$$\displaystyle{E[XY] = 0 \cdot 0 \cdot \frac{16}{36} + 0 \cdot 1 \cdot \frac{8}{36} + 0 \cdot 2 \cdot \frac{1}{36} + 1 \cdot 0 \cdot \frac{8}{36} + … 1\cdot 1 \cdot \frac{2}{36} +…}$$
Since we have $0$ in every product besides one, $\displaystyle{E[XY]=\frac{2}{36}= \frac{1}{18}}$
Consequently, $\displaystyle{Cov(X,Y)=\frac{1}{18}-\frac{1}{9}=-\frac{1}{18}}$
Secondly, we need variance of each random variable.
$$Vax(X)= Var(Y)= E[X^{2}]-(E[X])^{2}=\frac{7}{18} -(\frac{1}{3})^{2}=\frac{5}{18}$$
$$\rho (X,Y)= \frac{\frac{-1}{18}}{\sqrt{\frac{5}{18} \frac{5}{18}}=\frac{-1}{5}}=-0.2$$
This is another way to see if variables are independent. $\rho \neq 0$, hence $X$ and $Y$ are dependent.
(c) Distribution of $X+2Y$
Possible values of $X$ and $Y$ are still $0$, $1$ or $2$. However, we need the values of $X+2Y$. Lets make another table. Be careful, this is not a distribution table. It just helps us to find the right values. Even though we are finding the distribution of $X+2Y$ we still have to take into consideration that $X$ and $Y$ represent the number of $1$s and $6$s. As a result, we can’t have outcomes like before, i.e. $X=2, Y=1$.
We fill in the table in the following the $X+2Y$:
$\bullet$ $X=0, Y=0 \rightarrow X+2Y=0$
$\bullet$ $X=0, Y=1 \rightarrow X+2Y=2$
… and so on. The table looks like this.
We can see that the values that appear are $0$, $1$, $2$ $3$ and $4$. Consequently, those are the possible values for $X+2Y$.
Lets find their probabilities.
For $X+2Y=0$ there is only one possible solution $\rightarrow (X=0, Y=0)$. From the table of distribution (in part (a)) we read the probability of that outcome which is $\displaystyle{\frac{16}{36}=\frac{4}{9}}$.
For $X+2Y=1$ there is only one possible solution $\rightarrow (X=1, Y=0)$. It’s probability is $\displaystyle{\frac{8}{36}=\frac{2}{9}}$.
For $X+2Y=2$ we have multiple solutions
$\bullet$ $\displaystyle{(X=0, Y=1) \rightarrow P= \frac{2}{9}}$
$\bullet$ $\displaystyle{(X=2, Y=0) \rightarrow P=\frac{1}{36}}$
Those are mutually exclusive outcomes, so we sum up their probabilities. $\displaystyle{P=\frac{1}{4}}$
For $X+2Y=3$ there is only one possible solution $\rightarrow (X=1, Y=1)$. It’s probability is $\displaystyle{\frac{2}{36}=\frac{1}{18}}$.
For $X+2Y=4$ we have multiple solutions
$\bullet$ $(X=2, Y=1) \rightarrow P= 0$
$\bullet$ $\displaystyle{(X=0, Y=2) \rightarrow P=\frac{1}{36}}$
Combined probability is $\displaystyle{\frac{1}{36}}$.
Finally, the distribution of $X+2Y$ looks like this
