Notes on Covariance

\[X,Y\,independent \rightarrow cov(X,Y) = 0\]

Assume random variables \(X\) and \(Y\) are discrete. That is, assume that there is a finite or denumerable sample space which is a set of \(w_i\) and a set of quantities \(x_i\) and \(y_i\) defined.

Definition \(X\) and \(Y\) are independent if

\[prob((X = x)\,and\,(Y = y)) = prob(X = x)prob(Y = y)\]

in which \(x\) is some \(x_i\) and \(y\) is some \(y_j\).

Then if \(X\) and \(Y\) are independent,

\[E(XY) = E(X)E(Y)\]

Proof:

\[E(XY) = \sum _{i,j} \ x_iy_jprob(XY = x_iy_j)\]

\[= \sum _{i,j} \ x_iy_jprob((X = x_i)\,and\,(Y = y_j))\]

\[= \sum _{i,j} \ x_iy_jprob(X = x_i)prob(Y = y_j)\]

\[= \sum _{i} \ x_iprob(X = x_i)\sum _{j} \ y_jprob(Y = y_j) = E(X)E(Y)\]

Then if \(X\) and \(Y\) are independent,

\[cov(X,Y) = E[(X - E(X))(Y - E(Y))]\]

\[=E[XY - XE(Y) - YE(X)+E(X)E(Y)]\]

\[=E[XY] - E(X)E(Y) - E(Y)E(X) + E(X)E(Y) = 0\]