Question

Let Xand Y are two continous randoms variables with probability density functions $f_x\left(x\right)$ and $f_x\left(y\right)$ respectively. If the joint probability density function is $f_{XY}\left(xy\right)$ , then mutual information of can be expressed as . Note that, E[.] represents the expectation error.

• A. $E\left[\right(f_x\left(x\right)+f_r\left(y\right)\left)ln\right(f_{XY}(x,y)\left)\right]$
• B. X and Y is $f_{xy}(x,y)$
• C. $=E\left[ln\right(\frac{f_{XY}(x,y)}{f_X\left(x\right)f_r\left(y\right)}\left)\right]$
  
• D. X and Y $[i.e.,I(X;Y\left)\right]$

Answer 1

The correct option is C $=E\left[ln\right(\frac{f_{XY}(x,y)}{f_X\left(x\right)f_r\left(y\right)}\left)\right]$


$I(X:Y)=H\left(X\right)+H\left(Y\right)-H(X,Y)$
$={\int }_{-\infty }^{\infty }{f}_X\left(x\right)ln\left(f_X\right(x\left)\right)+{\int }_{-\infty }^{\infty }{f}_r\left(y\right)ln\left(f_r\right(y\left)\right)+{\int }_{-\infty }^{\infty }{f}_{XY}(x,y)ln\left(f_{XY}\right(x,y\left)\right)$
$=-E\left[ln\right(f_X\left(x\right)\left)\right]-E\left[ln\right(f_r\left(y\right)\left)\right]+E\left[ln\right(f_{XY}(x,y)\left)\right]$
$=E\left[ln\right(f_{XY}(x,y))-ln(f_X\left(x\right))-ln(f_r\left(y\right)\left)\right]$
$=E\left[ln\right(\frac{f_{XY}(x,y)}{f_X\left(x\right)f_r\left(y\right)}\left)\right]$