Question

If X and Y are continuous random variables and H(V) represents the entropy of the variable V, then consider the following relations.

Select the correct relation(s) using the codes given below.
$R1:H(X+2)=H\left(X\right)$
$R2:H(X+Y|X)\ge H\left(Y|X\right)$

• A. R2 only
• B. R1 only
• C. Neither R1 nor R2
• D. Both R1 and R2

Answer 1

The correct option is B R1 only

Ans : (a)

Let V = X + 2 Then,
$f_v\left(v\right)=f_X(v-2)$
$H(X+2)=H\left(V\right)=-{\int }_{-\infty }^{\infty }{f}_v\left(v\right)\text{ln}\left[f_v\right(v\left)\right]dv$

$=-{\int }_{-\infty }^{\infty }{f}_X(v-2)\text{ln}\left[f_X\right(v-2\left)\right]dv$
$\text{Let,}u=v-2\Rightarrow dv=du.$
$\text{So,}H(X+2)=-{\int }_{-\infty }^{\infty }{f}_X\left(u\right)\text{ln}\left[f_X\right(u\left)\right]du=H\left(X\right)$

$H(X+2)=H\left(X\right)\Rightarrow \text{Relation R1 is correct}$
$\text{Let,}Z=X+Y.\text{Then,}Y=Z-X\text{and}{F}_{XZ}(x,z)=f_{XY}(x,z-x)$

$H(X+Y|X)=H\left(Z|X\right)=-{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{f}_{XZ}(x,z)\text{In}\left[\frac{f_{XZ}(x,z)}{f_X\left(x\right)}\right]dxdz$

$={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{f}_{XY}(x,z-x)\text{ln}\left[\frac{f_{XY}(x,z-x)}{f_X\left(x\right)}\right]dxdz$

$={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{f}_{XY}(x,y)\text{ln}\left[\frac{f_{XY}(x,y)}{f\left(x\right)}\right]dxdy$

$H(X+Y|X)=H\left(Y|X\right)\Rightarrow \text{Relation R2 is incorrect}$