Question

$IfXandYaretwodiscreterandomvariablesandH\left(V\right)representstheentropyofthevariableV,thenconsiderthefollowingrelations:Selectthecorrectrelationsusingthecodesgivenbelow.R1:H\left(X\right)\ge H\left(X|Y\right)R2:H(X,Y)\ge H\left(X\right)+H\left(Y\right)R3:H\left(x\right)-H\left(Y\right)=H\left(X|y\right)-H\left(Y|X\right)$

• A. $R1andR3only$
• B. $R1andR2only$
• C. $R1R2andR3$
• D. $R2andR3only$

Answer 1

The correct option is A $R1andR3only$

$ThemutualinformationI(X;Y)canbegivenas,I(X;Y)=H\left(X\right)-H\left(X|Y\right)..\left(1\right)I(X;Y)=H\left(Y\right)-H\left(Y|X\right)..\left(2\right)I(X;Y)=H\left(X\right)+H\left(Y\right)-H(X,Y)..\left(3\right)I(X;Y)\ge \mathrm{0..}\left(4\right)Fromequations\left(1\right)and\left(4\right),H\left(X\right)-H\left(X|Y\right)\ge 0H\left(X\right)\ge H\left(X|Y\right)\Rightarrow R1iscorrectfromequations\left(3\right)and\left(4\right),H\left(X\right)+H\left(Y\right)-H(X,Y)\ge 0H\left(X\right)+H\left(Y\right)\ge H(X,Y)\Rightarrow R2isincorrectFromequations\left(1\right),\left(2\right)and\left(4\right),H\left(X\right)-H\left(X|Y\right)=H\left(Y|X\right)H\left(X\right)-H\left(Y\right)=H\left(X|Y\right)-H\left(Y|X\right)\Rightarrow R3iscorrectS0,onlyR1andR3arecorrect.$