Question

Two continuous random variables $X$ and $Y$ are related as $Y=4X+2.$ The correct relation between the differential entropies of the two random variable is equal to

• A. $H\left(Y\right)=4H\left(X\right)+2$
• B. $H\left(Y\right)=4H\left(X\right)$
• C. $H\left(Y\right)=H\left(X\right)$
• D. $H\left(Y\right)=H\left(X\right)+2$

Answer 1

The correct option is D $H\left(Y\right)=H\left(X\right)+2$

$\text{Let Y= aX+b where 'a' and 'b' are constants.}\begin{array}{cc}\text{then}x& =\frac{y-b}{a}\\ \text{and,}\frac{dy}{dx}& =a\\ \therefore f_Y\left(y\right)& =\frac{1}{a}{f}_x\left(\frac{y-b}{a}\right)\\ \text{thus, the differential entopy of Y,}& \\ H\left(Y\right)& =-{\int }_{-\infty }^{\infty }{f}_y\left(y\right){\log }_2{f}_y\left(y\right)dy\\ & =-{\int }_{-\infty }^{\infty }\frac{1}{a}{f}_x\left(\frac{y-b}{a}\right){\log }_2\left[\frac{1}{a}{f}_x\left(\frac{y-b}{a}\right)\right]dy\\ \text{Let,}\frac{y-b}{a}& =u\Rightarrow dy=adu\\ H\left(Y\right)& =-{\int }_{-\infty }^{\infty }{f}_x\left(u\right){\log }_2\left[\frac{1}{a}{f}_x\right(u\left)\right]du\\ & =-{\int }_{-\infty }^{\infty }{f}_x\left(u\right){\log }_2{f}_x\left(u\right)du+{\int }_{\infty }^{\infty }{f}_x\left(u\right){\log }_2\left(a\right)du\\ H\left(Y\right)& =H\left(X\right)+{\log }_2\left(a\right)\\ \Rightarrow H\left(Y\right)& =H\left(X\right)+{\log }_2\left(4\right)\\ H\left(Y\right)& =H\left(X\right)+2\end{array}$