The correct option is
C ab(π−2)As the rectangle lies inside a standard ellipse, the vertices of the rectangle can be assumed to be (h,k),(h,−k),(−h,k) and (−h,−k) without any loss of generality.
∴h2a2+k2b2=1⟹k=b√1−h2a2
Clearly, the sides of the rectangle based on the coordinates chosen can be determined as 2h and 2k.
∴ area of rectangle =A(h)=2h×2k=4hb√1−h2a2⟹4b√h2−h4a2
Now, let P(h)=h2−h4a2. Thus, the maximization of A(h) boils down just to the maximization of P(h).
Differentiating P(h) wrt h, we have
2h−4h3a2=0⟹h=2h3a2⟹2h2=a2⟹h=a√2⟹k=b√2
Hence, Amax=2ab
∴ area included between this ellipse and rectangle =πab−2ab=ab(π−2).
this is the required answer.