Let O be the centre of the circle in which ABCD is inscribed and let R be its radius. Using cosine rule in triangle AOB,
we get, 2+√2=2R2(1−cos135∘) [∵ In △ABC, cosA=b2+c2−a22bc where a, b and c are the sides of given triangle ]
⟹2+√2=R2(2+√2).
∴R=1.
Consider quadrilateral ABCD as in the second figure above. Join AC.
For [ADC] to be maximum, it is clear that D should be the mid-point of the arc AC so that its distance from the chord AC is maximum
∴AD=DC for [ABCD] to be maximum
Similarly, we conclude that BC=CD
∴BC,CD,DA subtends equal angles at the centre O.
Let ∠BOC=α,∠COD=βand∠DOA=γ.
Observe that [ABCD]=[AOB]+[BOC]+[COD]+[DOA]=12sin135∘+12(sinα+sinβ+sinγ).
Now [ABCD] has maximum area if and only if α=β=γ=(360∘−135∘)/3=75∘.
∴[ABCD]=12sin135∘+32sin75∘=12(1√2+3√3+12√2)=5+3√34√2.
Alternatively, we can use Jensen's inequality. Observe that α,β,γ are all less than 180∘. Since sinx is concave on (0,π), Jensen's inequality gives
sinα+sinβ+sinγ3≤sin(α+β+γ3)=sin75∘.
Hence [ABCD]≤12√2+32sin75∘=5+3√34√2,
With equality if and only if α=β=γ=75∘.