The correct option is C -1
Given ,
(tan−1x)2+(cot−1x)2=5π28
As we know that ,
tan−1x+cot−1x=π2
Squaring on both side , we get
(tan−1x+cot−1x)2=π24
(tan−1x)2+(cot−1x)2+2tan−1xcot−1x=π24
5π28+2tan−1xcot−1x=π24
2tan−1xcot−1x=−3π28
tan−1xcot−1x=−3π216
tan−1x[π2−tan−1x]=−3π216
Now, Let
y=tan−1x
The Above Equation becomes ,
16y2−8πy−3π2=0
On solving this Quadratic Equation in y , we get
y=−π4,y=3π4
Also
y=tan−1x
tan−1x=−π4 tan−1x=3π4
So , x=−1 x=−1