The correct option is D (cot1,tan1)
[cot−1x]+2[tan−1x]=0⇒ [cot−1x]=0,[tan−1x]=0
or [cot−1x]=2,[tan−1x]=−1
Now [cot−1x]=0⇒x∈(cot1,∞)
[tan−1x]=0⇒x∈(0,tan1)
Therefore, for
[cot−1x]=[tan−1x]=0,x∈(cot1,tan1)
[cot−1x]=2⇒x∈(cot3,cot2]
[tan−1x]=−1⇒x∈[−tan1,0]⇒No such exist.
Thus, the solution set is (cot1,tan1).