The correct option is C 5
Given : y=(cot−1x)(cot−1(−x))
⇒y=cot−1x[π−cot−1(x)]
As cot−1(x)>0, (π−cot−1(x))>0
Using A.M. ≥ G.M., we get
cot−1x+(π−cot−1(x))2≥√(cot−1x)[π−cot−1(x)]⇒0<√cot−1(x)(π−cot−1(x))≤π2⇒0<y≤π24⇒a=1, b=4∴a+b=5
Alternate solution
y=(cot−1x)(cot−1(−x))
⇒y=(cot−1x)(π−cot−1(x))
Let t=cot−1x
⇒y=−t2+πt
∵t∈(0,π)
f(0)=0
f(π)=0
and
−D4a=−π2−0−4=π24
Hence, y∈(0,π24]
⇒a=1,b=4
∴a+b=1+4=5