The correct option is B (π,3π2)
Given : P(x,y) lies in first quadrant
⇒x>0,y>0
for cos−1x to be defined x∈(0,1)⇒cos−1x∈(0,π2)
Now,
cos−1x+cot−1(−1y)+tan−1y
=cos−1x+π+tan−1(−y)+tan−1y
(∵ cot−1(1−y)=π+tan−1(−y); y<0)
=cos−1x+π−tan−1y+tan−1y
=π+cos−1x∈(π,3π2)