The correct option is D x2+y2+12=73x−40y13
Let the variable point be (h,k)
According to the given condition, we get
(h−3)2+(k+2)2=|5h−12k−13|13
Now
Case 1: Point (h,k) and origin lies on the same side of the line 5x−12y−13=0
(h−3)2+(k+2)2=−(5h−12k−13)13⇒13(h2+k2)+169−78h+52k=−5h+12k+13⇒13(h2+k2+12)−73h+40k=0
Therefore, the locus is
x2+y2+12=73x−40y13
Case 2: Point (h,k) and origin lies on opposite sides of the line 5x−12y−13=0
(h−3)2+(k+2)2=(5h−12k−13)13⇒13(h2+k2)+169−78h+52k=5h−12k−13⇒13(h2+k2+14)−83h+64k=0
Therefore, the locus is
x2+y2+14=83x−64y13