The correct option is C (0,−3)
Let the equation of required circle be
x2+y2+2gx+2fy+c=0
Above circle is passing through the points (1,0),(2,−7) and (8,1), so
2g+c=−1 ⋯(1)4g−14f+c=−53 ⋯(2)16g+2f+c=−65 ⋯(3)
Subtracting equation (1) from (2), we get
2g−14f=−52⇒g−7f=−26 ⋯(4)
Subtracting equation (2) from (3), we get
12g+16f=−12⇒3g+4f=−3 ⋯(5)
Using equation (4) and (5), we get
25f=75⇒f=3
Using equation (4), we get
g=−5
Using equation (1), we get
c=9
Therefore, the required equation of the circle is
x2+y2−10x+6y+9=0⇒(x−5)2+(y+3)2=25
By verification, (9,−6) and (0,−3) lie on the circle.
Alternate method :
Let the vertices of triangle be A(1,0),B(2,−7) and C(8,1)
Checking slope of sides
mAB=7−1=−7mBC=−8−6=43mAC=−1−7=17⇒mAB×mAC=−1
So, it is a right angle triangle at A(1,0)
Therefore, BC is a diameter of the circle, so writing circle in diametric form, we get
(x−2)(x−8)+(y+7)(y−1)=0
By verification, (9,−6) and (0,−3) lie on the circle.