Let the equations of given curve and straight line are
ax2+2hxy+by2=0 ......(1) and lx+my=1 ....(2)
Let A(x1,y1) and B(x2,y2) be the points of intersection of (1) and (2)
From (2) we have my=1−lx
or y=1−lxm .......(3)
Put the value of y in (3) in (1) we get
ax2+2hx(1−lxm)+b(1−lxm)2=0
⇒am2x2+2hmx−2hmlx2+b+bl2x2−2lbx=0
⇒(am2−2hml+bl2)x2−2(lb−hm)x+b=0 .....(4)
Clearly x1 and x2 are the roots of this equation
∴ Sum of the roots=x1+x2=2(lb−hm)(am2−2hml+bl2) and product of the roots=x1x2=b(am2−2hml+bl2) ......(5)
Eliminating x from (1) and (2) by substituting x=1−myl in (1) we get
(am2−2hml+bl2)y2−2(am−hl)y+a=0 ......(6)
Clearly y1 and y2 are the roots of this equation
∴ Sum of the roots=y1+y2=2(am−hl)(am2−2hml+bl2) and product of the roots=y1y2=a(am2−2hml+bl2) ......(7)
∴ equation of the circle with AB as diameter is
(x−x1)(x−x2)+(y−y1)(y−y2)=0
⇒x2−(x1+x2)x+x1x2+y2−(y1+y2)x+y1y2=0
⇒(x2+y2)−2(lb−hm)x(am2−2hml+bl2)+b(am2−2hml+bl2)−2(am−hl)y(am2−2hml+bl2)+a(am2−2hml+bl2)=0
⇒(x2+y2)(am2−2hml+bl2)−2(lb−hm)x+2(am−hl)y+(a+b)=0 is the required equation.