The correct option is A a2−c2=ab
Given that x2+px+1 is a factor of ax3+bx+c.
Let ax3+bx+c=(x2+px+1)(ax+λ), where λ is a constant.
Then equating the coefficients of line powers of x on both sides, we get
0=ap+λ,b=pλ+a,c=λ
⇒p=−λa=−ca
Hence,
b=(−ca)c+a
⇒ab=a2−c2