Let f(x)=x3−ax2−13x+b
Because (x−1) and (x+3) are the factors of f(x)
∴f(1)=0 and f(−3)=0
f(1)=0
⇒(1)3−a(1)2−13(1)+b=0⇒1−a−13+b=0
⇒−a+b=12 ..............(i)
f(−3)=0 ⇒(−3)3−a(−3)2−13(−3)+b=0
⇒−27−9a+39+b=0
⇒−9a+b=−12 .............(ii)
Subtracting equation (ii) from equation (i)
(−a+b)−(−9a+b)=12+12
⇒−a+9a=24⇒8a=24⇒a=3
Put a=3 in equation (i) −3+b=12
⇒b=15. Hence a=3 and b=15