The correct option is B ex3
Let tan−1y=t
Differentiate both sides, we get
11+y2dydx=dtdx
Now, given equation
dydx+(3x2tan−1y−x3)(1+y2)=0
⇒11+y2dydx+(3x2tan−1y−x3)=0
Substitute the value in above equation.
dtdx+(3x2t−x3)=0
⇒dtdx+3x2t=x3
Therefore, the integrating factor is
IF=e∫3x2dx=ex3