The correct option is B f(x)=x√1+x2
Given, ∫tan−1xx4dx=−tan−1x3x316x2−13logf(x)+c ....(1)
Now, ∫tan−1xx4dx=∫x−4tan−1xdx
Using Integration by parts, we get
=tan−1x(x−3−3)+13∫x−31+x2dx
=−13x3tan−1x+13∫x−31+x2dx
Now, substitute x=tanθ⇒dx=sec2θdθ
So, =−13x3tan−1x+13∫cot3θdθ
=−13x3tan−1x+13∫cotθ(cosec2θ−1)dθ
=−13x3tan−1x+13∫cotθcosec2θdθ−13∫dθ
Substitute cotθ=u⇒−cosec2θdθ=du
=−13x3tan−1x−13∫udu−13log|sinθ|+C
=−13x3tan−1x−13[u22]−13log∣∣∣x√1+x2∣∣∣+C
=−13x3tan−1x−16tan2θ−13log∣∣∣x√1+x2∣∣∣+C
On comparing with given equation (1), we get
f(x)=x√1+x2