The correct option is B tan−1(tanx−cotx)
Let I=∫dxsin6x+cos6x=∫dx(sin2x)3(cos2x)3
=∫dx(sin2x+cos2x)3−3sin2cos2x(sin2x+cos2x)
∫dx1−3sin2xcos2x=∫sec4xsec4x−3tan2xdx
=∫1+tan2x(1+tan2x)2−3tan3xsec2xdx
Substitute tanx=t⇒sec2xdx=dt
I=∫1+t2(1+t2)2−3t2dt=∫1+t2t2−t2+1dt
=∫1+1t2t2+1t2dt=∫d(t−1t)(t−1t)2+1
=tan−1(t+1t)=tan−1(tanx−cotx)