Let I=∫1(x2+1)√x2+2dx
Put x=√2tant⇒dx=√2sec2tdt
I=√2∫sect√2(2tan2t+1)dt=∫sect2tan2t+1dt
Multiplying numerator and denominator by cos2t, we get
I=∫cost2sin2t+cos2tdt=∫costsin2t+1dt
Now put u=sint⇒du=costdt
Therefore
I=∫1u2+1du=tan−1u=tan−1sint
=tan−1sin(tan−1(x√2))=tan−1(x√x2+2)
Hence
∫101(x2+1)√x2+2dx=[tan−1(x√x2+2)]10
=π6−0=π6
⇒k=6