The correct option is B −2tan−1(12tanx2) when x∈(π,2π)
Let θ=cos−13+5cosx5+3cosx
⇒cosθ=3+5cosx5+3cosx⇒1−tan2θ21+tan2θ2=3+5⎛⎜
⎜⎝1−tan2x21+tan2x2⎞⎟
⎟⎠5+3⎛⎜
⎜⎝1−tan2x21+tan2x2⎞⎟
⎟⎠⇒1−tan2θ21+tan2θ2=4−tan2x24+tan2x2⇒4tan2θ2=tan2x2⇒tanθ2=∣∣∣12tanx2∣∣∣⇒θ=2tan−1∣∣∣12tanx2∣∣∣
When x∈(0,π)∪(2π,3π)
θ=2tan−1(12tanx2)
When x∈(π,2π)∪(3π,4π)
θ=−2tan−1(12tanx2)