The correct option is D cos2ϕ
Multiplying C1 by a, C2 by b and C3 by c, we have
Δ=1abc∣∣
∣
∣∣a3+a(b2+c2)cosϕab2(1−cosϕ)ac2(1−cosϕ)ba2(1−cosϕ)b3+b(c2+a2)cosϕbc2(1−cosϕ)ca2(1−cosϕ)cb2(1−cosϕ)c3+c(a2+b2)cosϕ∣∣
∣
∣∣
Now take a,b and c common fromR1, R2 and R3 respectively, we have:
Δ=abcabc∣∣
∣
∣∣a2+(b2+c2)cosϕb2(1−cosϕ)c2(1−cosϕ)a2(1−cosϕ)b2+(c2+a2)cosϕc2(1−cosϕ)a2(1−cosϕ)b2(1−cosϕ)c2+(a2+b2)cosϕ∣∣
∣
∣∣
Applying C1→C1+C2+C3, we get
Δ=abcabc∣∣
∣
∣∣1b2(1−cosϕ)c2(1−cosϕ)1b2+(c2+a2)cosϕc2(1−cosϕ)1b2(1−cosϕ)c2+(a2+b2)cosϕ∣∣
∣
∣∣
[∴a2+b2+c2=1]
Applying R2→R2−R1,R3→R3−R1
Δ=∣∣
∣
∣∣1b2(1−cosϕ)c2(1−cosϕ)0(a2+b2+c2)cosϕ000(a2+b2+c2)cosϕ∣∣
∣
∣∣
Expanding along C1 we get Δ=∣∣∣cosϕ00cosϕ∣∣∣=cos2ϕ