3−2cosx−4sinx−cos2x+sin2x=0
⇒3−2cosx−4sinx−(1−2sin2x)+2sinxcosx=0
⇒2−2cosx−4sinx+2sin2x+2sinxcosx=0
⇒2sin2x−4sinx+2+2sinxcosx−2cosx=0
⇒2(sinx−1)2+2cosx(sinx−1)=0
⇒2(sinx−1)[(sinx−1)+cosx]=0
⇒(sinx−1)[sinx+cosx−1]=0
⇒(sinx−1)×√2[1√2sinx+1√2cosx−1√2]=0
⇒(sinx−1)[sinπ4sinx+cosπ4cosx−1√2]=0
⇒(sinx−1)[cos(x−π4)−1√2]=0
⇒(sinx−1)=0 or [cos(x−π4)−1√2]=0
⇒sinx=1 or cos(x−π4)=1√2
⇒sinx=sinπ2 or cos(x−π4)=cosπ4
We know the general solution of sinx=sinα is x=nπ+(−1)na,n∈Z
We know the general solution of cos x=cosα is x=2mπ±α,m∈Z
So, x=nπ+(−1)nπ2,n∈Z and (x−π4)=2mπ±π4∈Z
Taking positive sign, we get
x−π4=2mπ+π4,m∈Z
⇒x=2mπ+π2,m∈Z
Taking negative sign, we get
x−π4=2mπ−π4,m∈Z
⇒x=2mπ,m∈Z
Hence, the general solution is
x=nπ+(−1)nπ2,x=2mπ and x=2mπ+π2,n,m∈Z