We have sin(α−β)=sin(α−θ+θ−β)=sin[(θ−β)−(θ−α)]
=sin(θ−β)cos(θ−α)−cos(θ−β)sin(θ−α)=ba−√1−b2√1−a2
and cos(α−β)=cos(α−θ+θ−β)=cos[(θ−β)−(θ−α)]
=cos(θ−β)cos(θ−α)+sin(θ−β)sin(θ−α)=a√1−b2+b√1−a2
Substituting these values in the given expression, we get
cos2(α−β)+2ab=(a√1−b2+b√1−a2)2+2ab[ba−√1−b2√1−a2]=a2(1−b2)+b2(1−a2)+2ab√1−a2√1−b2+2a2b2−2ab√1−a2√1−b2=a2−b2