The given left hand side determinant is,
Δ=| 1+ a 2 − b 2 2ab −2b 2ab 1− a 2 + b 2 2a 2b −2a 1− a 2 − b 2 |
Apply row operation R 1 → R 1 +b R 3 ,
Δ=| 1+ a 2 − b 2 +2 b 2 2ab−2ab −2b+b−b a 2 − b 3 2ab 1− a 2 + b 2 2a 2b −2a 1− a 2 − b 2 | =| 1+ a 2 + b 2 0 −b( 1+ a 2 + b 2 ) 2ab 1− a 2 + b 2 2a 2b −2a 1− a 2 − b 2 | =( 1+ a 2 + b 2 )| 1 0 −b 2ab 1− a 2 + b 2 2a 2b −2a 1− a 2 − b 2 |
Apply row operation R 2 → R 2 −a R 3 ,
Δ=( 1+ a 2 + b 2 )| 1 0 −b 2ab−2ab 1− a 2 + b 2 +2 a 2 2a−a( 1− a 2 − b 2 ) 2b −2a 1− a 2 − b 2 | =( 1+ a 2 + b 2 )| 1 0 −b 0 1+ b 2 + a 2 a+ a 3 +a b 2 2b −2a 1− a 2 − b 2 | = ( 1+ a 2 + b 2 ) 2 | 1 0 −b 0 1 a 2b −2a 1− a 2 − b 2 |
Expand the determinant along R 1 ,
Δ= ( 1+ a 2 + b 2 ) 2 [ 1− a 2 − b 2 +2 a 2 +2 b 2 ] = ( 1+ a 2 + b 2 ) 2 [ 1+ a 2 + b 2 ] = ( 1+ a 2 + b 2 ) 3
Hence, the left hand side of the determinant is equal to the right hand side.