tr(AAT)=3
Let A=⎡⎢⎣a11a12a13a21a22a23a31a32a33⎤⎥⎦
Then AT=⎡⎢⎣a11a21a31a12a22a32a13a23a33⎤⎥⎦
tr(AAT)=a211+a212+a213+a221+a222+a223+a231+a232+a233=3
So, out of nine elements a′ijs, three elements must be equal to 1 or −1 and rest elements must be 0.
So, the possible cases will be:
For six 0′s and three 1′s, total possibilities is 9C6
For six 0′s , two 1′s and one −1′s, total possibilities is 9C6×3
For six 0′s, one 1′s and two −1′s, total possibilities is 9C6×3
For six 0′s and three −1′s, total possibilities is 9C6
∴ Total number of cases = 9C6×8=672