The correct option is C |ABA−1−A−1BA|=0
3ABA−1+A=2A−1BA
Adding 2A both sides,
3ABA−1+3A=2A−1BA+2A
⇒3A(BA−1+I)=(A−1B+I)2A
⇒3A(BA−1+AA−1)=(A−1B+A−1A)2A
⇒3A(B+A)A−1=A−1(B+A)2A
Taking determinant both sides,
|3A(B+A)A−1|=|A−1(B+A)2A|
⇒3n|A|⋅|B+A|⋅|A−1|=|A−1|⋅|B+A|⋅2n|A|
⇒3n|B+A|=2n|B+A|
⇒|B+A|=0
3ABA−1+A=2A−1BA
⇒3ABA−1−3A−1BA=−A−A−1BA
⇒3(ABA−1−A−1BA)=−(A+A−1BA)
⇒3(ABA−1−A−1BA)=−(I+A−1B)A
⇒3(ABA−1−A−1BA)=−A−1(A+B)A
Taking determinant both sides,
3n|ABA−1−A−1BA|=(−1)n|A−1||A+B||A|
So, |ABA−1−A−1BA|=0