The correct option is C R is an equivalence relation
For reflexive,
B=PBP−1
which is true for P=I
(B,B)∈R
∴R is reflexive.
For symmetry,
let (A,B)∈R for matrix P
⇒B=PAP−1
⇒P−1B=P−1PAP−1
⇒P−1BP=IAP−1P=IAI
⇒P−1BP=A
or A=P−1BP
∴(B,A)∈R for matrix P−1
∴R is symmetric.
For transitivity,
let (A,B)∈R for matrix P and (B,C)∈R for matrix Q
⇒B=PAP−1 and C=QBQ−1
⇒C=Q(PAP−1)Q−1
⇒C=(QP)A(P−1Q−1)
⇒C=(QP)A(QP)−1
∴(A,C)∈R for matrix QP
∴R is transitive.
So, R is an equivalence relation.