The correct option is
B a2(i+j)
The (x, y) co-ordinates of the masses at O, A, B and C respectively are
(x1=0,y1=0),(x2=a,y2=0),(x3=a,y3=a),(x4=0,y4=a).
Therefore, the (x,y) co-ordinates of the centre of mass are
xCM=a2andyCM=a2. The position vector of the centre of mass is
a2(i+j), which is choice (b)