Question

where does the center of mass of a two particle system lie when one particle is more massive than the other?

Answer 1

Center of mass of a two particle system lies in the reference frame where the net momentum of two particles equals zero. It is on the line connecting two particles and divides the segment in proportion to inverse of their masses.. Relative to center of mass, particles move in the opposite directions with equal but opposite momenta. In the given system it is much closer to the to the heavier particle than to the other.

Denote the two particles as M1 and M2 and say M1>M2 and their separation as x. Let a be the distance of M1 from the center of mass. The center of mass is the point where the moments of the two masses are equal.
Thus M1a = M2(x-a).
Hence M1a + M2a = M2x.
Therefore, a = M2x /(M1 + M2)