Lets say, the force between the mass placed at the centre and the mass "m" at the corner A (say topleft) is F = Gm^2/(a/root[2])^2
So the other forces will be 2F, 3F and 4F towards B,C,D respectively (say B is top right) and so on.
So the resultant Force will be 2root(2)F in the downward direction (vertical)
which is 4root(2)Gm^2/a^2 (vertically downwards. {for this configuratoin of masses).