Dear student,
Osmosis is the spontaneous net movement of solvent molecules through a selectively permeable membrane into a region of higher solute concentration, in the direction that tends to equalize the solute concentrations on the two sides. It may also be used to describe a physical process in which any solvent moves across a selectively permeable membrane (permeable to the solvent, but not the solute) separating two solutions of different concentrations.
Your statement in the question itself explains the answer. The semi-permiable membrane will only allow the solvent to move from higher concentration to lower concentration. The osmotic pressure of the solvent will be higher in the area where concentration of solvent is higher.
So higher pressure from the more concentrated area will prevent the back flow of solvent from lower concentration area by exerting more force on the membrane and also this will make the membrane to allow movement of solvent from the higher concentrtion area.
NB: In the above explanation, higher concentration area means only considering the concentration of solvent alone. Don't get it confused with concentrations of solute or the solution. Kindly mark this session as satisfied.