At sea level, the pressure is around 1 atmosphere and at high altitude, the density of air reduces.
Pressure of liquid,
The above equation shows that pressure depends on density. Therefore at 4oC, the density of water will be less at high altitude, compared to the density at sea level.