Well, it is minimum in relation to its surrounding.
To make this clearer, imagine two deep trenches on a plane area. Trench A is deep while trench B is not as deep as trench A.
When we've put the ball in Trench B, it'll be at stable equilibrium since it has the height to climb on its side and hence, analogous to increase in potential. As a result, it'll stay at relative lower potential to its surrounding.
Despite trench B being a stable position, it isn't the place with overall lowest potential. In fact, it is a trench A.
So, at a stable equilibrium potential energy is lower compared to the immediate surrounding.