You are right. Rainbows can occur all over the sky. However the traditional one and two internal reflections of the primary and secondary bows send light back towards the sun and hence their bows appear opposite the sun and centered on the antisolar point. The reflection of the main light makes these bows stand out.
And only the light that enters a droplet is reflected in some manner. Sometimes it is a single reflection and you get a primary rainbow which everyone is familiar with. Depending on how dense the droplets are some light will pass through all the drops and will not be reflected creating a hazy or dim rainbow.
If some of the light bounces inside the droplet (enters but does not exit and bounces inside twice more) you'll see a weaker double rain bow caused the the light that has bounced 2 times.