The air in a balloon is at a higher pressure than its surroundings because the elastic tension of the balloon skin is pulling inwards. When you stick a pin in the side it creates a tiny hole. The rubber around the edge of the hole isn’t being pulled uniformly in all directions any more because there isn’t any force exerted from the centre of the hole. So the net force pulls the rubber away from the hole, which makes it bigger and the force imbalance increases. In a fraction of a second, the entire skin of the balloon has contracted all the way back to a point on the opposite side from the pin. The high-pressure air that was inside the balloon is now free to expand and this creates a pressure wave that our ears hear as a bang.
If you put a piece of sticky tape on the balloon first and then push the pin through that, the balloon doesn’t go bang. That’s because the sticky tape isn’t under tension and it is strong enough to resist the force from the rubber as it tries to pull back. So the pinhole remains small and the air just leaks out slowly. Likewise, the silvery, Mylar balloons that usually contain helium don’t pop very well because Mylar is a pre-stretched plastic and so it isn’t very elastic.