Balloon is made up of an elastic material like rubber, latex, nylon, Polychoroprene etc. due to which the ability of balloon to inflate increases and it stretch to become larger than its original size. Also, the pressure inside the balloon of air is higher than outside, which creates a difference between the surrounding air pressure and the pressure inside the balloon.
Molecules of rubber in the balloon are made up of long molecules, i.e. polymers which are linked together and due to the elastic nature of a balloon it inflate up to a certain limit. But, if that limit is crossed, the tension pulls the cross-links of the chains become too great, causing them to break and Pop! Sound appears of the balloon. Basically, it breaks into two large fragments. But if you inflate it until it burst spontaneously, you get dozens of pieces.