Speed will depend on how much you have stretched the rubber. As when you stretch the rubber, you increase its potential energy and when you release the stretch, whole of the potential energy is converted into kinetic energy of the rubber. The velocity of the rubber will determine the momentum of the rubber and rate of change in momentum, when the rubber comes to halt will determine the force which it will apply on the hand when it collides with the hand.
As by newton's second law, the rate of change of momentum is directly proportional to the force applied and this change in momentum is in the direction of the force applied.