Our skin is porous and behaves as a semi permiable membrane. When we wash clothes, our hands are exposed to water for a long time. Our hands behave as hypertonic solution and the outside of our hands behave as hypotonic solution. Hence, water flows from high concentration to low concentration. The water is not entering your hands, it actually leaves them and hence, they shrink.