The brightness of an object is determined by the amount of light reflected of it.
Cloth is made up of thousands of microfibers which reflect light. When light falls on these fibers some light is absorbed and the remaining is reflected. This reflected light is what we see.
When a cloth gets wet the gaps between the microfibers gets filled up. Thus increasing the amount of light absorbed and reducing the light reflected.
Hence cloth when wet appears darker.