The incoming light interacts with the material (the electrons are excited by the electro magnetic field of the light). In every (crystal) layer of the material light is absorbed and re-emitted again with a phase lag.
If the phase lag is not very large (~90 degrees) the material is transparent. The effect of the phase lag is visible by dispersion. The phase lag is relatively larger for short wavelengths (blue) than for longer wavelengths (red). Red light is less affected and will bend less than blue light.