As we move upward at higher level the pressure of the air drops but our ear do have air at the surface level stored in it that causes the ear drum to push outwards.
This causes pain and loss problem in hearing as well because of the pressure in the eardrum causes difficulty in transmitting sound. so the causes are decrease in pressure and density of the air.