Carboxylic acids have an extra oxygen nearby. This oxygen is pulling on the covalent electrons at the OH-group and making it weaker.
Another factor is that the carboxylic acid ion has a resonance form, promoting stability within the molecule. The corresponding carbo-cation that forms when an alcohol loses a hydrogen does not have this resonance form an is highly reactive.