Sodium is a very reactive metal. It is kept in kerosene to prevent it from coming in contact with oxygen and moisture. If this happens, it will react with the moisture present in air and form sodium hydroxide. This is a strongly exothermic reaction, and lot of heat is generated.