The permanent magnetic moment of the atoms of a material is not zero. The material :
Unlike paramagnetic materials, the atomic moments in ferromagnetic materials exhibit very strong interactions. These interactions are produced by electronic exchange forces and result in a parallel or antiparallel alignment of atomic moments.
This results in permanent non-zero magnetic moment of atoms in these materials.