Energy is a scalar quantity because it does not have any direction. Also, the addition and subtraction of the energies are not possible using vector algebra. Therefore, energy is a scalar quantity.
While there is a flow of energy in one direction in a magnet because in a magnet electron in the atoms are responsible for this movement.