We know from the definition of the law of conservation of mass that the total mass of the products must be equal to the total mass of the reactants in any physical or chemical change. But in a nuclear reaction, some the mass gets converted into energy creating an imbalance between the mass of reactants and the products. Therefore, the total mass is not conserved. This is the limitation of the conservation of mass.
The other limitation of conservation of mass is according to Einstein’s theory, the relation between two quantities is given by E = mc2, which means that energy and mass are interconvertible. Therefore, for the law of conservation of mass to be valid mass and energy of the system must be conserved.