It is definitely very true. It is because the stiffness of a member is dependent on the amount of material present away from the centroid or centre of gravity of the section.
The more the material is concentrated nearer to the centroid, the lesser will be the stiffness and vice versa.
I shall put forth a small example- the rigidity of a member depends mainly on two factors, E and I. E is the young’s modulus and a material property whereas I is a variable and is a geometric property. The I of a rectangular section is bd^3/12 and for a hollow rectangular section it is BD^3/12 - bd^3/12 where the capital letter define the outer dimensions of the rectangle and the small case denotes inner dimensions of the same hollow rectange. Now, you can assume any cross sectional area for the matter of comparison. And you will find out that the I is much higher for a hollow section than for a solid one with the same cross sectional area (and therefore the same weight for the same length).