It refers to the proximity of electrons in an orbital to the nucleus.
Electrons in different orbitals have different wave-functions and therefore different distributions around the nucleus. Unlike popular belief, penetration is not the outer electron cloud penetrating through the shield of the core electron cloud; but is actually just how well the electrons(outer) feel the nucleus. They are not the same because the core electrons have more penetration than the outer electrons since they (the core electrons) feel the strongest pull.
The electron probability density for s-orbitals is highest in the center of the orbital, or at the nucleus.
In a multi-electron system, the penetration of the nucleus by an electron is measured by the relative electron density near the nucleus of an atom for each shell and subshell of an electron. For example, we see that since a 2s electron has more electron density near the nucleus than a 2p electron, it is penetrating the nucleus of the atom more than the 2p electron. The penetration power of an electron, in a multi-electron atom, is dependent on the values of both the shell and subshell of an electron in an atom.
Therefore, for the same shell value (n) the penetrating power of an electron follows this trend in subshells:
s>p>d>f
And for different values of shell (n) and subshell (l), penetrating power of an electron follows this trend:
1s>2s>2p>3s>3p>4s>3d>4p>5s>4d>5p>6s>4f....
and the energy of an electron for each shell and subshell goes as follows...
1s<2s<2p<3s<3p<4s<3d<4p....