In Physics point of view :
The answer of this cause lies behind the Newton’s First Law of Motion. Initially, leaves and tree both are in rest. But when the tree is shaken vigorously, tree comes in motion while leaves have tendency to be in rest. Thus, because of remaining in the position of rest some of the leaves may get detached from a tree if we vigorously shake its branch.
In general, the leaves which fall are usually old ones which has a weaker connection with the tree.
Sooner or later, they will get detached from the branches and new ones will take it's place.
Thus, the presence of old leaves are also a factor, as an answer to your question.