The correct option is D to leave
The part of a sentence excluding the subject is the predicate. The verb
in the predicate is often modified by an adverb or its equivalent, known
as the adverbial qualification in that predicate.
In the given sentence, "to leave" is an infinitive. Infinitives can modify verbs, so they can function as adverbs (adverbs modify verbs, adjectives and other adverbs). Hence, it is the adverbial qualification to the verb "wanted" and option D is correct. "We" is the subject, "all" is a determiner and "wanted" is the verb itself that requires an adverbial qualification. Hence options A, B and C are incorrect.