The correct option is D must
Modal verbs determine the mood of the sentence, by expressing its meaning, such as permission, obligation, necessity. The given sentence refers to an necessity, the speaker is inviting the listener "you" to have dinner together. From the given options, 'must' expresses necessity. It is necessary for the listener to go to dinner with the speaker. Hence option D is correct. 'Used to' is a modal verb that refers to something that happened in the past, the given sentence does not refer to the past hence option A is incorrect. 'Might' is used to express a possibility in the affirmative sense, it does not express a necessity hence option B is incorrect. 'Shall' and 'must' have similar meanings, but are slightly different in usage. 'Shall' refers to duty, an order or a future event, whereas 'must' refers to an obligation or necessity. Since the sentence refers to an necessity and not a duty or order, option C is incorrect.