The correct option is C hitting
Let’s examine the sentence: “Why don't you try ____ it with a hammer? It might work.” It states the fact of hitting rather than the action of hitting. The ‘fact’ of hitting is a noun or gerund and the action of ‘hitting’ is a verb. The gerund always has the same function as a noun (although it looks like a verb). The gerund ends in ‘ing.’ This rules out Options A, B and D. So, Option C is the correct answer.