Yes, the impact of removing all the organisms in a trophic level is different for different trophic levels.
For example, if plants are removed from a food chain, no organism will get food as plants are the primary producers in a food chain.
If herbivores are removed from a food chain, then carnivores will starve and die. In this case, producers are also affected and may die due to competition for space and nutrients.
It is not possible to remove a trophic level without causing damage to the ecosystem as they are all interdependent.