Yes, the impact of removing all the organisms in a trophic level is different for different trophic levels. For example, if plants are removed from a food chain, no organism will get food as plants are the primary producers in the food chain. If herbivores are removed from a food chain, then carnivores will starve and die, and producers are also affected and may die due to competition for space and nutrients.
It is not possible to remove a trophic level without causing damage to the ecosystem as they are all interlinked.