Well, even that wouldn't really defeat Chaos in the sense that Cactus is saying. It would go a long way towards making the armies pledged to Chaos easier to hunt down, and thus a less relevant military threat. But most of the Chaos forces that the Imperium faces aren't from a safe haven like that - they're just plain old people who come into conflict with the imperial authority for plain old imperial reasons, and are co-opted by the Ruinous Powers. The Chaos gods themselves don't require a permanent rift in realspace to interact with mortals. If you want to defeat "Chaos" itself, and not just the mortal armies of Chaos, you've got to find a way to un-make the Dark Gods themselves. Which may be possible, but I don't think humanity has ever had the ability to do it.