depending on the main god and the type of imperial world you compare it to I'd say 50:50 of the comparisons fall towards chaos beeing the more "comfortable" to live within.

slave labor isnt such an issue if you compare it to the low-hab workers of hive cities. the heresy novels show us that chaos is more about philosophy than actual good&evil. interchange the imperial doctrine with a greek/roman like pantheon of gods and you basically have "chaos" soziety.

it might be true that the extremists usually are crazy *******s but that is true for the imperial believes as well as allmost every religion we know (heck, christions whose god is supposed to be pure love and doesnt grant the champions magical powers at all managed to do abhorrent things like the witch-hutns or crusades in the name of their religion).



all in all I could envision a society based around the 4 gods of chaos to be quite enjoyable for the upper middle class. regular sacrifices in animals (or slaves) to the gods secure you their wellwishings. nurgle could protect you from his "gifts" and those who wouldnt want to die of old age could embrace him for eternal life as a semi-same undead. slanesh would grant exquisite pleasure for flesh and mind in his/her temple. khorn would maybe be the patron of justice, bloody justice that is but what would serve more of a detterent than trowing every criminal into a fighting pit with a khornate "gladiator"?
tzeentch would prolly be the most controverse god. known to the "normal" people as weaver of fate and made sacrifices to to save them from bad luck but fully embraced by high-politicans to guide them in their shemes or scientists/sorcerers to improve their abilities.

I'm not talking deamon world (warp leeching/overblending real-space) but just chaos pantheon absed soziety.