Fair points there Kahoolin, did you ever see the star wars christmas special? Nearly ruined star wars completely for me!
But more on topic, I think that the difference between something like star wars and 40k is that 40k is a game system at its heart, and while the fluff is clearly integral to the game, so is the system and so sometimes conflicts arise because of the system, not because of the fluff.
I mean 40k has changed an awful lot as things went on, and as far as I am aware the biggest changes came because of the game system necessitating major changes, all though im not a massively veteran player or anything so my direct experience is limited to second ed onwards