Yup, 'cos even if it isn't arounr or aimed at women it is tacit approval of both that behaviour and more directly ape itself if they don't get called out on it. There was a psychological study done on rapists which found that they believe ALL men do it, only unlucky ones get caught and hearing rape jokes feeds into that rape culture. They hear guys making rape jokes and think 'yeah, its normal'. Which isn't to say everyone who makes a rape joke is going to be a rapist but it does help perpetuate rape culture at an individual level, which goes back to what has been said about feminism at an individual level being important.
[URL="http://kedasederragar.tumblr.com/post/96688895932/cisyphus-slurs-are-not-oppressive-because-they"]Saw this on tumblr[/URL], a good way of explaining why slurs and offensive behaviour is not about people having hurt feelings (as anti-PC people often frame it as):
Slurs are not oppressive because they are offensive, they are oppressive because slurs by nature of being slurs draw upon certain power dynamics to remind their target of his/her/their vulnerability in a certain relation to power and as an extension of that, to threaten violence and exploitation of that vulnerability.