Not quite - Howard lost support from the far right at times due to his occasional more moderate stances (gun control, sending troops in to East Timor against Indonesia, organising a referendum to decide on becoming a republic despite being a staunch monarchist etc) but also never got much extra support from the left for refusing to apologise to the Aboriginal people, LGBT rights and so on.
Basically he kept his mouth shut and did his job and seemed to actually know what he was doing (even if we didn't always like it) rather than his contemporary opposition who were changing leaders every few months and Abbot who just seems to be a complete moron and appears ruled by his personal politics.
On a party level Labor were in shambles and never quite recovered from ditching Rudd (though he needed to go) and then the Liberals hammered them on the carbon tax and spending issue so people just looked at them scrapping the tax without reading the fine print of "and everything else too"