Most Jedi were being hunted for being Jedi. Obi-Wan and Yoda were being hunting specifically, as two known Jedi survivors that were powerful enough to be a direct threat to Vadar and Sideous's persons. They had to hide a lot deeper than the other survivors were.

Also I can't remember if this was explicitly mentioned in the movie or if it was a deleted scene, but they sneak into the Jedi temple because there is a beacon there transmitting a "the war is over, come home!" signal to all Jedi in order to lure in any survivors that thought it was just their clone troopers betraying them, or who happened to be separated from the clone troopers at the time. Yoda and Obi-Wan flip the beacon so it's transmitting "run away and hide!" and presumably any survivors that heard it would have done so. Kind of makes it hard to link up and coordinate though.