I have shoeboxes labelled for Imperial/Mech, Organic (Kroot/Ork/Tyranid/Chaos), and Tau/Eldar bits.
Then I sort bits into small plastic sandwich bags by categories like "Chaos weaponry" or "shoulderpads" and toss them into the appropriate shoebox.
I do a lot of kitbashing, so I've found it handy to sort by basic category rather than by army.