I'd say keep it white. It hides dings and is very low maintenance. Easy to touch-up/paint match too. Flat black has been 'played out' and would look weird with smooth black. Olive Drab and Tan will fade into an oxidized color after a while. The problem with flat paints is that you can't really do any maintenance on them. Once you start washing it and rubbing on the paint they wills start to gloss. it's also kind of hard to get even coats with a can. You can usually spot transitions and high and low spots unless you shoot it with a HVLP gun to get it evened out.
If I was going to paint it OD or Tan, I would paint all the accessories the same color. There is a OD TJ on the forums someplace, too.
