I was out there Sunday (and am out in the Borrego area at least monthly), came up out of Split Mountain and went over to Coyote Canyon. A good place for wheelin, lots of decent trails in the park, did a great midnight/full-moon run there recently.
But you're right, on holiday weekends it's packed with every lunatic within 200 miles on quads from PepBoys, wanna be pre-runners, moto-maniacs, $20k sandrails and a bare handful of serious Jeepers. I keep far away from there on busy weekends.
There is a whole world of trails out in Borrego desert. The area south of Ocotillo Wells is great and even on holiday weekends it's pretty lonely in spots and there are several good campgrounds or you can dry camp almost anywhere in the park. Some of the locally produced topo maps have trails on them and will lead you off on some good runs. Well worth exploring.
Let me know when you're down this way again and we can make a run.