A lot of people put down the 4 cylinder until they actually see one built correctly out-wheel their 6 cylinder. As already mentioned, the weakpoints of the 4 banger is the tranny (some people run them for 200K or more, others break down fairly early in their life; make sure you use the right tranny fluid, and it will become smoother and last longer; I recommend Redline MT90) and the rear axle (if you go to 33" and lock it, get ready for it to go KA-BOOM! In my case, I upgraded to alloys, and I have my fingers crossed, but again, some people wheel them for years without problems, and other just go out fairly early).
In case of the 6 cyl, the rear axle is the same, so there is no improvement there. What you will get is more power and a better transmission, but the 4 angry squirrels have lots of torque down low (you'll notice the difference mainly when driving on the interstate, not when wheeling; and even this can be corrected through gearing), but with less power, you will also have less chance of breaking the D35 on the back...
If given the choice, for the same amount, I would go with a 6, but I am happy with my 4...
Felipe