I'm not really sure about the engine mounts. I've never seen the CJ's. BUT with that said, the blocks are the same. So you could probably reuse your engine mounts.
Honestly, if you are going to drop the engine in, you might consider moving everything up a few inches. This would give you a lot more room for drivetrain. Might be something to take into consideration.
You'll need the engine wiring harness and PCM. You'll want to get the exhaust from the manifold to the down pipe. This has a bung for the O2 sensor. It might be a good time to change over to a serpentine belt if you're still running a V belt. You'd need to get the alternator and PS pump. You might also want the PS box. You could reuse yours but the fittings will probably be different and need adapters.
The engine used the VSS to help calculate the Air/fuel but it will run in default mode, I believe. I'd have to double check that though.
I'd go with an external fuel pump rather then a wrangler tank. The sender and pump are one unit on the wrangler which can be a pain if one goes bad. Also, the tank is rather wide and probably won't fit between your frame rails.
If have to deal with emissions, (even if you don't) it might be good to get the emissions equipment. All that you'd need is the charcoal evap canister under the brake booster. And the lines to the airbox and engine. You'd have to rig up something for the tank but I don't think it would be that difficult.
So, is it worth it? Well, if the engine is cheap, I think so. You'll be gaining around 25hp and MPFI.