1997 Dodge Royal Monaco pinnonion nut not torqueing ok

A dana 60 does not have a crush sleeve. It has shims behind the bearings as well as races to set the depth. You probably left those out and the nut can't take up the play. If you still have the old parts around, see if you can find the shims for it, otherwise you are going to h ave to take it to a dealer or axle repair center to set everything up correctly.