A/C compressor replacement

My question is regarding my air conditioning. When the AC is on, there is a loud rattling or rumbling noise that constantly occurs. This noise slightly diminishes the longer the car is run but it never fully goes away. The noise is much worse immediately after the car is cranked and the air is turned on. The noise completely goes away whenever the ac is turned off, however. That is what told me that it must be something to do with the AC. Can you help me know what might be causing this problem? Thank you!