Accessory mode will not shut off

My radio stays pm after I open the door. It also drains my battery down. I have removed the fuse for the radio and it has been okay, but I am driving without a radio through. I believe it is stuck in accessory mode. How do I turn the accessory mode of some I can listen to my radio again without having to remove the fuse when I get out the car to insure my battery will not be drained when I go to crank it back up?