Front passenger door panel area rattle noise

I am wondering what could be cause of a very noticeable and annoying rattle that seems to be coming from the front passenger door panel area. The only thing I could think of would be a bad clip. The sound does persist over regular road bumps in cold and warm weather. I noticed the rattle does go away when the window is rolled all the way down. When the window is up, the sound seems to go away if I push straight in on the arm rest of that door panel. Almost as if the clip isn’t holding as tight as it should be.