Passenger Power Windows (Sometimes work and sometimes don't)

Okay, So I have a 1998 Mustang Gt (manual, 4.6L, V8, 150,000 miles) and I have an issue with my (Passenger) window. My passenger window normally works with no issue, but if one of my friends sitting in my passenger seat decides to use the window, it will work for as long as I'm driving car. But once I stop and turn car on again and try to put window down with either driver window button or passenger window button it wont go down anymore. Normally I just wait about 3 days and it just magically starts working on its own again. But now its been a month and my window wont go down anymore. This has been going on for past 2 years since I have had the car so its nothing new but now since it stopped working completely I need to get it fixed. I cant imagine it being a fuse since my driver window works perfectly. I also don't think its the motor because when it wanted to work it went up and down with no problem (no noises, went up and down fast). I'm thinking there might be a short somewhere?