Lexus ES300 Shifting Issue

Friday, while going camping for the 4th, I proceeded up a hill, had my car downshift out of overdrive (pulls about 3200 at 60mph). I proceeded up the hill maintaining a constant speed when the car started to slow down like it was losing power and then it upshifted into overdrive. I kicked down on the gas pedal and it downshifted harshly out of overdrive again. Never had a problem with the transmission and we continued driving all weekend (200miles) to camp and around. Coming back up the hill from the other side, I turned the OD off button on to see if that would keep it from upshifting. I was pulling the hill, about 60mph again and after a couple miles (very long hill), it felt like it was losing power and then upshifted again. I kept pushing the OD off button but it wouldn't shift out of OD until I hammered the pedal again and it shifted pretty harshly. I'm thinking maybe a shift selenoid, is there anything else that may cause this. Thanks