2004 Acura TSX brake and shift

When I turn on my car, to put it in drive or reverse, I have to step into the brake pedal and at the same time move the stick into drive or reverse, the thing is that lately I've been having trouble to move the stick into drive or reverse it wont move, it is moving when I try it like 5 times or even more, and I always step into the break pedal and then I try to move it at the same time as usual, What is the problem? Does it have to be with the brake pedal or the stick.