2005 Toyota Camry Steering "feels" funny when goi

I have a 2005 Toyota Camry and I am having a slight problem with the steering. I'm not sure what it is. There's isn't any noise associated with it, and there's no wobbling or jerking, but it's how it "feels" when it comes up through the steering wheel/column. When I stop and begin to slowly accelerate, or when I'm driving very slowly (as in a parking lot), or when I do the slightest of turns, especially right turns, it's as if something in the drive column or steering wheel has had a delayed clicking over or something is settling in place. It's almost feels like something is tapping the steering column or wheel. This usually only happens at a very slow rate of speed. I feel this in particular when I'm driving through the Mall parking lot to work. I sometimes also feel the "click" come up through the steering wheel /column after I've gone over a speed bump. My Toyota has about 82,000 miles on it and so far has had no major problems. I was told that the power steering drive belt was wearing out and needed to be replaced, could this be the problem?