Clicking/knocking sound when foot off accelerator

I started noticing a clicking sound that sounds similar to a socket wrench whenever I am not accelerating. If I lightly tap the accelerator, it goes away. It doesn't start clicking until I accelerate from 0 up to ~10 MPH and then from there, it always clicks until I am stationary. The frequency is directly correlated to the speed with a less frequent click at slower speeds. The click isn't at random intervals either; if I were to stay at a consistent speed, the clicking would stay at the same tempo (although this is practically impossible because my foot isn't on the accelerator). This happens in reverse and neutral as well. I rotated my wheels about two weeks ago because I was patching a flat, but I don't think this has anything to do with that since it's dependent on my acceleration. Any ideas as to what may be causing this?

I've read that it may be the CV joint, but this seems unlikely because it's clicking when going straight, not turning.

I'm not much of a car person, but I do know the basics. That being said, if you could explain in a bit more detail than normal, it would be greatly appreciated.