2004 Toyota Highlander - A/C fan power level does not adjust correctly

2004 Toyota Highlander, no major issues known with the A/C unit in the past. The A/C works fine, hot and cold air work perfectly, but the fan level does not correctly adjust the level at which the air is blowing. I pulled the knob off and tightened the nut, but this only made it worse. The fan works best when the nut is just slightly loose. When the 'automatic' function is engaged, fan will usually adjust to the highest setting, so the fan WORKS properly, but the adjustment knob does not.

Is this a wiring issue? Since its all working correctly other than that, I'd prefer not to take it in to have somebody charge me $50 to tighten a different nut..