Mine does the same thing, 2005.
On my 2010 TW, the idle adjustment takes a lot of effort to lower the idle speed. No problem increasing the idle speed, but difficult to lower. It seems like the spring gets caught of in the mounting hardware and it is very difficult to turn the adjustment knob . Wondering if anyone has found a spring that works better?
I think I found the problem with my bike- I didn't put the washer back on the tip of the cable before threading into the carb. Mine seems a lot better now that I added the washer. do you have the spring on yours??