1988 Other Toyota Models difficult to start

Hi there, I am having problems starting my Toyota Masterace after it is left a while. If not used overnight the cranking starts slow and gets faster the longer I leave the key turned for, it takes longer to get cranking with each keyturn. If the car is not used all that I hear is a "click" and it wont crank. I tend to push or roll start it a lot because of this. The battery is 6 months old so I dont think that it is the problem.