No crank no start

Three weeks ago car started not starting when I would park. I had to wait five to ten minutes to get it to start. Finally it stopped completely a week ago. The lights would come on and everything would come on when I turned the key but it would not start. Now after thinking my starter went out, I got under there and touched the two bolts of the starter with my trusty screw driver and it didn't start. Now the car has went dead and I got no power to anything. No lights, no crank, anything. Hooked it up to jumper cables from my brother's truck and got lights..I hit it with the screwdriver (the starter) again and it turned a little bit like it wanted to start but it wouldn't. Got it towed home, now I still have nothing. Changed the ignition switch and got lights again, still no start. Then lights go out again. I noticed that when I turned the key with the lights on and it wouldn't turn on the lights would go dim and when I released the key they would come back on. When my starter went out the lights would just go out and not come back on. I am in need of help. I'm completely confused. I'm thinking I blew a fuse but not sure which one to check. Please help! And thank you in advance.