Car starting problems

My car has been sitting in the garage during the winter as "storage". Now I am getting ready to take it out and noticed that the battery ended up going bad, no big surprise. So after getting a new battery I am still having the issue of all the lights including dash and headlights come on but the car immediately shuts off the moment I try to flip it. So after doing some google search I look at the fuses. I didn't own one so took them all to AutoZone so I could test right then and there if I needed a new fuse. They ended up being all tested fine so after putting them all back I decided to do one last attempt for shits and giggles and the car turns on, I try again and still turns on. So the fuses have never been removed, certainly not in the five months of sitting there so I can't believe they were just loose. So is there something else that happened that causes this? It bothers me that I "fixed" the car but don't know how. I will be going back to get a test done on the alternator*, starter* and voltage regulator as well. Please explain this weird situation if you can. Thanks in advance!