I am trying to find out what kind of fuse I need.

How do you know it's a blown fuse? If you would have found one blown you would buy the same kind and size. If the fuse is not blown, there are plenty of other possible causes for all of the windows to not work. If the fuse is blown, it blew for a reason, and you have to fix that first, otherwise a new fuse will blow too.