Losing pressure in the brake pedal?

At first after sitting for a while the pressure will build back up however after driving for a short period and hitting the brakes a few times the pressure starts to give out until eventually you have to press the pedal all the way to the floor also it will sometimes lose pressure while already stopped and force you to move slightly I have already tried bleeding the brake line however I didn't drain the brake fluid completely and it is a rusty color before I pay for a new master cylinder I was wondering what else could be causing this problem