1992 Mazda MX3 Heater Hot and Cold

Heater problem
1992 Mazda MX3 4 cyl Front Wheel Drive Manual

When I start my car up in the mornings and let it idle for a little while, it gets warm and the heater works for a little bit. Once I've been driving for about 10 minutes, the temp gauge drops and it blows cold. I turn it off, keep driving, and it slowly warms up again. I turn the heater back on, it blows warm air for a few minutes, then turns cold and the temp gauge drops. Even when it's first warmed up, the temp gauge doesn't rise much, maybe a quarter of the way up past cold. When it drops to cold, it is very cold, not just down to the C, but it drops ALL THE WAY down. I have noticed in the afternoon, when it isn't as cold outside that it is a little warmer. The temp gauge raises up a little more than a quarter of the way, but I still have the same problem with the heater.