Why does my engine stall when the engine is cold?

I live in a hot and humid place, so when I start my car, after it's sat and cooled down, my engine will start right up, then the RPMs will reach about a thousand or so, they will drop down to about 500, barely idling, and my lights flickering while it's idling lower than normal. Then the RPMs will spike back up, and the engine runs fine, but then will repeat the process until my engine is warmed up. Once my engine warmed up, it runs and idles fine, that's once my temperature gauge starts to move. With that said, I think the issue is a faulty idle air control valve, but I don't know.