Is this an O2 sensor problem?

How on earth do you equate this to an oxygen sensor? First you have to determine if the "Battery" warning light is turning on simply because engine speed is dropping too low for the generator to keep charging the battery, or if it's turning on due to a generator failure. If you can increase engine speed and the warning light turns off, the charging system is working and you just need to determine why engine speed is dropping.

Since redesigning their generators for the '87 model year, GM has had a huge problem with them. One of the less-common failures is it locks up due to worn bearings. That will cause the engine to struggle to stay running, the "Battery" light will turn on, and the slipping belt will squeal and or smoke. The severe drag of a locked-up generator will cause slow / hard cranking, often to the point the engine doesn't spin fast enough to start.

The oxygen sensor(s) simply report the composition of the exhaust gas to the Engine Computer so the fuel / air mixture can be fine-tuned. All you'll notice with a sensor problem is the Check Engine light will turn on, and possibly the fuel mileage will drop a little.

Nope, nope, nope! Hold on. I don't want you wasting money on things that might not be needed. It's hard for you to provide enough information over a computer for me to properly diagnose the cause of the problem. I only gave you my best guess based on the symptoms and past experience.

The first thing is to watch the generator's pulley when the engine is running or as a helper is cranking the engine. If the generator is locked up due to a failed bearing, the pulley will not be spinning, and the belt will be sliding over it.

An even faster test is to watch the "Battery" light on the dash. That has to turn on when you turn on the ignition switch, then it must go off once the engine is running. If it does go off, the generator is working and can be eliminated as a suspect.

There's a quick test you can do to tell if the charging system is working, but I don't think we need to look at that, at least at this point. The problem, I believe, is we're looking for the cause of low idle speed. A dead charging system can cause engine running problems, ... And an engine running problem can caused low engine speed which will cause the charging system to stop working. The problem is to figure out which one is causing the other.

If it does come to a need to replace the generator, I'll post a copy of my standard reply to explain why you should replace the battery at the same time.