Voltage regulator for battery maintainer?

I suspect you're over-thinking this. That maintainer is rated at 1.5 watts, which is real low to start with, and that's only during bright sunlight. Ohm's Law says current is power divided by voltage. 1.5 watts / 12 volts = 0.125 amps, or 125 milliamps.

Due to all the computers on today's vehicles, they draw a little current to keep their memories alive. Same if you have a digital clock. Those are running all the time. The current needed to keep all those memories alive is called "ignition-off-draw", (IOD) current.
Back as far as the late 1980s, Chrysler said the maximum allowable IOD current was 35 milliamps, and starting with a good, fully-charged battery, it would be strong enough to crank the engine fast enough to start after sitting for three weeks. Today that is the industry standard unless specified otherwise by the manufacturer.

I've found my '93 Dynasty will still start after sitting for six weeks. My 2014 Caravan has sat as much as eight weeks and still started, so while that 35 milliamps is a maximum, it doesn't mean every vehicle is drawing that much. Most are closer to 15 to 20 milliamps.
The thing to remember is that IOD current is flowing all the time, while the solar charger's output will rarely be at maximum and for a good two thirds of the day will be nothing. You don't have to worry about it over-charging the battery. Your time would be better spent worrying that it can keep the battery charged.

Until you become more familiar with this charger, you might want to keep an eye on the battery's voltage for a few months. It should remain very close to 12.6 volts. A good battery that is fully discharged will measure around 12.2 volts. If you find it's higher than 12.6 volts, it likely has some "surface charge" that must be removed before you can get an accurate reading. Surface charge is the electrons that haven't made it into the plates yet to be stored there. Draw some current for a few seconds to remove that surface charge. Running the headlights will do it.

Over-charging is not the concern it was in the 1970s and 80s. Normal charging voltage with the engine running is between 13.75 and 14.75 volts. That is needed to force the charging to take place as quickly as possible to get the battery charged back up during numerous short-trip driving. When you go on a long trip, that voltage remains constant, but the battery does not overcharge. They are designed to handle that range of voltage, and in fact, most batteries will tolerate 15.5 to 16 volts without being damaged or boiling the water out of the electrolyte. Your charger is much too small to reach voltages that high, so it's going to be doing a smaller version of the vehicle's charging system if you were on a long drive, but only for the few hours the sunlight is hitting it.

It seems to me this solar charger is sized just about perfectly to keep the battery at it's full charge as long as it gets some sunlight for a few hours per week. That may not happen in November and December up here in Wisconsin. We usually get a bunch of bright days in January. By March I don't care. I'm ready to come out of hibernation.

Can't tell you that. I have two sets of large solar panels, and I can tell you, from talking with the experts in that field, those panels do charge through most cloudy conditions. They say that's because the panels use infra-red rays that penetrate the cloud layer. That would suggest to me your charger would not work under unusual lights if they don't emit that range of rays. You'll have to try it, then you will be my expert. My guess is the panel will charge more than nothing, but not as well as with sunlight.

I have some solar-powered lights that look like little rocks. Those will not turn on after months of sitting very close to an LED bulb I have turned on for more than eight hours each day, but they do work after letting them sit outside in the sun for just a couple of hours.

You shouldn't need a battery maintainer on a lawnmower unless there's a computer on it. The issue would be if it has settings that you can change and set yourself that have to be maintained in memory. Even when that is the case, a lot of electronic equipment uses "EEPROMs", (electrically-erasable programmable read-only memory). We had those in tvs as far back as the late 1990s. The way to tell is if you unplug the tv, even for months or years, when you plug it back in, it will come on right away if it was on when it was unplugged, and it will be on the same channel and volume as it was when last plugged in. No power is needed to maintain that memory. That's also because they put well over 100 service adjustments in one computer chip. You don't want a tv that has to be reprogrammed by the repairman every time you plug it in.

The same is true with your lawnmower. If you need to replace the battery, you don't want to have to reset all kinds of adjustments or settings. When they design products like that, they know a lot of people remove the batteries for the winter.

The point I'm not doing a good job of stating is I suspect there won't be a drain on the battery like there is in cars, but if there is, it's going to be extremely low. I'd start by placing the solar panel closer to a window if possible. Next, if you want to experiment, measure the current the panel is supplying, under various types of lights. Do you know how to use a milliamp meter or multimeter? If you do not, I can draw up a diagram to explain it better. You simply need to break a connection or wire, then insert the meter. Harbor Freight Tools has a perfectly fine little red meter for $7.00. I can help you set it up if necessary.