94 Honda Accord LX check engine light-EGR valve sensor

Computer problem
1994 Honda Accord Automatic 150000 miles

Shortly after I start driving (no more than a minute) the check engine light comes on. Reading the error code tells me that it's the egr valve sensor (code number 12). I checked my egr it seemed to be in good condition, but to make sure I cleaned it out and did the vacuum tests and whatnot and sure enough the light would still come on. I tried unplugging the sensor, figuring that if it were unplugged, it couldn't tell the computer there was a problem. However, the check engine light would still come on.
I thought that the egr doesn't even come on unless you are driving at highway speeds, but when the light comes on, I'm nowhere near highway speeds, so I don't think it's the egr, and I don't think it's the sensor because the light still came on when it was unplugged. This leads me to believe that it's the computer that's at fault, but I don't really know.
Also, the light does NOT seem to come on if I have recently driven the car (maybe an hour or so ago). For example, I drive to the grocery store and the light comes on soon after leaving my house. When I come back from the store the light doesn't come on at all, almost like it has to do with the engine temperature (even though I don't see how that would make any sense)
If I unplug the battery, then later plug it back in, I can drive the car several times before the light comes back on.

Any ideas?