2007 Subaru Outback uneven acceleration when the tank is ne

Ok, I'm afraid this may sound totally retarded, but I am going to post it anyways.

When my fuel tank indicator shows that my fuel is at E (empty) or below, the acceleration from stop to about 3rd gear (haven't really taken it beyond 4th gear) is very uneven and "jerky".

however, once I refill the tank above E, this problem disappears.

I've experienced this maybe 4 times over the past year.

I try not to allow this to happen, but sometimes life is so busy that the tank just gets empty without me noticing.

is this a sign of a serious problem? I have not experienced this with other cars in similar fuel indicator situations.

Am I damaging the engine or other hardware in any way by allowing this to occur?