2011 Toyota Avalon Check Engine Light

Light came on approx. 2 mos. Ago. Had checked at dealer; they said it was because of an O2 sensor. Part had to be ordered. I asked what would happen by driving w/light on - the only thing they could tell me was a slight decrease in gas mileage overall. Cost to replace the sensor was upwards of $300 or $400 (can't remember which). Does this seem crazy? Light went out the next day so I cancelled part order. Light came on next day and has stayed lit. I read somewhere there are 4 of these sensors. Should I, in fact, have the sensor replaced. If so, should I ask that all four be replaced (to save labor costs? It really doesn't seem to have affected gas mileage, which is significantly less than my 2006 Avalon (they tell me all Toyotas had a 10% decrease in gas mileage that year because of added computer "stuff"). I personally find the gas mileage to be 20% decreased - a really significant drop. Anyway, about the light. Can you help? Thanks.