Get it fixed. Your starter is pretty well sealed but oil can sneak in and cause the brushes to crumble. If the leak is really bad the engine may need to be pressure-washed first, then watch to see what gets wet first. If it's a real slow leak you can add a small bottle of dark purple dye, then search later with a black light. The dye will show up as a bright yellow stain that you can follow back to the source.