Electrical problems

I replaced my headlights with plug and play Xenon and LED's. When the ignition is on the lights work fine, they come on when I use the switch and go off when I use the switch. The problem is when I turn off the ignition the lights come back on.