What could be making the rear end feel like its swaying from side to side?

Car will be driving smooth but if I hit a rough patch on the road the rear end feels like its moving side to side kinda like the tire is about to fall off. The swaying stops if a slow way down. There is now movement in the stearing wheel.

I had a bad tire one the rear so I got a two new ones. Didnt help