Rear of car swerves when running over bumps or potholes in the road

I have been noticing for a while now that the rear end of my car swerves when I run over road imperfections like potholes or even poorly removed bumps. In the middle of this experience, I have changed the rear tires because the old ones at the back were worn out and even hydroplane, did wheel-balancing and alignment, the camber was adjusted and I installed new sets of sway bar linkage (because they were missing initially and there is room for them. I think the link point on the control arm broke on the right side, the first owner now decided to remove the sway bar linkage on both sides), after all these, I still feel the swerve. There doesn't seem to be uneven wear on the tires since like 5 months now since the alignment job, but the rear shift/swerve/fishtail/wobble (I don't know what exactly to call it) never stopped. What could be the possible problems and solutions please?