1999 Volkswagen Passat Control arm OR ball joint

That's what I should be asking YOU. You're the one with the eyeballs by the car. I can suggest that you typically will not cause or feel movement in a worn ball joint by doing anything to the car other than setting it up properly on a hoist to inspect those parts. The car needs to be supported solidly under the frame or body, not the control arm. The control arm has to be allowed to hang freely so you can cause movement if the part is worn so you can see it. When the car is sitting on the tires, there's too much pressure on the ball joint to allow it to move under hand pressure.

Once supported properly, you can use the wheel and tire as a lever and turn it quickly left and right just a little while a helper watches the ball joint. You're looking for signs of the ball and stud moving sideways inside the socket. No sideways movement is allowed on any ball joint on any vehicle because that will prevent it from holding that wheel in proper alignment.

Next, use a pry bar to pry up and down on the control arm to try to move the ball inside the socket. There are a few vehicles such as the Dodge Dakota that you will find up to 1/8" movement up and down, and that is normal, but there still can't be any sideways movement. When there is only up and down movement, that will not cause an alignment or handling problem but it can cause a clunking or rattling noise.

How involved do you want to get in this inspection? Your best bet is to have the steering and suspension systems inspected at a tire and alignment shop. The typical do-it-yourselfer can learn how to inspect parts, but they typically stop looking once they find the first defective one, and they overlook other, often more serious safety or alignment concerns. You also need to "read" the tire wear patterns to get clues for things to look for. With the mileage on your car, you can expect to find more than just one worn part. The proof is in your comment about handling and steering play. One worn part will rarely cause that on its own.

Control arm bushings have become a common problem. Those may or may not cause clunking and rattling but they will allow the wheel to shift position based on cornering and road forces. Same is true of a worn strut when there's sideways movement between the shaft and body.