Welcome to 2CarPros.
What you are describing sounds like an issue with the idle air control valve (IAC). It could also be a vacuum leak or other issue, but the IAC is my first suspect.
Take a look through this link it talks about how to service the IAC.
https://www.2carpros.com/articles/idle-air-control-valve-service
Here is a link that explains how to check for an engine vacuum leak:
https://www.2carpros.com/articles/how-to-use-an-engine-vacuum-gauge
Since you replaced the fuel pump, I will assume you confirmed the fuel pressure is correct and there is no issue with the regulator. Also, I attached a picture below. It shows the IAC on a Honda Accord with the 2.4L 4 cylinder engine. If that isn't correct, let me know.
Let me know if you have questions or what you find.
Joe
Sep 29, 2019 at 6:24 PM