Engine knocking in '99 Sienna

Hello, I have a '99 Sienna mini van with a 3.0L V6 engine. The oil light came on in it last week and the engine started knocking. I just bought the van and the dealer said that he had installed a new oil pump in it.

I put a new oil sending unit on it but still got the light and the knocking. I took it to the Toyota dealer to see if it was full of "sludge", but the mechanic there said it was clean. So yesterday I decide to drive it home from the Toyota dealer and my wife drove it all the way home and there was no oil light or knocking.

I decided to take it out last night, I drove it around for an hour and everything was fine, The engine was as quiet as could be and no oil light came on.

The Toyota dealer told me the engine was bad and I'd need a new/ rebuilt one, but if the engine was bad, wouldn't it be knocking all the time? Something is wrong with it, but I just don't think the engine is shot like the mechanic told me. Possibly a clogged oil pump screen? Why only intermitten knocking?

I appreciate all suggestions and apologize for such a long first post.