I finally fixed the issue. I was replacing the coils and spark plugs with original Ford coils and some NGK iridium plugs. When I was removing the coils, I found the coil on cylinder 2 and 3 were soaked in oil and the spark plug well was full of oil.
Anyway I got a new gasket kit from super cheap and I then removed valve cover. The old gasket was an absolute pain to remove, took me 3 hours. I put everything back, and I removed as much oil as possible from the spark plug wells and tried it out. At first it gave me the same symptoms on first drive, but since then, it’s been perfect. Maybe there was still some oil in there, but now it’s burnt up. I thought I got it all. Anyway it all works now. The past 3 weeks it started doing it every single time I drove the car, and now no longer.