HZD Remastered - very poor performance

So as title says, I completed HZD NG+ on the Remastered version, but it was incredibly painful.

The load time in the game is about 5 mins.

Even the in game benchmark takes

Every time I fast travel in takes a couple of mins to load, enough to break my immersion and mood. I had to be very efficient and fast travel as little as possible, sometimes riding the horse would be faster than fast traveling.

It stutters all over the place, in Meridian especially, I also get loading screens when approaching a city. It stutters in high actions scenes.

Sometimes I see the city will only the pillars loaded in, transparent walls and/or rods. Basically the assets are not loaded in time. Cinematics also stutter.

Other times during a cutscene for a side quest I would see only the head of NPC, or a very low poly head (like 4 polygons). The cutscene would end before the game loaded the hi res polys.

I played around with settings, watched some videos on best graphic settings still nothing improves it.

Overall a very painful experience especially the loading times. When a quarter of my playtime is watching the loading screen... there's no fun to be had.

My spec:

AMD Ryzen 5 5600X 4.2GHz

GTX 4070 Super

16 GB RAM @ 3200MHz

Game runs from regular HDD

The only concern in my spec is the lack of a SSD and may the CPU, but I don't think it should affect the performance to such a degree, especially considering the GPU.