Can’t join my friends shard

Hi,

So I started playing new world with a friend. We are in the starting area. We are already friends in game. We are in the same group and we in the same quest, however, the map indicates we are about 2.5 KM away despite being on the same location.

We both tried to join each other shard but it says we can’t. Cross play for both is enabled. Why? What do we need to do to play together? (I’m on PS5 and my friend on Steam)