I can’t get closer to my destination

I’m trying to get to Vasagan and but no matter which way I face the sail, the distance between me and my destination keeps growing. I’ve even tried rotating the receiver too. The only thing I can kinda do is paddle but do I really have to paddle 1.8 km continuously while the wind pushes me away?