Genesis 24:1-9 (CSB): Abraham was now old, getting on in years, and the Lord had blessed him in everything. 2 Abraham said to his servant, the elder of his household who managed all he owned, “Place your hand under my thigh, 3 and I will have you swear by the Lord, God of heaven and God of earth, that you will not take a wife for my son from the daughters of the Canaanites among whom I live, 4 but will go to my land and my family to take a wife for my son Isaac.”
5 The servant said to him, “Suppose the woman is unwilling to follow me to this land? Should I have your son go back to the land you came from?”
6 Abraham answered him, “Make sure that you don’t take my son back there. 7 The Lord, the God of heaven, who took me from my father’s house and from my native land, who spoke to me and swore to me, ‘I will give this land to your offspring’—he will send his angel before you, and you can take a wife for my son from there. 8 If the woman is unwilling to follow you, then you are free from this oath to me, but don’t let my son go back there.” 9 So the servant placed his hand under his master Abraham’s thigh and swore an oath to him concerning this matter.
Why did it matter so much to Abraham that he made his servant swear an oath not to take Isaac back to Abraham's homeland? Abraham talks about God's promise to Abraham to give him the land, but it's not like Isaac going on a short trip elsewhere would invalidate God's promise. So what did Abraham care so much about this?