The entire question is premised on a mistranslation and misunderstanding, albeit a rather prevalent one, of Genesis 3:6. Looking more closely at the Hebrew should clear this up.
The most common Hebrew words translated to English as "with" come from two separate roots. Before explaining their respective nuances, let us compare two separate verses from Genesis 3 that both use this same Hebrew word root.
And when the woman saw that the tree was good for food, and that it
was pleasant to the eyes, and a tree to be desired to make one wise,
she took of the fruit thereof, and did eat, and gave also unto her
husband with her; and he did eat. (Genesis 3:6, KJV)
And the man said, The woman whom thou gavest to be with me, she gave
me of the tree, and I did eat. (Genesis 3:12, KJV)
Notice that in verse 12 where this same "with" is used, the application appears to be rather general, and is not applicable to a specific moment in time. Should Eve happen to leave Adam's side for awhile, it would not cease to be true--she was still the one God had given to be "with" Adam. The sense of this word is not one relating specifically to time, nor to physical propinquity (nearness), but more one of occurrence in conjunction with or along with something or someone.
This word for "with" used in Genesis 3 is the word "עִם/`im", which Blue Letter Bible shows HERE. Interestingly, the BLB website seems to give an incomplete attestation of occurrences for this word, because its first entry for it occurs in Genesis 18:23 where Abraham asks: "Wilt thou also destroy the righteous withH5973 the wicked?" Again, this instance of Genesis 18:23 shows the true usage of the word. The righteous in Sodom were in their own home, and were not "with" the wicked in their own homes--they were not beside them, nor associating with them. The word "with" here, as described by my Hebrew teacher, actually implies being at a distance; i.e. not close by, but linked in some manner.
And this word for "with" in Hebrew is far less common than the ordinary "with" we might be thinking of. In Genesis 4:8, Cain talks "with" (KJV) Abel his brother: Hebrew "אֶל/’el" (H413)--in this case it might better be translated as "to". In Genesis 5:22, however, we see an example of the more common "with" that in Hebrew implies closeness, or nearness: "אֶת/et": "And Enoch walked with God after he begat Methuselah three hundred years, and begat sons and daughters."
This "אֶת/et" is not the "with" used in Genesis 3. The one used in Genesis 3 does not imply that Adam was at Eve's side during her temptation. In fact, the story makes no mention of Adam throughout the ordeal until the end of it when Eve takes him some of the fruit. This would be quite surprising, as the question already implies, if Adam had actually been there.
Consider: If Adam had been at the tree with Eve when Eve was deceived, how could it be truthfully stated that Adam was not also deceived?
But, in fact, Adam was not there. Eve had wandered from Adam's side, and was the first to fall--just as Paul indicates. When Adam was presented with the fruit Eve brought him, and heard her tale of how she had obtained it, he was most certainly upset and instantly knew what her fate must be, for God had told them they would die if they ate that fruit. While he was not deceived into eating it, as Eve was, he was induced to join her in her fate, as he could not bear to live without her.
Eve showed her weakness toward the food; Adam showed his weakness toward Eve herself. And these tendencies are still prevalent in society today among our modern Adams and Eves.