One of the main points being made by Paul here is that the Law is a covenant between God and Israel and was mediated by a man (Moses) whereas the promise to Abraham was made by God and mediated by God alone. Many folks tie this in to the land promise made by God to Abraham in Genesis 15 but that is not strictly accurate:
And he said, Lord GOD, whereby shall I know that I shall inherit it? And he said unto him, Take me an heifer of three years old, and a she goat of three years old, and a ram of three years old, and a turtledove, and a young pigeon. And he took unto him all these, and divided them in the midst, and laid each piece one against another: but the birds divided he not. And when the fowls came down upon the carcases, Abram drove them away. And when the sun was going down, a deep sleep fell upon Abram; and, lo, an horror of great darkness fell upon him. And he said unto Abram, Know of a surety that thy seed shall be a stranger in a land that is not theirs, and shall serve them; and they shall afflict them four hundred years; And also that nation, whom they shall serve, will I judge: and afterward shall they come out with great substance. And thou shalt go to thy fathers in peace; thou shalt be buried in a good old age. But in the fourth generation they shall come hither again: for the iniquity of the Amorites is not yet full. And it came to pass, that, when the sun went down, and it was dark, behold a smoking furnace, and a burning lamp that passed between those pieces. In the same day the LORD made a covenant with Abram - Genesis 15:9-18
Seed in the passage above is a collective singular (functionally plural) and, as Paul later points out in Romans 9:7-8, it is not ultimately the natural descendants of Abraham but the children of promise. Christ is the seed and we are God's children when we are in Christ.
But now notice that your assigned passage takes great pains to announce:
Now the promises were made to Abraham and to his offspring. It does not say, “And to offsprings,” referring to many, but referring to one, “And to your offspring,” who is Christ.
This singular use of seed or offspring rules out the Genesis 15 passage because all of these usages are, contextually, collective singular. The actual singular usage is instead found in Genesis 22:
And the angel of the LORD called unto Abraham out of heaven the second time, And said, By myself have I sworn, saith the LORD, for because thou hast done this thing, and hast not withheld thy son, thine only son: That in blessing I will bless thee, and in multiplying I will multiply thy seed as the stars of the heaven, and as the sand which is upon the sea shore; and thy seed shall possess the gate of his enemies; And in thy seed shall all the nations of the earth be blessed; because thou hast obeyed my voice. - Genesis 22:15-18
Notice three instances of "thy seed" in the above passage. The first instance is plural (collective singular) and refers back to the Genesis 15 promise. The next two instances are actually singular because of it's reference by the singular pronoun and, as your passage in Galatians makes clear, refers to Christ.
There is a grammatical suffix in the Hebrew attached to the word for enemies which refers it back to seed or offspring. So the word "seed" is the same but it is made singular by "his enemies" referring back to seed. Translations that rely heavily on the Septuigant sometimes miss it because it is not carried over into the LXX but it is there in the Hebrew in, for example, the Westminster Leningrad Codex and the Masoretic Text. The link following is particularly helpful because it parses out the text to a high degree and highlights the suffix which you can then find in just about any Hebrew text.
http://www.qbible.com/hebrew-old-testament/genesis/22.html
The grammatical suffix that you see highlighted on the left side in the masoretic text is a personal possessive pronoun. As it appears in the masculine singular form of the verb "hating" (translated enemies) it forces us to refer back to the one who drives out and occupies (rendered possess) whom we know from the context to be the seed.
The inheritance is of promise but the Law, says Paul, was added through a mediator because of transgressions until the coming of the seed to whom the promise was made. A mediator, says Paul referring to the Law, is not a mediator of one but, he says referring to the promised seed, God is one.
Therefore, the promise that the singular seed (Jesus Christ) will possess the gates of his enemies and that all the nations of the earth shall be blessed in Him is God (the Father) mediating a covenant with Himself (the Son).