Commendably, the KJV steers you in the right direction by including the definite article "the" before "north" and "south". This is an interpretation, as the MT could be translated correctly either with or without "the" in this verse.
So, according to the KJV, it is not abstract North and South as directions, but "the north" and "the south", like the way Americans refer to "the North" and "the South", referring to either a geographical or cultural divide.
The question is then what "the north" and what "the south" the verse intends. Is it "the north country", i.e. Aram and the Lebanon, and "the south" country, i.e. Egypt and Kush, or is it north and south something else such as the north and south wind?
The context of preceding verse, 11, (KJV)
The heavens are thine, the earth also is thine: as for the world and the fullness thereof, thou hast founded them.
is the dichotomy between heaven and earth in the first half of the verse. Verse 12 continues with a dichotomy in the first half of the verse applying to the earth, "the north" and "the south". So the intent is "the north" and "the south" ends of the earth. That is, God is the creator from one end to the other, and is not localized to one land as the ancient pagans understood their gods to be localized. This is monotheistic propaganda at its best.
Note that the succession of the imagery progresses from general to specific. In verse 11, "heaven" and "earth", In the first half of verse 12, "the north" and "the south", and in the second half of verse 12, "Hermon" and "Tabor", two mountains in the land of Israel itself.