An Expanded Context May Help
I don't dispute the OP's desire to translate this as "from," and think that is correct. What I think it means in context is that "from Mount Horeb" is being used as figure of speech known as metonymy, where the mountain is substituting for the idea of what happened at the mountain, and so means something to the effect of "from the Mount Horeb failure and chastising they just had."
Context makes this more clear. The OP gave v.6, but in v.3 God declares "for I will not go up in your midst, lest I consume you on the way, for you are a stiff-necked people" (NKJV, bold added) and then the reaction is vv.4-6 (NKJV, except the word in question translated in brackets to from):
4 And when the people heard this bad news, they mourned, and no one put on his ornaments. 5 For the LORD had said to Moses, “Say to the children of Israel, ‘You are a stiff-necked people. I could come up into your midst in one moment and consume you. Now therefore, take off your ornaments, that I may know what to do to you.’ ” 6 So the children of Israel stripped themselves of their ornaments [from] Mount Horeb.
So there is a shame Israel is feeling about what they have done at Mount Horeb, and how that has affected God's relation to them in that He will not be "in the midst" of them the same way He was planning to be before. So the ornaments they were no doubt wearing during the golden calf festivities (Exo 32:5-6), they are commanded to set aside (v.5) and had done so in their mourning (v.4). So "from" (i.e. because of; the preposition min can have a causal idea) Mount Horeb as the place of the experience they had there that they mourn having ever occurred.
Immediately after this, the tabernacle of meeting came into being, which was pitched outside of Israel's tents (v.7) because God was not going to dwell in their midst.
And then Israel sticks around Horeb, Moses gaining new tablets and renewing the covenant (Ch. 34), and freewill offerings for the tabernacle are called for (35:5, 9; which was God's plan for the ornaments before [see Exo 25:1-9] and still as here in ch. 35) and the congregation responds in 35:21-29 (NKJV, bold added):
20 And all the congregation of the children of Israel departed from the presence of Moses. 21 Then everyone came whose heart was stirred, and everyone whose spirit was willing, and they brought the LORD’s offering for the work of the tabernacle of meeting, for all its service, and for the holy garments. 22 They came, both men and women, as many as had a willing heart, and brought earrings and nose rings, rings and necklaces, all jewelry of gold, that is, every man who made an offering of gold to the LORD. 23 And every man, with whom was found blue, purple, and scarlet thread, fine linen, goats’ hair, red skins of rams, and badger skins, brought them. 24 Everyone who offered an offering of silver or bronze brought the LORD’s offering. And everyone with whom was found acacia wood for any work of the service, brought it. 25 All the women who were gifted artisans spun yarn with their hands, and brought what they had spun, of blue, purple, and scarlet, and fine linen. 26 And all the women whose hearts stirred with wisdom spun yarn of goats’ hair. 27 The rulers brought onyx stones, and the stones to be set in the ephod and in the breastplate, 28 and spices and oil for the light, for the anointing oil, and for the sweet incense. 29 The children of Israel brought a freewill offering to the LORD, all the men and women whose hearts were willing to bring material for all kinds of work which the LORD, by the hand of Moses, had commanded to be done.
From these ornaments the majesty of the tabernacle and its instruments was made (Exodus chapters 35-39).
So "from" the Mount Horeb experience of calf worship and the result of God's no longer dwelling in their midst, the ornaments are set aside (by God's command), to be picked up again soon as freewill offerings to adorn the tabernacle. God made beauty from ashes.