The three festivals (Pesach, Shavuot, Sukkot) were times when everybody was commanded to assemble in Jerusalem. They were celebrated with festive meals, including some of the meat that had been offered on the altar. (These offerings are listed throughout Exodus, Leviticus, and Numbers.) The rabbis of the talmud understood that Shabbat should also include a meat meal. My impression is that, otherwise, meat was not incredibly common.
According to Isaac Abarbanel (15th century), boiling meat in milk was a pagan practice particularly associated with harvests. From this translation (thanks @msh210):
It seems to me… that idolators would do this when they got together: that is, they'd boil kids in milk when they harvested grain, thinking that they would thereby appeal to their god and get close to him, and he would instill blessing in what they were doing…. And certainly that the shepherds would do so when they gathered to do their thing: their food then was kids boiled in milk and all sorts of cooked meat-and-milk dishes. [...]
I think that really it was for this reason that God warned them that when they gather at Sukos they must not cook a kid in milk as [non-Jews] do. And to distance them utterly from the way of idolatry, He forbade eating it, deriving benefit from it, and cooking it….
I do not know Abarbanel's sources for the historical information on which he draws.
If Abarbanel is correct, then this prohibition would follow logically from a discussion of holidays, paricularly Sukkot (the fall harvest festival). The passage in Ex 34 is near a discussion of all the holidays and Shabbat, perhaps (my speculation here) as a reminder that this is a general prohibition and not Sukkot-specific.
Please note: This answer was written for a neutral, academic audience
and is not intended to be interpreted in the context of a religious
belief or doctrine.