Quite difficult to explain, as it was noticed many times that it is not typical of the law codes of the Torah generally to explain the basis for their requirements or prohibitions. So maybe we have to take the risk of an "I think that the significance of leaven in the OT was this" kind of answer.
One very practical explanation:
Daat Zkenim on Leviticus 2:11:1
The reason was that God had said that every offering had to be
accompanied with a certain amount of salt (Leviticus 2:13) and the two
ingredients mentioned here are not compatible with salt.
This, involving salt, would get things even more complicated on a certain level. But let's return to yours':
First of all, yes you are right, we find many times in the OT that leaven or anything containing leaven is not to be served to God. But this is not general. Actually leaven and honey CAN be offered in sacrifice, but not burnt and not associated with blood shedding. Apparently leaven and honey may not be burned onto the altar, however they can be offered as a thanksgiving or peace-offering, when they don't need to be burned (TWOT, 1999, p. 298).
Amos 4:5
And offer a sacrifice of thanksgiving of that which is leavened, And
proclaim freewill-offerings and publish them; For so ye love to do, O
ye children of Israel, Saith the Lord GOD. (JPS Tanakh) See also Leviticus 7:13; 23:17.
D.C. Fleming summarized a traditional explanation saying that leaven and honey were not to be offered on the altar, because of their tendency to spoil foodstuffs, whereas salt preserves things (see above Daat Zkenim on Leviticus 2:11:1). See for instance:
Ibn Ezra on Leviticus 2:11:1
.... leaven the agent that causes bread to rise, honey also a
leavening agent.
I would not 100% buy this explanation, simply because wine was fermented too, and yet it was proper for libations poured onto the altar.
In addition to this, Fleming makes a very interesting observation: burnt offering and blood shedding is about consecration to God, while peace offering is about fellowship with God. (see D.C Fleming, Concise Bible commentary, AMG Publishers,Chattanooga, Tenn., 1994, p. 53).
I would rather suggest to seek an explanation that is tied in specifically with consecration to God vs. fellowship with God. I would focus on the "consecration to God" direction, as this would probably lead to an answer to your question.
Therefore, a connection between the prohibition stated in Leviticus 2:11 (also in Leviticus 6:17; Deut 16:3-4 etc) and the Passover laws is certainly to be assumed. Passover is given an historical and commemorative explanation and it is also about consecrating the people of Israel to God.
Now here the classical explanation would go on saying that when Israel went out of Egypt, bread was made without leaven because there was not enough time to leaven it. Therefore unleavened bread, “the bread of affliction”, is eaten the week after Passover as a celebration, as a commemoration of this event (Deut. 16:3).
Deuteronomy 16:3
You shall eat no leavened bread with it. Seven days you shall eat it
with unleavened bread, the bread of affliction—for you came out of the
land of Egypt in haste—that all the days of your life you may remember
the day when you came out of the land of Egypt. (ESV)
And the teaching would be that having been redeemed from Egypt, Israel should leave its old life quickly and set out toward the promised land by faith (see TWOT, 1999, p. 298). This movement of leaving the old life behind and turning to the promised land can be understood indeed as a consecration, as assuming the call addressed to Abraham in Genesis 12:1 and, together with that, the Covenant:
Genesis 12:1
The Lord told Abram, "You are to leave your land, your relatives, and
your father's house and go to the land that I'm going to show you. ...
(ISV)
It is all about this "leave ... go ..." thing.
Jewish commentators would connect the idea of consecration to God with that of atonement for one's sins, expressed in Exodus as the previous state of slavery in Egypt. Before consecration, there is this need of atonement:
Rabbeinu Bahya, Vayikra 2:11:1-3
According to the plain meaning of our text the purpose of the
offerings is to atone for one’s sins. Had it not been for the evil
urge man would never have committed a sin in the first place. There
would not have been any need for such offerings then. Our sages in
Mechilta Pischa, 8 commening on Deut. 16,3 לא תאכל עליו חמץ, explain
this. The reason that חמץ is not to be eaten at a time when the
Passover lamb can be brought (on the 14th of the month of Nissan after
noon) is that the lamb atones for the idolatrous practices of the
Jewish people prior to the Exodus. God wanted to keep anything which
reminded the Israelites of the evil urge away from them so that they
would not be tempted to return to such practices. Not only chametz,
but also honey is a symbol of the evil urge as the letters of the word
דבש have the same numerical value as the letters of the word אשה,
woman (306). We find that when the first human being, Adam, sinned and
God asked him how this could be, he blamed his wife, i.e. woman as
such saying that “the woman You God have given me gave me of that
fruit and I ate.” (Genesis 3,12). Clearly, he equated woman with the
personification of the evil urge, his temptress. Woman herself said
that she had been seduced by the serpent, the creature embodying the
concept of the evil urge. This is the mystical dimension of the
relationship between woman-the serpent- and the inherent hostility
between the serpent and woman, i.e. man born by woman. This is why both leavened goods and honey have generally been declared as unfit to be the means through which man atones for his sin when bringing a sacrifice. Another reason for this is the well known statement of our sages that “one cannot immerse
oneself in a ritual bath and expect it to cleanse one while holding
the source of one’s ritual impurity, i.e. a dead mouse or such like in
one’s hand” (Taanit 16).
Now I think this would offer a hint for your observation with respect to the fact that the leaven taboo cannot be adequately explained without taking into account the honey prohibition as well. This prohibition may represent indeed a reaction against the widespread use of honey in pagan cults, as indeed honey was frequently offered to pagan gods in the ancient Near East. I would not go into detail with this, it would really take to much (See for instance the Ugaritic epic of Keret, line 72, in H. L. Ginsberg, The Legend of King Keret: A Canaanite Epic of the Bronze Age, American Schools of Oriental Research, New Haven, 1946). So, by prohibiting the use of honey on the altar, the Exodus + priestly laws may have been directed at eliminating pagan practices (see Maimonides, Moreh Nevuchim 3,46). Yet, taking into account the comment above and all its suggestions with respect to דבש / אשה, it is clearly more than that.
No wonder some of the Christian commentators would understand the biblical "honey" from a moral point of view, where honey is read as a sign of sensual pleasure:
Jerome: They quote the passage which says that “the lips of a strange woman drop as honeycomb,” which is sweet indeed in the
eater’s mouth but is afterward found more bitter than gall. This,
they argue, is the reason that neither honey nor wax is offered in the
sacrifices of the Lord, and that oil, the product of the bitter olive,
is burned in his temple.
Jerome, Letter 128.2.4
Jerome: Under no circumstances is there an offering of honey. “Whatever happens,” it says, “will be impure.” Honey is a sign of
pleasure and sweetness, and believe me, sensual pleasure always brings
death; sensuality as such is never pleasing to God. Jerome, Homily
75.5
More or less the same idea, when leaven was also used metaphorically to refer to the heart being soured or embittered by sin
Psalm 73:21
When my soul was embittered, when I was pricked in heart ... (ESV)
For my heart was in a ferment, And I was pricked in my reins ...
(JPSTanakh)
In this respect, leaven and honey share the same meaning, really. It is that that has to be left behind in order to focus on a different direction. It is atonement and consecration, expressed in a symbolical way.
Indeed, leaven and honey have had a symbolic significance to the ancient Israelites, a significance just good enough to make leaven and honey suitable for celebrating fellowship with God. So leaven and honey per se are just fine.
Why then the leaven prohibition in Exodus 23:18 and Leviticus 2:11? Because, from an OT outlook and in a symbolical way, Passover is about atonement and consecration, about leaving something behind (even if you like that thing very much) and about focusing on something even if this something is not looking very promising to you in the present. Not at all far from:
Luke 5:37-38
And no one puts new wine into old wineskins. If he does, the new wine
will burst the skins and it will be spilled, and the skins will be
destroyed. But new wine must be put into fresh wineskins. (ESV)
I am well aware this is not a full answer to your question, yet perhaps it is offering you some ideas to get to a solution. It was too long for a comment, this is why I decided to post it as an answer.